Weld pool image procession based on the Fourier-DNA low-pass filtering

Abstract

In order to extract the character of weld pool, a new method for weld pool image procession based on the Fourier-DNA low-pass filtering is proposed. Firstly, the TIG welding technique is chosen to be the researched object. The welding experimental system is setup, which is used to capture weld pool images in real time. On this basis, Fourier transformation is performed to weld pool image, and changed the space domain to frequency domain. Then the DNA low-pass filtering is constructed. The DNA intelligent algorithm is used to obtain the optimum low-pass cut-off frequency, and the low-pass filtering can be obtained. The binaryzation operation by the optimum low-pass filtering method and the traditional otsu method have been done. The results show that the method proposed in the paper has a better binaryzation performance. In the end, canny edge extract operation are performed. So the weld pool image edge can be obtained, which has made some basis for the extraction of weld pool character latter on.

Keywords

Weld pool fourier transformation DNA weld pool image procession

1. Introduction

Welding is one kinds of common technique for metal connection, which melts the welding stick, the welding rod, or the workpiece itself. It is widely used in the field of aerospace, railway traffic, shipbuilding, mechanic manufacture, and so on. However, the arc, the splash and the strong smell, which are produced during the welding procession, have made great damage to workers’ health. It has great social and economic benefit to realize the welding automation. How to obtain a beautiful and perfect weld seam has become the researched hot point in the field of automation. In order to realize the goal, the seam tracking and welding forming control should be performed [1].

Seam tracking is to check the hot source can point to the seam during the welding procession. In the field of weld seam tracking, the teaching method has been applied to welding equipment. However, due to the welding fixture error and the heat transformation during the welding procession, the heat source will still deviate the seam, if it moves along the teaching path. To realize the seam tracking, several sensors, such as the mechanic probe, the electromagnetism sensor, the ultrasonic sensor, the arc sensor, the photoelectricity sensor, and so on, are applied [2, 3, 4, 5].

The formation of weld seam has affected the metal structure and its character of the weld seam and the heat affected zone, which determines the mechanical character of final weldment and affect the reliability of production greatly. The change of weld pool surface has been tested to gain the welding penetration condition indirectly.

In [6], the passive vision sensor is used to obtain the pool images of MAG welding. Then the visual characters of weld pool under different penetration conditions were researched, which supplied the experimental data and control model for welding quality control on line. In [7], weld pool images were captured by visual sensor, and the gray-level distribution of pool images for GTAW welding is analyzed. By the information, the images procession algorithm is designed. The boundary of the pool and several geometry parameters can be detected. Then a fuzzy controller is designed. The input of the controller is the error and its changing rate of the weld width on the back. The output of the controller is the current values.

Paper [8] set up a trinocular vision system, and the images information of the front and back pool for GTAW welding can be captured in real time. The pool width and its length on the front of the pool can be obtained, and deduced to the MLD model. By the model, the strategy for GTAW welding penetration control is researched. Paper [9] proposed a new measure method for pool surface by 3d laser vision sensor, the information on the width and depth of pool surface is researched during the dynamic changing of the weld pool surface for GTAW. Yue captured images and temperature information for GMAW by visual and temperature sensors. By the images on the front of the pool and their temperature information, a dynamic analytical three-dimension temperature fields model of weldments can be set up. The model can be used to closed-loop control for weld seam at back of weldment. Cai [10] researched the visual characters detection method around the pool district for GTAW. A controller system for seam forming has been designed based on the detected pool character.

Actually, in the welding procession, experienced workers observe the shape of weld pool and the weld seam information to realize the seam tracking and formation control. It can be speculated that the weld pool image may contain very rich information, which can be used to speculate the seam tracking error and formation condition indirectly. To achieve the goal, enough welding information and the latest intelligent algorithm is necessary [11, 12, 13]. Gao and Na [14], Gao et al. [15], Ding [16, 17, 18] analyzed the weld pool information to construct the welding procession control model. By the setup model, the welding tracking and penetration control can be realized. It is very important to extract the pool information in weld image correctly. Therefore, a new method for weld pool image procession is proposed in this paper, which is based on Fourier and DNA transformation. First, the welding experimental system is setup and can be used to capture weld image in real time. Then the captured images were changed to frequency domain by Fourier transformation. The optimum cut-off frequency can be obtained by the DNA intelligent algorithm. So the optimum low-pass filter can be realized. The image after low-pass filter is changed to space region. The binaryzation procession is performed to the image. In the end, the Canny algorithm is used to extract the edge of weld pool, which made some basis for the obtainment of pool character.

2. Experimental system

The argon tungsten-arc welding (TIG welding) is chosen to be the research welding technique. The welding experimental system is setup, which is shown in Figs 1 and 2. The whole experimental system includes the mechanic body, the CCD visual sensor, the electricity sensor, the industry control computer, the image capture system, the motion control system and so on. The mechanic body is constituted by portal frame and welding workbench, which are driven by step motors. There are several fixtures on the welding workbench, which are used to clamp workpiece. The size of the mechanic body is shown in Table 1.

Table 1
Parameters of mechanic body

Axis	Max travel (mm)	Effective travel (mm)	Screw diameter (mm)	Screw lead (mm)
X axis of table	$\pm$ 100	$\pm$ 75	9.4	1
Y axis of table	$\pm$ 100	$\pm$ 75	9.4	1
X axis of frame	$\pm$ 490	$\pm$ 270	20	25
Y axis of frame	$\pm$ 490	$\pm$ 270	20	25

Figure 1.

Structure of welding experimental system.

Figure 2.

Welding experimental system.

There are several travel switches installed along the axis of the portal frame and welding workbench, which are used to prevent moving over travelling. When welding, the Y axis is set to be the feeding direction. The welding images are captured by CCD sensor, decoded by image capture system, and A/D transformation. Then the digitization images are sent to industry control computer and saved to the hard disk. In order to reduce the arc disturbance producing during the welding procession, there is optical filter installed in front of the CCD camera. So the weld pool images can be obtained.

Based on the experimental system, welding experiments can be performed. The parameters of welding technique is shown in Table 2. In the paper, the argon tungsten-arc welding (TIG welding) is chosen to be the welding technique. The material of size weld workpiece is Q235, and its size is 200 mm $\times$ 150 mm $\times$ 2 mm. The argon is determined to be the protection gas and its flow is 9 L/min. The welding current is set to be 80 A. The welding speed, which is also the feeding speed of Y axis on the working table is set to be 2.5 mm/s. The sample time of CCD camera is 40 ms. So several groups of welding images can be captured.

Table 2

Welding experiment conditions

Weldment Q535 (mm $\times$ mm $\times$ mm)	Argon flow (L/min)	Weld current (A)	Weld speed (mm/s)	Sample time (ms)
200 $\times$ 150 $\times$ 2	9	80	2.50	40

3. Obtainment of weld pool image

In actually welding procession, the light produced by the weld arc is very strong. The weld pool is usually covered by the strong light. In order to acquire an ideal weld pool image, the light disturbance should be filtered. If only use the narrow band filter method, the light can’t be filtered completely. The weld arc light is still very strong, and the weld pool can’t still be distinguished, which is shown in Fig. 3a. If only use the traditional neutral filter method, the contrast of the weld pool image will be decreased vastly, and the detail information of weld pool can’t be distinguished, which is shown in Fig. 3b. In this paper, a composite filter system is setup, which contains a 650 nm narrow band filter and a 7# filter. And a wonderful weld pool image is captured by the visual sensor, which is shown in Fig. 4.

Figure 3.

Images by narrow band filter method and neutral tilter method.

Figure 4.

Image by composite filter system method.

4. Chosen of the weld pool procession district

The chosen of procession district should highlight the information of weld pool and weld seam, and the disturbance of arc light should be avoided. In order to cover most pool information, the whole pool and some weld seam in front of the pool are determined to be the procession district, which is shown in in Fig. 5.

Figure 5.

The weld pool procession district.

5. Fourier transformation of weld pool image

Space zone and frequency zone are two different angles of view to recognize the weld pool image. In space zone, the independent variable ( $x, y$ ) is seem to be a point value of two-dimension space, and the digital image $f(x,y)$ is determined to be a disperse in the two dimension space. In this view of point, if the $f(x,y)$ is seem to be a two dimension signal, whose amplitude change continuously, the Fourier transformation can be performed to the digital image and change the image to frequency zone to made some analyzation. The Fourier transformation of two dimension continuous function can be shown as follow [19, 20]:

$\displaystyle F(u,v)=\int_{-\infty}^{\infty}\int_{-\infty}^{\infty}f(x,y)e^{-j% 2\pi(ux+vy)}dxdy$ (1)

The inverse Fourier transformation of two dimension continuous function is shown as follow:

$\displaystyle f(x,y)=\int_{-\infty}^{\infty}\int_{-\infty}^{\infty}F(u,v)e^{j2% \pi(ux+vy)}dudv$ (2)

For the digital image, its data are discrete gray values.

$\displaystyle F(u,v)=\sum_{x=0}^{M-1}\sum_{y=0}^{N-1}f(x,y)e^{-j2\pi(ux/M+vy/N)}$ (3)

The discrete Fourier inverse transformation can be shown as follow:

$\displaystyle f(x,y)=\frac{1}{MN}\sum_{u=0}^{M-1}\sum_{v=0}^{N-1}F(u,v)e^{j2% \pi(ux/M+vy/N)}$ (4)

In the formula, $x=0,1,2,\ldots,M-1$ , $y=0,1,2,\ldots,N-1$ . The Fourier transformation of weld pool image is shown in Fig. 6.

Figure 6.

Fourier transformation of weld pool image.

6. Frequency filtering of weld pool image

The Fourier transformation can change the image from the space zone to frequency zone. And the inverse Fourier transformation changes the image from the frequency zone to the space zone, which can be recognized by human’s eyes. By the relationship between the space zone image and the frequency zone image, the processing of image can be performed in the frequency zone firstly. After that, the images would be changed back to space zone, so the goal of image enhancement can be reached, which can be shown as follow:

$\displaystyle f(x,y)\ast h(x,y)<==>F(u,v)H(u,v)$ (5)

In the formula, $f(x,y)$ is the image function expression, and $h(x,y)$ is the function expression for treatment of image. $F(u,v)$ and $H(u,v)$ are the Fourier transformation expressions of $f(x,y)$ and $h(x,y)$ . The symbol $<==>$ stands for the Fourier transformation. The detail steps of frequency filtering is shown as follow:

•

Make the Fourier transformation of the original image $f(x,y)$ , and the frequency zone of the image $F(u,v)$ can be obtained.

•

Move the zero frequency position to the center of the spectrogram.

•

Compute the result $G(u,v)$ between $H(u,v)$ and $F(u,v)$ .

•

Move the zero frequency point of $G(u,v)$ back to the top left corner of the spectrogram.

•

Make the inverse Fourier transformation, and the expression $g(x,y)$ in the space zone can be obtained.

•

Get the real part of $g(x,y)$ to be the final filtering result of weld pool image.

From the computing procession above, it can be easily found that the result of the image filtering affection is determined by the filtering function $H(u,v)$ , which can reduce some frequency component in the spectrogram and remain other frequency component of the spectrogram. In the spectrogram of weld pool image, the low frequency components stand for the total gray level distribution condition, and the high frequency components stand for the detail part of the image. In actually application, the lower pass filter and high pass filter are used to get the low frequency component and the high frequency components.

In Fig. 5, it can be found that the process region contains the background, the weld pool, weld seam, and so on. The gray levels in the same object change gently. If use the high pass filter, the inner region will be weakened, which is bad for the extraction of weld pool later on. Therefore, the low pass filter is applied for the pool image procession to object the pool district.

The Fourier transformation of weld pool image is shown in Fig. 6. The center zone of the spectrogram is the low frequency zone. In the zone, the gray changed slowly,which may be the weld pool. And the other zone is the high frequency zone, where the gray changed greatly, which may be noise. So the low pass filter can be used to distinguish the pool and noise. In the operation, the high frequency component can be filtered by setting a cut-off frequency. The frequency component higher than cut-off frequency is set to zero, while the lower frequency is remained the same. The filter result can be shown in Fig. 7. It may be found that the low frequency zone can be seem to be a circle, which is shown in Fig. 7. The diameter $D$ or radius $R$ can be seem to be cut-off frequency. In traditional application, the cut-off frequency is determined by testing, which is time costing and the testing value of the cut-off frequency can’t be made sure to be the optimal. In this paper, the DNA algorithm is used to obtain a optimal value automatically.

Figure 7.

The lower pass filter operation.

Suppose the width of image is $M$ and the height is $N$ , then the formula of low-pass filter can be expressed as follow:

$\displaystyle H(u,v)=\left\{\begin{array}[]{ll}1,&\left[\left(u-\frac{M}{2}% \right)^{2}+\left(v-\frac{N}{2}\right)^{2}\right]\leqslant D\\ 0,&\left[\left(u-\frac{M}{2}\right)^{2}+\left(v-\frac{N}{2}\right)^{2}\right]% \geqslant D\\ \end{array}\right.$ (6)

In the formula, $D$ is the cut-off frequency of low pass filtering.

7. Low pass filter based on the DNA computing

In order to strengthen the weld pool zone, the low pass filter has been employed in the paper. From the Eq. (6), the value of cut-off frequency is very important for the low pass filter. However, in actual application, the value of cut-off frequency is determined by trial and error method, which is very time costing. And the cut-off frequency may not be the optimum value. A new method for the cut-off frequency of the low pass filter is researched in this paper, which is based on the DNA algorithm. The detail procession is as follow [19, 20].

7.1 The coding and decoding principle

The original DNA chain is produced randomly by computers. And the 4 basic groups in DNA chain are firstly defined. T is defined to be 0. C is defined to be 1. A is defined to be 2. G is defined to be 3. The length of DNA chain is 3, and the decoding formula can be shown as following.

$\displaystyle y=a_{3}\times 4^{3}+a_{2}\times 4^{2}+a_{1}\times 4^{1}+a_{0}% \times 4^{0}$ (7)

In the formula, $y$ is the cut-off frequency. $a_{i}$ is the $i$ basic group on DNA chain. The values of $a_{i}$ can be 0, 1, 2 and 3, which strand the basic group T, C, A and G.

7.2 Determination of fitness function

When $u=v=0$ in Eq. (3), $F(0，0)$ stands for the average brightness of the whole image. In Fourier transformation, the brightness of images are set to be variable. And images are expressed as the overlay of a series of sine signal. From this view of point, the Eq. (3) can also be seen to be the sum of energy of all the signal at certain frequency. The brightness of a object must be the same in a image, and the energy should change least. The energy zone between two near object will change greatly. So the maximum energy difference between the adjacent frequency bands is defined to be the fitness function:

$\displaystyle G(u,v)=\textit{max}\left|{F(u,v)-F(u-1,v-1)}\right|$ (8)

In the formula, $u$ and $v$ are the frequency of signal along the horizontal and vertical direction. The expression of function $F(u,v)$ is shown in Eq. (3). In actual application, a circular region is usually set to be the filtering element, whose radius is the cut-off frequency. The frequency in the circular region is set to be 1, while the frequency outside the circular region is set to be 0. Under the situation, $u$ and $v$ are equal. And the Eq. (8) can be expressed as follow.

$\displaystyle G(u)=\textit{max}\left|{F(u)-F(u-1)}\right|$ (9)

7.3 Chosen operation

In the operation, the optimum DNA individuals are chosen from the population in some probability, which can be used to be parents to raise up seed. The new DNA individuals will be put into the next generation and join the DNA genetic manipulation in the next generation. Suppose $d_{i}$ is the DNA chain, and $G(d_{i})$ is its fitness function value. $g$ are the number of offspring, $N$ is the total number of DNA chains. So the number of offspring of the DNA chain can be shown as follow.

$\displaystyle g_{i}=\textit{round}\left(N\times\frac{G(d_{i})}{\sum\limits_{j=% 1}^{N}{G(d_{j})}}\right)$ (10)

7.4 Crossover operation

The crossover operation is shown in Fig. 8. In this operation, a pair of random DNA chain individuals exchange their DNA contents. The crossover position is randomï¼Œwhich is shown in Fig. 8. Then the above DNA chain and below DNA chain exchange their DNA contents.So a new DNA individual, which is at the bottom of Fig. 8, can be produced.

Figure 8.

Crossover operation of DNA.

7.5 Variation operation

In this operation, several DNA chain individuals are firstly chosen. Then the base sequence at some position of the DNA chain variates. T changes to C, and A changes to G. The variation operation is shown in Fig. 9.

Figure 9.

Variation operation of DNA.

7.6 Inversion operation

In this operation, several DNA chain individuals are firstly chosen. Then a random position on the DNA chain is determined and inverts their basic group sequence. The inversion operation is shown in Fig. 10.

Figure 10.

Inversion of DNA.

7.7 Memorize the optimum individual

In this operation, the fitness of new DNA individual would be firstly calculated. If the fitness of the current optimum DNA individual is better than the last generation, the memorize operation will be performed to take place the optimum DNA individual in the last generation. If not, the memorize operation will not be performed, and the optimum DNA individual will be remained.

8. Experimental result

Based on the experimental system shown in Figs 1 and 2, several groups of weld pool images can be captured. The captured time of the CCD is 40 ms, and the captured pool images are shown in Fig. 4. In this paper, the total generation numbers of DNA population for weld pool image gray is set to 50. The rate for variation operation is set to 0.001. So the optimum cut-off frequency can be obtained, which is 40. The results of low pass filter in frequency spectrum are shown in Fig. 11. The Fig. 11a is the Fourier transformation of weld pool image. The Fig. 11b–d is the low pass filter operation of Fig. 11a by the cut-off frequency are 20, 40 and 60. From Fig. 11b–d, it can be found that the remained low frequency zone is become lager with the increase of cut-off frequency.

Figure 11.

Low pass filtering results in frequency spectrum under different cut-off frequency.

Figure 12.

Low pass filtering results in space zone under different cut-off frequency.

Figure 13.

Binaryzation by the new method and the traditional otsu method.

The low pass filtering results in space zone are shown in Fig. 12. In Fig. 12b, the weld pool district has become dim. In Fig. 12d, the weld pool district is more clearer, while the noise around the pool has become clear, too. The Fig. 12c is the low pass filtering result with the optimal cut-off frequency calculated by the DNA algorithm. It can be easily found that the pool district is clear in Fig. 12c and the noise has been reduced at the same time.

After the low-pass filtering operation based on the optimum cut-off frequency, the noise of the pool image can be greatly reduced. In order to extract the feature information in weld pool, binaryzation operations was performed. The binaryzation operation by the new method and traditional otsu method is shown in Fig. 13. From the results shown in Fig. 13a, it can be found that the new method have a wonderful performance for the image than the other operations. The Fig 13b is the result by the traditional otsu operation. It can be found that there are full of burrs around the pool, and there exists some single noise points. Moreover, the disturbance zone is seem to be weld pool, which is in front of the pool. The Fig. 13c is the result by the GA method. In the figure, it can be found that some information of pool image has missed, and some single noise points around the pool appeared.

Figure 14.

Edge detection of weld pool image.

The canny edge extraction is then performed. The edge of the pool has been extracted successfully. Based on the edge, the shape and size can be obtained, and it can be used for seam tracking and welding penetration prediction [14, 15, 16, 17, 18], which has made some basis for welding automation.

9. Conclusion

In this paper, a new low pass filtering method is proposed, which is based on the Fourier-DNA method. Firstly, the Fourier transformation has been performed to the captured weld pool image, and changed the image to frequency zone. Nextly, a DNA low pass filter has been setup. The optimum cut-off frequency has been obtained by the DNA algorithm. From the result, it can be found that the low pass filter, whose cut-off frequency is obtained by the DNA algorithm, has better performance. The weld pool zone has become clearer and the noise around the pool has been greatly reduced at the same time. On this basis, the binaryzation operation is performed to the image and some comparison experiments by the otsu method has been performed. The result showed that the optimal low pass filtering image has better result, which the pool district is gained. However, the binaryzation image by the traditional otsu method is not favorable. Some pool information missed and there exists some single noise points round the pool. In the end, the canny edge detection have been performed. So the edge of weld pool can be obtain, which can be used for seam tracking and welding penetration prediction later on.

Footnotes

Acknowledgments

This work is supported by the major nature science project of Guangdong colleges and universities (No. 2019GZDXM016), Dongguan social science and technology development project (No. 2019507156583), excellent young teachers cultivation project for institution of high learning of Guangdong Province (No. YQ2015232), the skill expert project of Dongguan Polytechnic (No. 2019JY03), the college students’ off-campus practice base poject of the innovation strong school project of Guangdong Province (No. CXZ003) and the project for technical R&D and service team of DongGuan Polytechnic – Intelligent Terminal Product R&D and Technical Service Teamï¼ˆNo. CXTD201802).

References

Yan

Yang

Liu

et al., Overview on ultrasonic-assisted welding and its scientific issues, Journal of Mechanical Engineering 51(24) (2015), 41–49.

Ren

W.J.

, Vision-based Sensing of the We1d Pool Penetration in PAW, M.D. DissertationShangdong, University, 2014.

Lan

and Liu

L.J.

, On-line monitoring of penetration status based on characteristic analysis of arc sound signal in MAG welding, Transaction of The China Welding Institution 31(5) (2010).

J.B.

Hiroyuki

and Yasuo

, Oscillation of molten poolby pulsed assist gas oscillating method and penetration control using peculiar frequency, Journal of High Temperature Society 30(5) (2015), 263–269.

Alfaro

S.C.A.

and Franco

F.D.

, Exploring infrared sensoring for real time welding defects monitoring in GTAW, Journal of Sensors 10 (2010), 5962–5974.

Wang

, Detection of Weld Penetration Based upon Visual Sensing during Arc Welding, M.D. Dissertation, ZhenZhou University, 2012.

Ding

Y.M.

H.B.

and Chen

, Research on simulation method for penetration control of GTAW welding process based on MLD modeling, Journal of Shanghai Jiaotong University 44 (2010), 1–3.

Zhang

Shi

and Li

et al., Research on the correlation between the status of three dimensional weld pool surface and weld penetration in TIG welding, Acta Metallurgica Sinica 50(8) (2014), 95–1002.

Yue

J.F.

L.Y.

and Wang

T.Q.

et al., Penetration control of GMAW based on multi-informational fusion from the front surface, Journal of Mechanical Engineering 45(11) (2009), 283–286.

10.

Cai

, Visual characters extraction of weld pool and shape control during Aluminum Alloy GTAW process, M.D. Dissertation, Shanghai Jiao Tong University, 2013.

11.

Jiang

Zhang

and Zhang

et al., An LSTM-CNN attention approach for aspect-level sentiment classification, Journal of Computational Methods in Sciences and Engineering 19(4) (2019), 859–868.

12.

Guo

Yang

R.F.

and Zhao

L.B.

, The internal parameter calibration based on sub-pixel threshold segment algorithm, Journal of Computational Methods in Sciences and Engineering 18(1) (2018), 3–11.

13.

Chen

Xiao

Q.M.

Matsumoto

Yoshida

et al., Select this result for bulk action Similarity analysis and repeating pattern detection in fingerprint features, Journal of Computational Methods in Sciences and Engineering 18(3) (2018), 553–562.

14.

Gao

X.D.

and Na

S.J.

, Detection of weld position and seam tracking based on Kalman filtering of weld pool images, Journal of Manufacturing Systems 24(1) (2005), 1–12.

15.

Gao

X.D.

Ding

D.K.

Song

Y.W.

and Zhao

, Weld position measuring model based upon weld pool image centroid arithmetic, Transactions of the China Welding Institution 28(3) (2007), 1–4.

16.

Ding

D.K.

, Study on the Visual Model Theory and Experiments for Measurement of the Weld Seam Deviation during the Arc Welding Process, M.D.Dissertation, Guangdong University of Technology, 2007.

17.

Ding

D.K.

, Design of integrated neural network model for weld seam tracking and penetration monitoring, Cluster Computing 20(4) (2017), 3345–3355.

18.

Ding Dukun

D.K.

, Visual neural network model for welding deviation prediction based on weld pool centroid, International Journal of Pattern Recognition and Artificial Intelligence 32(8) (2018), 1–21.

19.

Yang

S.Y.

, Genetic algorithm clustering analysis, in: Pattern recognition and intelligent computing, S.Y. Yang ed., Electronic Industry Press, Beijing, 2015, pp. 298-318.

20.

Kang

, Evolutionary computation, in: Computational intelligence Kang

ed., Beijing: Science press, Beijing, 2016, pp. 3-107.

Weld pool image procession based on the Fourier-DNA low-pass filtering

Abstract

Keywords

1. Introduction

2. Experimental system

Table 1 Parameters of mechanic body

7.1 The coding and decoding principle

8. Experimental result

Footnotes

Acknowledgments

References

Table 1
Parameters of mechanic body