Fusion of multi-voltage digital radiography images based on nonsubsampled contourlet transform

1 Introduction

The quality detection of some important products in the industrial field depends mainly on X-ray testing. There are some complicated components whose structure and appearance are very complex, the thicknesses of them usually change from several millimeters to several hundred millimeters, so the DR images of them have over-exposed or under-exposed areas in normal single or dual energy X-ray testing [1 –3]. The lack of the structural information is very serious, and it makes the degradation of the DR images’ quality. The different DR images are captured at different voltages so as to get the structural information at different thickness region, and then image fusion can be done to these images to get a high quality image.

Figure 1(a) shows the view of a steel engine cylinder head, and Fig. 1(b) shows its DR images of some angle at different voltages. The structure of the product is shown gradually with the increase of voltage, and all images have over-exposed or under-exposed areas. There are many false edges which don’t exist in the component. Image fusion can be done to these images so as to get a high quality image. Figure 1(c) shows the fused result by our proposed method. The structural information at different region can be shown from the fusion image so it is useful to the fast and accurate quality judgement of the product.

At present, the research on the fusion of multi-voltage DR images is very rare. On the research of the dual-energy X-ray radiography, Yang Pei et al. proposed a method based on wavelet transform to fuse the dual-energy DR images [4], Yang Min et al. improved the above method [5]. Yang Ying et al. proposed a method using multi-scale image decomposition to extend the dynamic range of DR equipment [6, 7]. But the DR images as Fig. 1(b) can’t be fused by these above methods directly. There will be disorder of the gray values in the fusion image, and the false edges will be strengthened. The fused result by the proposed method in [5] is shown in Fig. 1(d). The massive false edges are strengthened and the gray values are disordered, so it cannot see the structural information of the original product.

On the fusion of the multi-voltage DR images, Duan Yanjie [1, 3] and Chen Fanglin [2] proposed an image enhancement method, but they must adjust the inversion of the gray values depending on prior knowledge, so the method can’t be applied in the engineering application. The method proposed by [8] is based on the wavelet transform and independent component analysis (ICA). The wavelet transform is sensitive to the edges, so if it is used to fuse the DR images shown in Fig. 1(b), the false edges will be enhanced in the fusion image. Wei Jiaotong et al. proposed a method based on principle component analysis [9] and a method based on gray consistency [10] for multi-voltage DR images. A comparatively clearer and more accurate fused result will be got with these two methods. But a same weight will be assigned to the whole DR image at every voltage in these two methods, so the influence of the under- or over-exposed areas to the fused result will be the same as the influence of the well-exposed areas. So the structure of the component is incomplete or the details are unclear in the fused result.

The nonsubsampled contourlet transform (NSCT) [11] is a flexible multi-resolution, multi-direction and shift-invariant transform, so it can eliminate the pseudo-Gibbs phenomena around singularities. As no sampled, all subband images have the same size as the original image, so the false edges of the multi-voltage DR images can be easily eliminated. So we choose NSCT as the multi-resolution decomposition tool. In order to avoid the disorder of the gray values in the fusion image, the low-frequency subbands are fused by PCA. The modified central energy role is used to carry out the high-frequency directional subbands fusion, in order to increase the clarity of the details.

2 The Nonsubsampled contourlet transform (NSCT)

NSCT is proposed based on the contourlet transform [12]. Compared with the contourlet transform, NSCT eliminates the downsampling and upsampling in the process of the decomposition and reconstruction, so it is a flexible multi-scale, multi-direction and shift-invariant transform.

Similar to the contourlet transform, NSCT separates the multi-scale decomposition and the multi-direction decomposition. First, NSCT uses the Nonsubsampled Pyramids (NSP) to obtain the multi-scale decomposition. Then, it uses the Nonsubsampled Directional Filters Banks (NSDFB) to obtain the multi-direction decomposition. So the subbands of multi-scale and multi-direction can be got. Figure 2(a) displays an overview of the NSCT. The structure consists in a bank of filters that splits the 2-D frequency plane in the subbands illustrated in Fig. 2(b).

NSCT uses the equivalent shift nature of Z transform, removing the downsamplers after analyzing and upsamplers before integrated filtering in the Laplacian pyramid and in the Directional Filters Banks, and then upsampling the filters accordingly. So NSCT has the characteristic of shift- invariant and the subbands have the same size as the source image.

If applying NSDFB to make l levels decomposition of the subbands under one scale, the result images of 2 ^l directional subbands are the same size as the source image. So the NSCT has a redundancy of 1 + ∑ j = 1 J 2 l j, where J is the number of decomposition stages, l _j denotes the number of levels in the NSDFB at the jth scale.

3 Fusion method

The main steps of the proposed multi-voltage DR images fusion method are as follows:

Step 1: Decompose the source DR images with NSCT, and get the low-frequency subbands and the high-frequency directional subbands in multi-scale and multi-direction;

Step 2: Fuse the low-frequency subbands with the Principle Component Analysis (PCA);

Step 3: Fuse the high-frequency directional subbands with the role of the modified central energy;

Step 4: Apply the inverse NSCT on the attained coefficients to get the final fused image.

Perform a J-level NSCT on the source DR images, and attain the NSCT coefficients { C j 0 i ( m , n ) , C j , l i ( m , n ) } ( i = 1 , 2 , ⋯ , N ; j ≥ j 0 ), where C j 0 i ( m , n ) is the low-frequency subband coefficients of the i-th source image, and C j , l i ( m , n ) is the high-frequency directional subband coefficients at the j-th scale and on the l-th direction of the i-th source image, and N is the number of the source images.

A. Fusion of low-frequency coefficients

The low-frequency part mainly reflects the approximate and average characters of the source image, so the fusion of the low-frequency information must keep the global characteristics of the fusion image, and avoid the disorder of the gray values. The edges and the contours will be obscured by simple averaging, and the role of the local variance proposed in [5] will cause the disorder of the gray values, so the low-frequency coefficients will be fused by the PCA role proposed in [9].

Rearrange each image of the low-frequency coefficients of C j 0 i ( m , n ) ( i = 1 , 2 , ⋯ , N ) into a row vector, denoted as x _i = (x _i1, x _i2, ⋯ , x _iM ) (i = 1, 2, ⋯ , N), where M is the pixel numbers of each source image. Let X = [ x 11 x 12 ⋯ x 1 M x 21 x 22 ⋯ x 2 M ⋮ ⋮ ⋱ ⋮ x N 1 x N 2 ⋯ x NM ], and seek the eigenvalues of the covariance matrix of X, denote as λ _i (i = 1, 2, ⋯ , N). Then, sort the eigenvalues from big to small, that is λ ₁ ≥ λ ₂ ≥ ⋯ ≥ λ _N . The corresponding orthogonal and normalized eigenvectors are e _i = (e _i1, e _i2, ⋯ , e _iN ) ^T (i = 1, 2, ⋯ , N). Compute the principle components according to e _i :

PF i = e i T X = e i 1 x 1 + e i 2 x 2 + ⋯ + e iN x N(1)

Take the sum of the first n principle components as the low-frequency approximate image of the fusion result: C j 0 F = PF 1 + ⋯ + PF n ( n = 1 , 2 , ⋯ , N )(2)

B. The preprocessing of the false edges

In order to reduce the influence of the false edges in the fusion image, the high-frequency subband coefficients of the false edges must be reduced. As the size of the subband images is the same as the source image in NSCT, so the false edges of the source images can be determined first, ant then process them in the fusion of the image. Figure 1(b) shows that the false edges appear around the saturation areas except for the DR image in the lowest voltage. So we consider the edges of the saturation areas as the false edges.

The saturation areas can be defined as follow (to the 12-bits detector): A i ( m , n ) = { 1 , I i ( m , n ) ≥ 4090 0 , otherwise ( i = 2 , ⋯ , N )(3)

where I _i (m, n) is the gray value of the pixel (m, n) in the i-th source image to be fused.

Then the edges of the saturation areas can be got: FS i ( m , n ) = A i ( m , n ) - ( A i ( m , n ) Θ B )(4)

where B is a 5 × 5 structuring element, Θ is the erosion operator.

Define a mask that can reduce the influence of the false edges according to FS _i : T i ( m , n ) = { 0.0001 , FS i ( m , n ) = = 1 1 , otherwise ( i = 1 , 2 , ⋯ , N )(5)

where FS _i (m, n) is the value of the pixel (m, n) in the i-th saturation edge image.

C. Fusion of high-frequency subband coefficients

Multiply the high-frequency subband images with the mask T _i before the fusion of the high-frequency subband coefficients: HC j , l i ( m , n ) = C j , l i ( m , n ) × T i ( m , n ) ( i = 1 , 2 , ⋯ , N )(6)

To the fusion of the high-frequency subband images, the energy of each pixel will be computed in a window of 3 × 3. In order to increase the influence of the central pixel [13, 14], the modified central energy role is used to carry out the high-frequency directional subbands fusion.

To the HC j , l i ( m , n ) got by Equation (6), the modified central energy of the pixel (m, n) is: E j , l i ( m , n ) = ∑ p ∑ q W ( p , q ) [ HC j , l i ( m , n ) ] 2(7) where W ( p , q ) = 1 15 ( 1 2 1 2 3 2 1 2 1 ).

So the fused high-frequency subband coefficient of the pixel (m, n) at the j-th scale and on the l-th direction is: C j , l F ( m , n ) = { HC j , l t ( m , n ) ; when E j , l t ( m , n ) is the only biggest one , t ∈ ( 1 , 2 , ⋯ , N ) 1 N ∑ i = 1 N HC j , l i ( m , n ) ; others(8)

In the experiments, the decomposition level of NSCT is 3, and the direction numbers of each scale are (1, 2, 8) respectively. The ‘maxflat’ filter is used for multi-scale decomposition, and the ‘dmaxflat7’ for multi-direction decomposition. In order to testify the effectiveness of the proposed method, the first tested workpiece is a stepped block made of steel. The thicknesses of the stepped block are 1mm, 1.5mm, 2mm, 4mm, 6mm and 8mm. Some bent wires are stuck to the bottom of it. The diameters of the wires are 0.4mm, 0.6mm and 0.8mm. The different DR images are captured first at different voltages. The X-ray generator is 450kV, and the A/D detector PaxScan2520 is 12-bits. The tube current is 1.5mA. The tube voltages are 80kV, 90kV, 100kV, 110kV, 120kV, 130kV, 140kV, 150kV, 160kV, 170kV, 175kV, 180kV, 185kV and 190kV. All the 14 images will be used to get the fusion image. Figure 3 shows only parts of the DR images.

We can see from Fig. 3 that the workpiece can’t be penetrated effectively in the single or dual energy X-ray testing. When the voltage is low, the wire in the thinner areas can be seen clearly, but the detail information in the thicker areas is missing. When the voltage is higher, the detail information in the thicker areas increases, but the thinner areas become saturated since the X-ray energy is too large.

Fuse the 14 DR images of the stepped block by the proposed method. In the fusion of the low-frequency subband coefficients, the first principle component is selected as the fused low-frequency coefficients, that is, C j 0 F = PF 1. The fusion image by our proposed method is shown in Fig. 4(a). The fusion image by the method in [2] is shown in Fig. 4(b), and the fusion image by the method in [8] is shown in Fig. 4(c).

Compared with the source DR images, the whole structural information can be seen in the fused results by the above methods, and there is no disorder of the gray values in the results. Compared with the Fig. 4(a) and (b), the detail of (c) is not clear, and the bent wire is obscure. The difference of Fig. (a) and (b) is very small. The structural information is whole, and the bent wire can be seen clearly in these two fused results. But it needs prior knowledge to adjust the disorder of the gray values in the method proposed in [2]. If the X-ray system is changed, the prior knowledge is correspondingly changed. So, it is not suitable for the practical work. The proposed method needn’t any prior knowledge, and there is no disorder of the gray values, so it is effective in the multi-voltage DR images fusion.

In order to further testify the effectiveness of the proposed method, fuse the DR images of an electronic password lock by the proposed method. The DR images of the electronic password lock are shown in Fig. 5. There is plastic shell, wires and composite socket outside the workpiece, and the core part of it is made of the steel. The single or dual energy X-ray can’t penetrate it effectively.

Fuse the 9 DR images of the electronic password lock by the proposed method. In the fusion of the low-frequency subband coefficients, the first three principle components are selected as the fusedlow-frequency coefficients, that is, C j 0 F = PF 1 + PF 2 + PF 3. The fusion image is shown in Fig. 6(a). In order to compare the fused results, Fig. 6(b) shows the fusion image by the method in [9], and Fig. 6(c) shows the fusion image by the method in [10].

The whole structure can be seen in the fused results by the above three methods, and there is no disorder in all results. Although the high-frequency components are strengthened in our proposed method, yet the high-frequency coefficients of the false edges are reduced, so there aren’t so many edges in the fusion image. Compared with Fig. 6(a) and (b), the outside part of Fig. 6(c), for example the plastic shell, can be seen clearly. But the core steel part is obscure. The plastic shell of the Fig. 6(a) is obscurer than the Fig. 6(c), but the inside structure of it is much clearer than that of in the Fig. 6(c). Overall, the fused result by our proposed method has the best visual effect.

In order to quantitatively analyse different fusion images, we evaluate the effect of the fused results of the electronic password lock shown in Fig. 6. To the images in the same gray-level range, we often use the entropy, the spatial frequency and the variance to evaluate the effect of the fusion images [15]. But the gray-level range of the fusion images shown in Fig. 6 are all extended, and the ranges are different. The gray histogram of the different fusion images of Fig. 6 is shown in Fig. 7. The max gray value of the fused results by our proposed method and the method proposed in [9] is more than 16000, and the max gray value of the fusion image by the method in [10] is more than 12000, so the gray-level range of the three fusion images is different. There are no comparability of the entropy, the spatial frequency and the variance to the three fusion images. When observing an image, we often pay more attention to the local contrast, so we compute the local index with the window of 5 × 5. The entropy can measure the amount of the information content of the image, and the variance and the spatial frequency can describe the detail information and the texture features of the image. The variance and the spatial frequency are sensitive to the edges of an image. We can see from the DR images of the electronic password lock shown in Fig. 5 that there are many false edges in the DR images. If there are many false edges in a fusion image, then the variance and the spatial frequency increase correspondingly. So the variance and the spatial frequency are not suitable to evaluate the effect of the multi-voltage DR fusion image. We select the local entropy [16] as the index to evaluate the effect of different fusion image, and the size of the window is 5 × 5. The local entropy of different fusion images in Fig. 6 is listed inTable 1.

We can see from Table 1 that the local entropy of the fusion image by our proposed method is maximal. So the fusion image by our proposed method has the most information content, and the proposed method has the best effect. It is in accord with the subjective evaluation.

5 Conclusions

Aiming at the fusion of multi-voltage DR images, a method based on the nonsubsampled contourlet transform is proposed. Apply the method to the X-ray testing of the complicated industrial components, the whole structural information can be seen clearly in the fusion image. Compared with the other methods, the proposed method needn’t any prior knowledge, and there is no disorder of gray values in the fused result. The fused DR image brings more detailed information, and the structure of the component can be seen clearly, so it is useful to the fast and accurate quality judgements of thecomponent.