Abstract
In the work, the Contourlet transform is modified for a more complex situation in order to obtain a subtler decomposition of the warp-knitted fabric image. The new Contourlet transform is named the non-subsampled wavelet-packet-based Contourlet transform (NWPCT) and it consists of wavelet-packet transform and a non-subsampled directional filter bank. Firstly, the fabric image is processed by means of wavelet-packet transform with segmented threshold de-noising to acquire the subtle frequency coefficients. Secondly, the more elaborate directional coefficients will be obtained by decomposing the wavelet-packet coefficients with non-subsampled directional filter bank. Then the directional coefficients with higher energy are chosen to reconstruct the wavelet-packet coefficients. Finally, the iterative threshold method and object operation based on morphology are applied to segment the defect profile. The final experimental result demonstrates that NWPCT has excellent properties to segment out the defects (broken wrap, oil and width barrier). The defect profile is distinct enough for the further work concerning warp-knitted fabric defect recognition.
Keywords
Although the fabric defect artificial detect still exists in less developed areas of the textile industry, the fabric defect automation detect goes with the trend of modern textile industrial intelligent production and it is being a hot spot because of its high efficiency and accurateness in most areas. 1 Over the years many methods have been introduced into the field of the fabric defect detect and they are roughly divided into two variants, frequency domain methods: wavelet transform (applied by Tsai and Chiang 2 ), Gabor transform (applied by Kumar and Pang Grantham 3 ) and Fourier transform (proposed by Chan and Pang Grantham 4 ); and spatial domain methods: Markov random field (present by Cohen Fernand et al. 5 ), gray-level co-occurrence matrix (proposed by Shimizu 6 ) and so on. However, almost all of the research is focused on the areas of woven fabric defect detection; work in warp-knitted fabric defect detection is uncommon. In order to meet the needs of the growing warp-knitted production, a valid way for warp-knitted defect automatic detection is in demand. Nevertheless, the main methods mentioned above are powerless for the more accurate profile segmentation. So it is indispensable to introduce a new one.
Different from the normal image, the information of the warp-knitted fabric image is concentrated on the low-frequency and intermediate-frequency coefficients. The common methods just act on the low-frequency coefficients, so the Contourlet transform is suitable for this matter. In this paper, the improved Contourlet transform, the non-subsampled wavelet-packet-based Contourlet transform (NWPCT), is proposed to apply in the segmentation of warp-knitted fabric defect detection. The NWPCT makes up for the disadvantages of the Contourlet transform, as it can work out the more positive effect of defect segmentation. In the NWPCT, the wavelet-packet transform takes the place of the LP in the Contourlet transform and the directional filter bank (DFB) is reformed to a non-subsampled DFB. By means of the NWPCT, this work segments the defect, such as the broken warp, oil and width barrier, out of the warp-knitted fabric image while taking the loop structure of the fabric into account. Consequently, the results indicate that the NWPCT has an excellent profile holding property.
Theory of the Contourlet transform
The Contourlet transform is one of the multiresolution geometrical image transforms that has the capacity of subtle decomposition in the image process, both angularly and radially.7–9 While the image contains a great deal of information regarding high-dimensional singular geometrical characteristics, such as texture, profile and edge, the methods mentioned above (Fourier transform, wavelet transform and so on) are out of use. However, the Contourlet transform can solve this matter. It is widely acknowledged that the Contourlet transform can acquire more directional information and extract more geometric features, so it has occupied a significant role in the area of image processing in recent years.
Traditionally, the Contourlet transform is composed of a Laplacian Pyramid (LP) and DFB (presented by Smith and Bamberger
10
). Shown in Figure 1, it demonstrates the operating theory of the Contourlet transform. The LP decomposes the input image into low-frequency and high-frequency coefficients
11
and after that the high-frequency coefficient is decomposed through the DFB angularly and radially. Finally, the directional coefficients will be obtained.
The operating theory of the Contourlet transform.
The LP consists of a decomposition filter, synthesis filter, down-sampled matrix and up-sampled matrix. The input image is filtered by a decomposition filter and down-sampled matrix first and the low-frequency coefficient will be formed. Then the low frequency is filtered by means of a synthesis filter and is up-sampled; a prediction image, which is an approximate image, will be obtained. Eventually, the high-frequency coefficient can be acquired, which is the D-value of the original image and prediction image.
After getting the high-frequency coefficient, the DFB will decompose it into several sub-bands at every level.
12
The traditional DFB consists of the two-band fan filter bank and the sampled matrices. Shown as the following equations, they are respectively sampled matrices Q1 and Q2 and the sampled unimodular matrices R1, R2, R3 and R4:
The high-frequency spectrum is filtered by the two-band fan filter bank in the horizontal and vertical directions, while the sampled matrices are used for down-sampling the frequency spectrum. Theoretically, the down-sampled operation of matrices Q1 and Q2 can change the data sampling rate and rearrange the filtered result, but the down-sampled operation of unimodular matrices R1, R2, R3 and R4 are merely used for result rearrangement. Overall, the operation procedure is as presented in Figure 2; the fan filter bank is cascaded with the down-sampled operation of these sampled matrices.
The decomposition steps of the directional filter bank.
Non-subsampled wavelet-packet-based Contourlet transform
Wavelet-packet transform
The Contourlet transform has undergone great development over the years. Previous research of the Contourlet transform had emphasized the DFB to the Contourlet transform, although some studies improved the LP of the Contourlet transform. Much of the research in the Contourlet transform in the last two decades has displayed that its overall effect is no better than wavelet transform or others, while it has a positive profile holding property. The reasons behind this matter can be summarized by two points: the LP and the DFB.
As mentioned above, the LP decomposes the input image into low-frequency and high-frequency parts. Seen in Figure 1, level 1 is the input original image and at the last level is the approximate coefficient. With the decomposition procedure, the resolution of the original image will gradually decrease. For instance, if an input image is of size 2
J
*2
J
(J = log2N), with the decomposition of K + 1 levels, the resolution will decrease into 2
J-K
*2
J-K
. As the following equation demonstrates, as the decomposition goes on, the length and width of the approximate image will be halved relative to the last level image and that means the size of the approximate image will be a quarter of the last one. Adding the sizes of these approximate images up, the total pixels of K + 1 levels can be obtained:
It can be seen clearly that the final calculated pixels during the decomposition processing are
Wavelet transform is a good choice to substitute for the LP and it will not cause the redundancy mentioned. However, wavelet transform is limited for more flexible decomposition, because the traditional wavelet transform just decomposes the low-frequency band. However, the fabric image also contains much information in other frequency bands. 14 The wavelet-packet transform15–17 is the extension of the wavelet transform and it overcomes the disadvantages of the wavelet transform. It can decompose all frequency bands flexibly and extract the features more roundly.
Shown in Figure 3 are the wavelet-packet parse tree of three levels and the corresponding wavelet-packet decomposition. The schematic diagram shows that the input image will be decomposed into four coefficients (approximate coefficient C, horizontal coefficient H, vertical coefficient V and diagonal coefficient D); every coefficient will be sequentially decomposed into four coefficients (such as CC, CH, CV and CD).
The schematic diagram of wavelet-packet transform: (a) wavelet-packet parse tree; (b) wavelet-packet decomposition.
Similarly, the decomposition procedure can be expressed in the same manner as the wavelet transform. For instance, Wf is one of the wavelet subspaces regarding a one-dimensional (1D) input signal W. Its decomposition is displayed in the following equations:
Here, Uf is defined as wavelet-packet subspaces, f is the scale parameter and k is a positive integer (
Moreover, the wavelet subspace is decomposed by means of the wavelet library. The library L is made up as formula (5) shows. This means that they are the wavelet functions around 2-ft, in which order of magnitudes regarding the support is 2-f and the number of oscillations is n; f is defined as the scale parameter and t is the translation value at the x-axis:
In summary, the decomposition algorithm is expressed as formula (6) and the reconstruction algorithm is as formula (7) shows. In the formulas,
Non-subsampled directional filter bank
In order to acquire more subtle sub-bands and to hold much more geometrical information, the DFB is indispensable. Although Eslami and Radha 18 20 improved the Contourlet transform through the wavelet transform to remove the redundancy, the actual effect is not promising. The reason behind this matter is image artifacts the DFB produced.
In fact, the traditional DFB with down-sampling operation gives rise to translation invariance being destroyed, which will result in image artifacts. The artifacts are obvious, especially when the dramatic changes of the gray level exist in the image. As is known, in the fabric image the gray level changes rapidly when the defect appears and that means the image artifacts undoubtedly will be introduced into the defect segmentation. So, it is necessary to get into the down-sampling operation seriously. 19
The following equations show the down-sampling operation.
Commonly, the determinant of the sampled matrix M is 2. Consequently, Equation (9) can be expressed like equation (10):
From Equation (10), it is clearly found that The down-sampling operation.
If the down-sampling operation is removed, the matter can be solved. In this paper, the non-subsampled DFB (NDFB) is constructed in which the down-sampling operation is substituted for the up-sampling operation. The up-sampling operation can be expressed as in the following equations.
21
Here,
Figure 5 represents the NDFB this paper constructs. In the NDFB, there are three kinds of filters named fan filters, checkboard filters and parallelogram filters. The fan filters are formed through modulating the two-band filters by row. The checkboard filters will be constructed when the fan filters are up-sampled by Q1 and Q2. Cascading these two kinds of filters can achieve the four-band decomposition. However, the eight-band decomposition will be formed through the parallelogram filters. It is composed by means of up-sampling the fan filters with unimodular matrices R1, R2, R3 and R4.
The decomposition steps of the non-subsampled directional filter bank.
Eventually, the new Contourlet transform (NWPCT) is formed, which consists of wavelet-packet transform and the non-subsampled DFB. The NWPCT overcomes the drawbacks of the Contourlet transform and it can get subtler sub-bands and reduces the loss of information. Shown in Figure 6 is the schematic diagram of the NWPCT.
The schematic diagram of the non-subsampled wavelet-packet-based Contourlet transform.
Defect segmentation
Decomposition
During the decomposition and reconstruction, the fundamental wavelet for wavelet-packet transform and the NDFB should be considered. As mentioned above, when the defect occurs in the warp-knitted fabric, the gray level of the fabric image will change dramatically. That means there will be peak-shaped bulge in the spatial distribution of the gray level. Accordingly, a fundamental wavelet with a peak shape is suitable for decomposition and reconstruction. The bior2.8 is one of biorthogonal wavelets that meets the demand mentioned above. Figure 7 shows parts of the bior2.8 wavelet-packet function applied in the wavelet-packet transform.
The fundamental wave bior2.8.
Taking consideration of the biorthogonality of bior2.8, the fundamental wavelet for the NDFB can be decided easily. The series of pkva wavelet functions is suitable for the NDFB. In this work, the concept of information entropy is introduced in order to select the best one for the NDFB. The calculation formula of information entropy is shown as follows. Here, T is the value of information entropy and In is the occurrence probability of gray level n:
As is known, the larger the information entropy, the more information the sub-band coefficients contain. In this paper, the fundamental wavelets pkva, pkva6 and pkva8 are compared to decompose three kinds of warp-knitted fabric detection images (broken wrap, oil and width barrier) into four directional sub-bands. The experiment result demonstrates that the information entropy is largest when the fundamental wavelet for NDFB is pkva6. This means that pkva6 is the most suitable one for the NDFB.
Furthermore, in the decomposition of the wavelet-packet transform, the best tree selection
22
and sub-bands de-noising should be taken into account. Although wavelet-packet transform is more flexible than wavelet transform, the calculated amount of it is larger. With regard to the three-scale wavelet-packet tree, there are a total of 83,533 kinds of potential decomposition. The way to choose the best one is indispensable. Traditionally, the best tree selection is based on entropy. This kind of method is simple and efficient. This work defines the square of the norm as the energy of every node. The equation is as follows. Here, Ei is the energy of the node coefficient and
For instance, EP is the parent node energy and EC, EH, EV, ED are the child node energies. If the total energy of these child nodes is less than their parent node, the child nodes will be retained in the wavelet-packet parse tree. If not, they are removed from the tree. Shown in Figure 8(a) is the schematic diagram of the wavelet-packet best parse tree and Figure 8(b) shows the corresponding decomposition.
The schematic diagram of wavelet-packet best decomposition: (a) wavelet-packet best parse tree; (b) wavelet-packet best decomposition.
However the sub-band coefficients acquired are noisy and need to be de-noised. In this work, the segmented threshold de-noising is applied. The sub-band coefficients are classified as the signal coefficient, transition coefficient and noise coefficient based on energy. The calculation formula of energy is as shown in Equation (14).
In this paper, the four coefficients that have the lowest energy are considered as the noise coefficient and they will be de-noised by the Sqtwolog rule. This method has a strong property of de-noising and the formula of the threshold is as follows. Here, Tn is the threshold of the Sqtwolog rule, N is the number of total sub-band coefficients and σ is noise signal deviation.
Then the mean value EM of these four coefficients of energy is calculated. When anyone of the rest of the coefficients is larger than
Finally, the remaining coefficients are treated as the transition coefficient and they should be de-noised by the Minimaxi threshold rule. The equation is shown below:
In general, the segmented threshold de-noising can improve the reliability of de-noising and retain the effective information as far as possible. Figure 9(a) displays a warp-knitted fabric (256*256) with broken warp and Figure 9(b) is the result after the wavelet-packet transform (the best tree selection and segmented threshold de-noising).
The wavelet-packet decomposition of the warp-knitted fabric with broken warp: (a) the warp-knitted fabric with broken warp; (b) the result after wavelet-packet decomposition.
Eventually, these de-noised wavelet-packet coefficients will be decomposed by the NDFB and the subtler directional sub-band coefficients will obtained. Shown in Figure 10 are the eight-band directional coefficients of one of the wavelet-packet coefficients.
The illustration of eight directional sub-band coefficients.
Reconstruction
The directional sub-band coefficients are formed after the decomposition of the NWPCT. However, not all of the directional sub-band coefficients are of value to reconstruct the wavelet-packet coefficients, and some of them are useless and unwanted. So the inutile coefficients should be removed in order to reconstruct the expected wavelet-packet coefficients. This paper selects the valuable directional sub-band coefficients based on energy. The calculated formula of coefficient energy is shown in Equation (14).
Regarding different levels of wavelet-packet coefficients, the number of directional sub-band coefficients is different. As shown in Figure 6, there are eight sub-band coefficients at the third level of wavelet-packet coefficients and four coefficients concerning the second level. For example, when the wavelet-packet coefficient is decomposed into eight-band by the NDFB, the selection rule is as follows. Calculate the mean value of the energy of these eight sub-band coefficients. If the energy of any one of the directional sub-band coefficients is smaller than the mean value, it will be removed. The remaining coefficients are the ones wanted to reconstruct the wavelet-packet coefficient. Figure 11 shows the schematic diagram of directional sub-band coefficients selection. The black regions are the chosen coefficients and they will recompose the wavelet-packet coefficients by means of the inverse NDFB. Then the new formed wavelet-packet coefficients will be used to reconstruct the image through inverse wavelet-packet transform.
The schematic diagram of energy selection.
Referred to as the theory of wavelet-packet transform and non-subsampled DFB, the procedure of the reconstruction is just the inverse process of decomposition and here no longer expatiatory in this work. Consequently, Figure 12 displays the recomposed image of Figure 9(a) and it retains most of the effective information.
The reconstruction of the broken warp.
Iterative threshold segmentation and morphological operation
In this paper, the iterative threshold segmentation is applied and it is simple and effective to segment the reconstructed image. Although this method is traditional, it has excellent segmentation ability when the image waiting for the process is of high quality, which means the target object is distinguished form the background in the image. The reconstructed image acquired in this paper meets the demand. The theory of iterative threshold segmentation is just about obtaining the best threshold to extract the target object. The following shows the formula of the threshold:
Here, T1 is the first threshold this work decided. The result after iterative threshold segmentation.
However, it is clearly found that in Figure 13 there are a great number of intermittent miscellaneous points. Figure 14 explains the reasons behind the points. According to the definition of the warp-knitted fabric, it is the fabric that makes the yarn bent to loop and draws the new loop through the old loop.
23
As shown, it will form a tiny hole in the fabric. This kind of hole gives rise to the dramatic gray level change like the defect. Therefore, in the final segmented image, it will be filled with a great deal of unpromising points.
The loop structure of warp-knitted fabric.
It is indispensable to get rid of the miscellaneous points. This work introduces the morphological operation24,25 for processing. In the segmented image, the point will be removed when the area of it is smaller than the set value. However, the area of the unpromising points is far smaller than the object. Consequently, they can be rejected easily. Then the rest of the object has to be processed by morphological opening in order to smooth the profile. It is widely acknowledged that the theory of morphological opening is image erosion and dilation, which can smooth the large object while erasing the tiny ones. Eventually, all of the miscellaneous points are removed and the profile of the object is held distinctly. Figure 15 shows the final result after iterative threshold segmentation and morphological operation.
The final binary image.
Results and discussion
In summary, the NWPCT can achieve the best segmentation effect when the fundamental wavelet for the wavelet-packet transform and NDFB is bior2.8 and pkva6, respectively. The NWPCT can hold most of the warp-knitted fabric defect profile information and, as Figure 15 shows, the segmented defect is extremely similar to the real defect profile. This means that the result can be used for the next warp-knitted fabric defect automatic identification. Figures 16 and 17 display the results of the oil and width barrier. As shown, although the defect of the width barrier is smaller than the other defects, the NWPCT still can achieve a subtle effect.
The results of the oil: (a) original; (b) reconstructed; (c) result. The results of the width barrier: (a) original; (b) reconstructed; (c) result.

Conclusion
This paper substitutes the wavelet-packet transform for the LP of the Contourlet transform to avoid redundancy of the calculating process and introduces the best tree and segmented threshold de-noising into the wavelet-packet transform. It is necessary to improve the DFB into the non-subsampled DFB in order to get rid of the image artifacts. Consequently, the NWPCT is constructed and it is applied in the segmentation of warp-knitted fabric defect for the first time.
After the decomposition of the NWPCT, the input fabric image is decomposed into several groups of directional sub-band coefficients. The wavelet-packet coefficients are recomposed through selecting the superior directional sub-band coefficients based on energy. The image is reconstructed by means of the inverse wavelet-packet transform. Then the segmented result is obtained through the simple iterative threshold segmentation and morphological operation. It can be found that the final segmented defect profile is quite similar compared with the real input fabric image, and in the meantime the background of the segmented image is noiseless. In general, the segmented result is good enough so that it can be applied for the further research, such as warp-knitted fabric defect automatic inspection and warp-knitted fabric defect type judgment. However, it is also crucial to enhance the computational efficiency for the further research.
Footnotes
Declaration of conflicting interests
The authors declared no potential conflicts of interest with respect to the research, authorship and/or publication of this article.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the National Science Foundation of China (No.11302085), the Fundamental Research Funds for the Central Universities (JUSRP51404A) and the Innovation fund project of Cooperation among Industries, Universities & Research Institutes of Jiangsu Province (No. BY2014023-34 and BY2014023-20).
