Image inspection of knitted fabric defects using wavelet packets

Abstract

Image inspection by wavelet packets and a neural network classifier is presented for non-defect and six kinds of defects in knitted fabrics. The types of defect include a hole, set mark (coarse), dropped stitch, oil stain, streak, and tight end. In this study, wavelet packet decomposition of a sample image is carried out based on the best-basis wavelet packet tree with three resolution levels. The lowest-two entropy among all sub-band images and the standard deviation for the original image are selected as feature inputs of the neural network classifier. These textural features are shown in seven groups, which are separately distributed in the feature space. We gathered a total of 112 experimental samples, with 16 samples in each of the seven aforementioned categories. The results demonstrate that with the three features, 56 test samples are correctly inspected. However, the lack of one of the three features yields wrong classification of some samples. Therefore, the three features selected are definitely suitable for recognition of our knitted fabric defects and also are the smallest number of features required to give accurate inspection.

Keywords

image inspection wavelet packet neural network knitted fabric defect

Defect inspection of textile products is one of the steps for quality assurance. Knitted fabrics are one of the widely used textile products, and their defects, such as holes, set marks (coarse), dropped stitches, oil stains, streaks, and tight ends, are frequently encountered. Hence, inspection of these defects is helpful to enhance knitted fabric qualities. Currently, the defects are inspected by human inspectors, a tedious and fatiguing work when a great deal of inspection is to be done, and suffering from inconsistent outcomes owing to the subjective criteria of inspectors. Moreover, incorrect judgments are often made due to poor mental and physical conditions. Image inspection provides accuracy, consistency, repeatability, and a low-cost solution to the drawbacks associated with human inspection.¹

In the past, much effort has been devoted to defect inspection of woven fabrics. Chen et al.² selected 93 feature values from power spectra and a back-propagation neural network to classify 12 fabric defects. Tsai et al.³ used a neural network classifier for defect inspection of neps, broken ends, broken picks, and oil stains with features of six contrast measures. Chan⁴ extracted seven significant characteristic parameters from the central spatial frequency spectrums for defect classification. The fabric defects include double yarn, missing yarn, broken fabric, and yarn density variation. Kim et al.⁵ calculated the signal-to-noise ratio based on the results of wavelet transform to detect four defects of different styles of fabric.

Feature extraction is a crucial step in pattern recognition.⁶ The wavelet packets have been shown to be useful for extraction of textural features and have been successfully used in texture classification. Laine and Fan⁷ computed energy and entropy matrices for each wavelet packet, and employed a neural network and a minimum-distance classifier to classify 25 natural textures. Pun and Lee⁸ proposed a two-stage wavelet packet feature approach, involving extraction of the dominant wavelet features in the first stage and wavelet packet features from the polarized form of the sample image in the second stage, for rotation-invariant texture classification of 20 textures. Lee and Pun⁹ used dominant energy features from wavelet packet decomposition and a Mahalanobis distance classifier to classify 20 classes of natural textures. The good texture classification capabilities enable the wavelet packets to be applicable to defect inspection in various areas. Kumar and Pang¹ used the values of the shift invariant measures and their location on the best-basis wavelet packet tree as features to classify defect and defect-free fabrics. Hu and Tsai¹⁰ selected the values and positions for the smallest-six entropy in a wavelet packet best tree as the feature inputs of the neural network to identify missing ends, missing picks, oily fabric, and broken fabric. To inspect defects of cold rolled stripes, Lee et al.¹¹ developed an adaptive wavelet packet scheme to automatically determine the optimal quadtree and extract a small number of features for each sub-band.

In this study, we decomposed a sample image based on three resolution levels of the best-basis wavelet packet tree. The wavelet packet decomposition generated a large number of wavelet packet coefficients at every resolution level, exhibiting different information content of the original image in sub-bands. Since the shift-variant coefficients are not suitable for direct use, shift-invariant measures, such as the standard deviation and entropy for each channel, can be used as textural features.¹ Accordingly, to reduce the feature set, we selected the lowest-two entropy among the decomposed sub-images, together with the standard deviation of the original image, as features. Moreover, we adopted the back-propagation neural network as a classifier because it has shown good classification abilities in textile products.^12,13

In previously published papers,^10,14 the authors mainly focused on the identification of four common kinds of fabric defects. However, wavelet packet transform was used to inspect six kinds of fabric defects (namely holes, set marks, dropped stitches, oil stains, streaks, and tight ends) with knitted fabric and standard knitted fabric. The wavelet packet transform decomposes an image into smooth sub-images and detailed sub-images. In addition, the features (lowest and second lowest entropy) of entropy from the sub-images and standard deviation from the original image were extracted for effective and precise classifications.

In this paper, a high performance in defect inspection of knitted fabrics can be achieved with the smaller feature set, and the necessity of all three features to yield accurate classification is experimentally justified.

Wavelet packets

Multi-resolution representations are very effective for analyzing the information content of images.¹⁵ The wavelet packets, a generalization of multi-resolution decomposition, recursively decompose not only the low-frequency components but also the high-frequency components. The wavelet packet bases are used to obtain multi-resolution decomposition of a signal. A sequence of functions ${W_{n}}$ ( $n = 0, 1, 2, \dots$ ) can be generated from a given function $W_{0} (x)$ as below:¹⁶

W_{2 n} (x) = \sqrt{2} \sum_{k} h (k) W_{n} (2 x - k)

(1)

W_{2 n + 1} (x) = \sqrt{2} \sum_{k} g (k) W_{n} (2 x - k)

(2)

where

W_{0} (x)

can be identified with the scaling function

ϕ (x)

and

W_{1} (x)

becomes the mother wavelet

ψ (x)

. h(k) is the impulse response of a low-pass filter satisfying the following orthonormal conditions:⁸

\sum_{n} h (n - 2 k) h (n - 2 l) = δ_{kl}

(3)

\sum_{n} h (n) = \sqrt{2}

(4)

and g(k) is the impulse response of a high-pass filter with

g (k) = (- 1) k h (1 - k)

, which are called quadrature mirror filters (QMFs).⁶ The library of the wavelet packet bases is the collection of orthonormal basis functions of the form

2 j / 2 W_{n} (2 j x - k)

, where the indices j, k, and n are the scaling, dilation, and oscillation parameters, respectively.

The wavelet packet decomposition of a signal f(x) generates the wavelet packet coefficients simply by the inner products of f(x) with distinct wavelet packet bases $á f (x), 2 j / 2 W_{n} (2 j x - k) ñ$ . For discrete signal sequences $c_{k}^{0}$ , $k \in Z$ , the coefficients at any resolution level can be efficiently computed by the following recursive algorithms:¹⁷

c_{m}^{p} = \sum_{k} h (k - 2 m) c_{k}^{p - 1}

(5)

d_{m}^{p} = \sum_{k} g (k - 2 m) c_{k}^{p - 1} m \in Z

(6)

where the sequences

{c_{m}^{p}}

and

{d_{m}^{p}}

are the low-frequency approximation and high-frequency detail, respectively, at resolution level p. This decomposition process can be viewed as passing the signal

c_{k}^{p - 1}

through the QMFs and down-sampling the filtered signal by two.¹⁸

A set of two-dimensional (2D) wavelet packet basis functions can be expressed by the tensor product of two one-dimensional (1D) wavelet packet basis functions in the horizontal and vertical directions. The corresponding 2D filters can be obtained as follows:⁸

h_{LL} (k, l) = h (k) h (l) h_{LH} (k, l) = h (k) g (l) h_{HH} (k, l) = g (k) g (l) h_{HL} (k, l) = g (k) h (l)

(7)

The four filters can decompose an image into four sub-images that contain the low-frequency approximation and high-frequency details in the horizontal, vertical, and diagonal directions. This decomposition process is recursively applied to each sub-image, thus generating more and more sub-images of smaller size as the resolution level increases. As a demonstration, the original image of $N \times N$ pixels can be decomposed into four $\frac{N}{2} \times \frac{N}{2}$ sub-images, W(1,0), W(1,1), W(1,2), and W(1,3), at resolution level 1, as shown in Figure 1(a). W(1,0) is the low-frequency component and W(1,1), W(1,2), and W(1,3) are high-frequency components in the horizontal, vertical, and diagonal directions, respectively. At resolution level 2, the sub-image W(1,0) is further decomposed into four sub-images W(2,0), W(2,1), W(2,2), and W(2,3) and each of the other three sub-images is decomposed in the same way, as displayed in Figure 1(b). The sub-images at resolution 2 are of $\frac{N}{22} \times \frac{N}{22}$ pixels. These sub-images not only provide multiple looks of the original image in different frequency bands, but also require less time for feature extraction.

Figure 1.

Two resolution levels of wavelet packet decomposition: (a) resolution level 1; (b) resolution level 2.

Experiment and results

A total of 112 samples of defect-free and fabrics with defects are prepared. The sample is horizontally placed, and is tightly held at two ends by two rollers. A charge-coupled device (CCD) camera is placed directly above the sample. Three lamps of 60 W at 25 cm above the sample and one lamp of 60 W at 8 cm directly below the sample are used to illuminate the sample. The three lamps above the sample are equally placed at an angle interval of 120^o and direct to the sample at angle of 45^o with the horizontal direction. The sample image, covering an area of 25 cm² in the fabric, is captured and digitized by the CCD camera and frame gabber at a resolution of $256 \times 256$ pixels. Each pixel has a gray level between 0 and 255. In this study, the knitted fabric samples with seven categories, non-defect and six kinds of defects: a hole, set mark (coarse), dropped stitch, oil stain, streak, and tight end, are to be classified. The sample images of these categories are shown in Figure 2.

Figure 2.

Sample images: (a) non-defect; (b) hole; (c) set mark (coarse); (d) dropped stitch; (e) oil stain; (f) streak; (g) tight end.

The wavelet packet decomposition of every image is performed based on the best-basis wavelet packet tree. The image is decomposed into four sub-images by the filtering process. The original image and its four sub-images can be viewed as the parent node and children nodes, respectively, of a tree. The “Shannon’s entropy” of a $N \times N$ image is defined as¹

ɛ = - \sum_{m = 1}^{N} \sum_{n = 1}^{N} [ω (m, n)] 2 \log [ω (m, n)] 2

(8)

where

ω (m, n)

denotes the wavelet packet coefficients of the image. If the sum of the entropy of four children nodes is lower than the entropy of the parent node, the decomposition is further applied to each of the children nodes; otherwise, the decomposition is terminated. In this paper, three resolution levels for best-basis wavelet packet decomposition are used, and the sub-images are displayed in Figure 3.

Figure 3.

Best-basis wavelet packet decomposition of sample images: (a) non-defect; (b) hole; (c) set mark (coarse); (d) dropped stitch; (e) oil stain; (f) streak; (g) tight end.

The wavelet packet coefficients from each of the sub-images are used for feature extraction; thus, we computed the entropy for all the sub-images using Equation (8). In order to reduce the dimension of the feature space, we chose the lowest and second lowest entropy, $ɛ_{1}$ and $ɛ_{2}$ , as two of the features.¹⁰ The third feature selected is the standard deviation based on the original image:¹

σ = \sqrt{\frac{1}{N 2} \sum_{m = 1}^{N} \sum_{n = 1}^{N} [ω (m, n) - μ] 2}

(9)

where μ is the mean. The entropy and standard deviation are shift-invariant measures, which are suitable for being textural features. The feature values must be normalized between 0 and 1.

Previously, Guan¹⁹ used the standard deviation of the warp and weft sub-window as the extracted features for determining defect existence. If the standard deviation of test sub-window exceeds the standard deviation of the normal fabric sub-window, it can judge the existence of defects. If the defect is detected, it will continue to extract energy and texture entropy, with very poor texture features. The standard deviation represents the changes of the texture surface. The greater change indicates the uneven texture surface; on the contrary, small changes represent consistency of texture surface. Therefore, the lowest and second lowest entropy as well as standard deviation are applied in this study for effectively and precisely inspecting fabric defects.

Figure 4 shows the normalized feature values of our training samples in the projection planes, $ɛ_{1}$ – $ɛ_{2}$ and $ɛ_{1}$ –σ, of the feature space. From either of the planes, each category of the features is distributed in a group. Most groups are separated from the others and some groups are close to each other. The closeness of any two groups may lead to the consequence that for new samples, one of the corresponding categories is easily misclassified to be the other if only two features are used. Hence, besides the two features, we need the third feature to increase the separation degree of groups in the three-dimensional (3D) feature space, enabling the recognition of seven categories to be enhanced. As a result, seven groups of the features are separately distributed in the feature space, illustrating that our features selected are definitely appropriate.

Figure 4.

Normalized feature values in the projection planes of the feature space: (a) ɛ₁–ɛ₂ plane; (b) σ–ɛ₁ plane (♦ non-defect, ▪ hole, ▴ set mark (coarse), x dropped stitch, * oil stain, • streak, + tight end).

We chose a three-layer back-propagation neural network, consisting of one input layer, one hidden layer, and one output layer, as a classifier. The input layer has three nodes, representing three features of the lowest entropy, second lowest entropy, and standard deviation. The output layer has seven nodes, denoting seven categories to be classified. Five nodes were selected in the hidden layer. Thus, the network contains 15 and 35 weights in the input-to-hidden and hidden-to-output connections, respectively, as well as five and seven thresholds in the hidden and output layers, respectively. The sigmoid function was used to compute the output of each node in the hidden and output layers. The input vector consists of the normalized feature values. The non-defect, hole, set mark (coarse), dropped stitch, oil stain, streak, and tight end correspond to the first entry to the seventh entry of the output vector in order. The desired output vector of each category has 1 in the corresponding entry and 0 in the remaining entries. For example, the desired output vectors for non-defect, hole, set mark, dropped stitch, oil stain, streak, and tight end are (1, 0, 0, 0, 0, 0, 0), (0, 1, 0, 0, 0, 0, 0), (0, 0, 1, 0, 0, 0, 0), (0, 0, 0, 1, 0, 0, 0), (0, 0, 0, 0, 1, 0, 0), (0, 0, 0, 0, 0, 1, 0), and (0, 0, 0, 0, 0, 0, 1), respectively. Fifty-six samples, eight samples for each category, were randomly selected for training. The inputs and the desired outputs are used to recursively adjust the weights and thresholds until an acceptably small root mean square error (RMSE) is reached. For the network’s learning parameter configuration, this experiment’s hidden layer nodal number used (input layer nodal number + output layer nodal number)/2. The learning rate and momentum factor were set using a comparison of the network convergence factor at 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8 and 0.9. The result was that when the hidden layer nodal number was 5, the learning rate was 0.9 and the momentum factor was 0.3. The RMSE decreases rapidly before 2000 epochs, and is about 0.05 at 10,000 epochs where the final weights and thresholds determine the neural network for testing. The feature values of a testing set of 56 samples, eight samples for each category, are normalized and transmitted to the input layer, and then forward computed from layer to layer until the output layer produces its outputs. The largest entry value among seven entries in the resulting output vector, which is formed by the outputs of the output layer, determines the corresponding category. Our results demonstrate that with the three features, the neural network can inspect the test samples with an accuracy of 100%. Table 1 lists the output vectors of 21 test samples. It takes about 0.6–0.8 seconds to complete classification of a sample.

Table 1.

Output vectors of 21 test samples

Category	Target vector	Output vector
Non-defect	(1, 0, 0, 0, 0, 0, 0)	(0.9253, 0.0020, 0.0086, 0.0404, 0.0000, 0.0001, 0.0413)
		(0.9504, 0.0079, 0.0000, 0.0999, 0.0000, 0.0000, 0.0000)
		(0.9473, 0.0000, 0.0213, 0.0516, 0.0039, 0.0091, 0.1705)
Hole	(0, 1, 0, 0, 0, 0, 0)	(0.0039, 0.8166, 0.0086, 0.0001, 0.0137, 0.0138, 0.0020)
		(0.0000, 0.8700, 0.0560, 0.0032, 0.0000, 0.0329, 0.0000)
		(0.0185, 0.8375, 0.1321, 0.0000, 0.1429, 0.0000, 0.0221)
Set mark (coarse)	(0, 0, 1, 0, 0, 0, 0)	(0.0027, 0.0132, 0.9237, 0.0248, 0.0160, 0.0589, 0.0000)
		(0.0248, 0.1950, 0.8772, 0.0116, 0.0000, 0.0000, 0.0022)
		(0.0000, 0.3473, 0.8387, 0.0000, 0.0856, 0.0552, 0.0115)
Dropped stitch	(0, 0, 0, 1, 0, 0, 0)	(0.0000, 0.0223, 0.0345, 0.9534, 0.0780, 0.0482, 0.0000)
		(0.0210, 0.0109, 0.0195, 0.9575, 0.0051, 0.0091, 0.0000)
		(0.0005, 0.0216, 0.0421, 0.9580, 0.0000, 0.0574, 0.0075)
Oil stain	(0, 0, 0, 0, 1, 0, 0)	(0.0108, 0.0723, 0.0025, 0.0087, 0.9208, 0.0111, 0.0102)
		(0.0156, 0.3212, 0.0849, 0.0000, 0.8814, 0.0055, 0.0000)
		(0.0000, 0.1089, 0.0248, 0.0189, 0.9619, 0.0047, 0.0000)
Streak	(0, 0, 0, 0, 0, 1, 0)	(0.0000, 0.0393, 0.0070, 0.0086, 0.0046, 0.9167, 0.0000)
		(0.0060, 0.0294, 0.0055, 0.0153, 0.0000, 0.9301, 0.0033)
		(0.0000, 0.0330, 0.0000, 0.0253, 0.0583, 0.9146, 0.0017)
Tight end	(0, 0, 0, 0, 0, 0, 1)	(0.0484, 0.0021, 0.0000, 0.0000, 0.0000, 0.0060, 0.9440)
		(0.0762, 0.0036, 0.0289, 0.0032, 0.0000, 0.0085, 0.9422)
		(0.0009, 0.0000, 0.0592, 0.0037, 0.0027, 0.0000, 0.9426)

In our experiments, all three features are required to yield accurate classification. The lack of one of the three features will give the wrong classification. When only two features of the lowest and second lowest entropy are used, two samples of holes and two samples of set marks (coarse) are misclassified as set marks (coarse) and holes, respectively. The choice of only the lowest entropy and standard deviation leads to incorrect classification of five samples of holes. The five defect samples are identified to be set mark (coarse). When only the second lowest entropy and standard deviation are selected, seven samples of holes are falsely inspected. Three of them become oil stains and the other four are set marks (coarse). Table 2 lists the output vectors of these misclassified samples, and the underlined entry in each vector, the largest one among seven entries, corresponds to the wrong category.

Table 2.

Output vectors of misclassified samples using only two features

Feature	Category	Target vector	Output vector
ɛ₁ and ɛ₂	Hole	(0, 1, 0, 0, 0, 0, 0)	(0.0030, 0.3251, 0.6012 , 0.0000, 0.0052, 0.0131, 0.0010)
	Hole	(0, 1, 0, 0, 0, 0, 0)	(0.0140, 0.3011, 0.5127 , 0.1232, 0.0456, 0.1329, 0.0000)
	Set mark (coarse)	(0, 0, 1, 0, 0, 0, 0)	(0.0000, 0.5473 , 0.4387, 0.0000, 0.0856, 0.0552, 0.0115)
	Set mark (coarse)	(0, 0, 1, 0, 0, 0, 0)	(0.0000, 0.5247 , 0.3984, 0.0287, 0.0150, 0.0981, 0.0214)
ɛ₁ and σ	Hole	(0, 1, 0, 0, 0, 0, 0)	(0.0130, 0.3928, 0.5748 , 0.0041, 0.0000, 0.0000, 0.0029)
			(0.0230, 0.3647, 0.5478 , 0.0058, 0.0006, 0.0510, 0.1085)
			(0.0151, 0.3191, 0.6547 , 0.1560, 0.1232, 0.0456, 0.0156)
			(0.0153, 0.3594, 0.5791 , 0.0041, 0.0000, 0.0000, 0.0029)
			(0.0000, 0.2791, 0.6981 , 0.0954, 0.0219, 0.0032, 0.0510)
ɛ₂ and σ	Hole	(0, 1, 0, 0, 0, 0, 0)	(0.0021, 0.3958, 0.0819, 0.0040, 0.6487 , 0.0359, 0.0471)
			(0.0947, 0.3214, 0.5147 , 0.0810, 0.0000, 0.0010, 0.0023)
			(0.0147, 0.3089, 0.0201, 0.0098, 0.6102 , 0.0798, 0.0405)
			(0.0047, 0.4120, 0.0798, 0.0405, 0.5142 , 0.0008, 0.0018)
			(0.0947, 0.3154, 0.5102 , 0.0204, 0.0079, 0.0117, 0.0709)
			(0.0439, 0.3519, 0.6012 , 0.0099, 0.0156, 0.1212, 0.0032)
			(0.0033, 0.3171, 0.5942 , 0.1560, 0.1232, 0.0546, 0.1329)

Conclusions

The wavelet packet decomposition displays the information content of a knitted fabric image in different directions and frequencies. In this paper, the feature set includes the lowest-two entropy based on three resolution levels of best-basis wavelet packet decomposition, and the standard deviation of the original image. The wavelet packet decomposition and the smaller feature set lead to less computation time in classification while still achieving a high accuracy rate. If only any two of three features are used in this study, the experiment results will give wrong classification. However, if the three features are applied in this study, an accuracy of 100% inspection will be achieved. The experimental results are indicated in Tables 1 and 2.

Footnotes

Funding

This study was supported by the National Science Council of the Republic of China (grant no. 97-2221-E-011-030-MY3).

References

Kumar A and Pang G. Identification of surface defects in textured materials using wavelet packets. In: proceedings of IEEE conference on industry applications, Vol. 1, 2001, pp.247--251.

Chen

Liang

Yau

Sun

Wang

Lin

Lien

. Classifying textile faults with a back-propagation neural network using power spectra. Text Res J 1998; 68: 121–126.

Tsai

Lin

. Applying an artificial neural network to pattern recognition in fabric defects. Text Res J 1995; 65: 123–130.

Chan

. Fabric defect detection by Fourier analysis. IEEE Trans Ind Appl 2000; 36: 1267–1276.

Kim S, Lee MH and Woo KB. Wavelet analysis to fabric defects detection in weaving processes. In: proceedings of IEEE international symposium on industrial electronics, Vol. 3, 1999, pp.1406--1409.

Wang

Zhu

Tang

. Feature extraction of radar multiple-target echoes using wavelet packet transform with the best bases. Int J Pattern Recognit Artif Intell 2003; 17: 127–150.

Laine

Fan

. Texture classification by wavelet packet signatures. IEEE Trans Pattern Anal Mach Intell 1993; 15: 1186–1191.

Pun

Lee

. Rotation-invariant texture classification using a two-stage wavelet packet feature approach. IEE Proc Vision Image Signal Proc 2001; 148: 422–428.

Lee MC and Pun CM. Texture classification using dominant wavelet packet energy features. In: proceedings of IEEE southwest symposium on image analysis and interpretation, 2000, pp.301--304.

10.

Tsai

. Fabric inspection based on best wavelet packet bases. Text Res J 2000; 70: 662–670.

11.

Lee

Choi

Chol

. Surface defect inspection of cold rolled strips with features based on adaptive wavelet packets. IEICE Trans Inf Syst 1997; 5: 594–604.

12.

Tsai

. Automatic inspection of fabric defects using an artificial neural network technique. Text Res J 1996; 66: 474–483.

13.

Fan

Wang

Chang

. Fabric classification based on recognition using a neural network and dimensionality reduction. Text Res J 1998; 68: 179–185.

14.

Tsai

. The inspection of fabric defects by using wavelet transform. J Text Inst 2000; 91: 420–423.

15.

Mallat

. Theory for multi-resolution signal decomposition: the wavelet representation. IEEE Trans Pattern Anal Mach Intell 1989; 11: 674–693.

16.

Yen

. Wavelet packet feature extraction for vibration monitoring. IEEE Trans Ind Electron 2000; 47: 650–667.

17.

Hou

Liu

. Wavelet packet image coding algorithm combined with the actual coding costs. Opt Eng 2003; 42: 1640–1648.

18.

Chang

Kuo

CCJ

. Texture analysis and classification with tree-structured wavelet transform. IEEE Trans Image Process 1993; 2: 429–441.

19.

Guan, S. Fabric defect detection based on fusion technology of multiple algorithm. In: 2010 2nd international conference on signal processing systems (ICSPS), Dalian, China, 5–7 July 2010, paper no. 11516437, pp.V3-553--557, USA: IEEE.