A thermal-based defect classification method in textile fabrics with K-nearest neighbor algorithm

Abstract

In this study, fabric defects have been detected and classified from a video recording captured during the quality control process. Fabric quality control system prototype has been manufactured and a thermal camera was located on the quality control machine. The defective areas on the fabric surface were detected using the heat difference occurring between the defective and defect-free zones. Gray level co-occurrence matrix is used for feature extraction for defective images. The defective images are classified by k-nearest neighbor algorithm. The image processing stage consists of wavelet, threshold, and morphological operations. The defects have been classified with an average accuracy rate of 96%. In addition, the location of the defect has been identified and the defect type and location are recorded during the process via specially designed image processing interface. According to the experimental results, the proposed method works effectively.

Keywords

Quality control classification thermal camera feature extraction

Introduction

The defects that spoil the fabric appearance and performance can occur during the fiber–fabric manufacturing or fabric-finishing processes. These defects are not desirable and affect quality level of the fabric negatively. The quality control process of the fabric is carried out by the human eye over a lightened table. The fabric layer is passing through from a lightened and leaned quality control table. During this process, defective regions are detected by a quality control staff. The quality control staff has to detect approximately 2-m wide region making the process time-consuming and tedious for the person.

Based on the information obtained from the literature, good quality control staff can detect 60–70% errors and is able to control maximum 2-m width of the fabric region [1,2]. This way the quality control procedure is not objective and does not provide a statistically meaningful data. In literature, some works have been performed to make fabric quality control process automatically. In these studies, camera system and image processing routines have been employed. By using different filters and methods, defective regions have been detected both from an image database and from a real-time image capture.

A system has been presented for the defect detection and classification of textile fabric in real time [3]. It was consisted of double thresholding, binary filtering and labeling, decomposition with wavelet transform, and statistical texture feature extraction. Mak and Peng [4] have used Gabor wavelet network for feature extraction and defect detection process. The defect detection system consists of Gabor filter and Gaussian smoothing filter. The system performance was evaluated by both online and offline tests.

A machine vision system has been developed for fabric inspection and classification process. Defect detection algorithm was based on wavelet transform and morphological operations and was programmed real time in MATLAB [5]. Hu and Tsai have proposed tree-structured wavelet transform to detect and classify defects. Missing picks, missing ends, broken fabrics, and oil stans were classified using back propagation neural network (NN) algorithm [6]. Jing investigated real-time inspection of diversified fabric texture. Gabor filter was improved to detect fabric defects [7]. An effective defect detection scheme has been designed. Gabor filter has been adjusted to match with the texture information of nondefective fabric image via genetic algorithm. Then adjusted optimal Gabor filter, has been used for detecting defects on defective fabric images and the texture of the background has been kept the same for both defective and defect-free fabric images [8]. Jianli and Baoqi have proposed a method consisting gray-level co-occurrence matrix (GLCM), principal component analysis (PCA), and NN. They have used GLCM for feature extraction and reduced the number of dimensions for input vector using PCA. PCA establishes a low representation of the high-dimensional space. These components that were obtained from high dimension are independent [9]. Three-layer back-propagation NN was used to classify defects [10]. Kuo and Su have used NN and GLCM methods for fabric defect classification. Energy, entropy, contrast, and dissimilarity were extracted with GLCM, and then four types of defects were classified with NN [11]. Zhi et al. [12] used adaptive wavelet transform for fabric defect detection. They designed adaptive wavelets for five different kinds of fabric defects. Their work has given promising results. Zhiqiang et al. [13] proposed fabric defect detection technology based on wavelet transform and NN. They combined two methods for effective defect system. Guan et al. [14] proposed new method based on wavelet characteristics. The experimental study confirms that the new method’s detection accuracy is 92.5%. Li and Di [15] used wavelet decomposition for their image-processing approach to defect detection process. Their study shows that the proposed system is effective. The k-nearest neighbor (KNN) algorithm [16] classifier is a simple nonparametric, supervised distance-based learning algorithm where the pattern is assigned to the class shared by the majority of the KNN. In López et al., [17,18] used KNN to classify ceramic tile surfaces based on chromatic features extracted from individual channels. In Chan and Pang [19] classified four types of fabric defects by fitting into the expected feature model.

This study differs from other studies with two major properties. In this study, a thermal camera was used for defect detection. Thermal camera is an imaging system that uses infrared (IR) energy emitting from the objects, then these cameras convert the IR energy in certain colors and images based on their energy levels. The images obtained by the thermal camera illustrate the hot areas with light colors and cold areas with dark colors. The reasons for using thermal camera is to find all defects, even invisible by human eye and to detect all defects difficult to capture by other type of cameras. Second, a prototype of the fabric quality control machine, integrated by an IR heater on it, was manufactured and used for detection of defects stemming from the heat difference between the defective and defect-free zones from the fabric surface. The main purpose of this prototype is simulation of the ram machine (drying unit) where fabric has already been heated at the exit from system. Thus, it is possible to detect the defects by both human eye as in a conventional method and by thermal camera allowing the evaluation of the accuracy of the system. Results proved that this prototype can detect fabric defects adequately. The thermal camera can be added directly at the end of the ram machine eliminating the fabric quality control process yielding time and cost savings.

The defect control system developed here with the experimental results of prototype embodiments is described. According to the results, designed system produces effective results.

Material

In this study, a light-weighted 100% cotton plain-weaved woven raw fabric (weight 95 g/m²; warp 46 threads/cm; weft 30 threads/cm) was used for the experimental process. The sample of fabric has got four types of defects: hole, tear, nope, and foreign yarn. Hole and tears are erupted that are occurring in fabric during weaving. Small fibers are seen on the surface region of the fabric that are called nope. Foreign yarn is a thread that is involved in weft yarn during the yarn production or weaving. Figure 1 shows these defects, respectively. Some of the defects are made randomly with different length and width dimensions due to the need for visible defects.

Figure 1.

Images of the defective fabrics.

Overview of the system architecture

A machine vision system is designed for real-time fabric inspection. The system consists of an industrial fabric inspection machine, thermal camera, and attachment equipment, and computer. Figure 2 shows the overview of the system architecture. The developed system will be appended directly to the end of the ram machine with an extra image-processing software and the thermal camera required for defect detection. ETK 1500 model fabric inspection machine is used for fabric winding ensuring a smooth and uniform movement. The fabric winding speed can be adjusted between 0 and 60 m/min. The machine allows the adjustment of the fabric tension, to achieve the appropriate tension during the process. IR heating system devices as in the ram of the fabric surface of all areas to be heated are designed to provide an equivalent level. Ufo industrial system has been used for IR heating in the system. The image acquisition process is achieved by using a Testo thermal camera positioned 25 cm away from the fabric surface. The resolution of the camera is 160 × 120 pixel and it has a precision lower than 50 mK. In this study, 20–30℃ has been used for heating the fabric surface with the IR heater. The fabric-winding speed was about 20 m/min and the fabric width was 75 cm in experimental process.

Figure 2.

Overview of the system architecture.

Operation of the system architecture is as shown in Figure 3. The input of the system is a video that was captured by thermal camera via fabric. It was recorded by a software. Initially, fabric is heated by an IR heater in a balanced way, and then fabric images are recorded instantaneously. Defective zones become visible because of the heat difference between defect-free areas, then images of these defective areas are detected from video and feature extraction process. The obtained properties are classified by KNN algorithm. Defective areas are detected on the fabric simultaneously by using specially designed image-processing software. The software user could record the type and the position of the defect as an output of the system. The block diagram of the image-processing system is shown in Figure 5.

Figure 3.

Flowchart of the designed system.

Figure 5.

Flowchart of the image-processing algorithm.

Image processing and feature extraction approach

Image processing. In this study, wavelet transform is used at the image-processing stage. Each signal can be described as the combination of certain signals in various frequencies. The Fourier transform converts the signal as linear combinations of the infinite number of sine and cosine values. Fourier transform of a signal provides information only in frequency space. However, a change in frequency that occurs does not provide information about the time history. Fourier spectrum shows information about frequency values on an image; however, it does not provide information about the location of the frequency values. Signal has to be divided into the smaller parts and processed locally to determine a change to the location of the signal component. In the wavelet process, irregular, time-limited and asymmetric signal components as base wavelet are used by shifting at different scales. Different types of wavelets are shown as in Figure 4: Meyer, Haar, Daubechies-4, Morlet, and Mexican Hat [20].

Figure 4.

Different types of wavelets [20].

As a result of wavelet transform, a signal or an image is subjected to high-pass or low-pass filters, to parse into detailed and approach subbands. Approach subband coefficients are high-scale and low-frequency signal. Detailed subband coefficients are high-frequency signal components at the lowest scale. According to the desired resolution, approach subband is stripped continuously. According to the application form, wavelet transform is divided into two groups as discrete and continuous. In continuous wavelet, transform parameters do change constantly and the calculation of wavelet coefficients takes time. Thus, the discrete wavelet transform (DWT) is performed more frequently [21].

Wavelet transform method is often used in fabric defect detection. By using wavelet transform method, good defect detection performances have been obtained in the literature. Additionally, this method requires less statistical calculation compared to other methods [12].

The algorithm developed for fabric defect detection consists of a noise removing filter, DWT, thresholding, binarization, and dilation and erosion operations. The image is converted to grayscale and denoised form via Wiener filtering [22], a decomposition by using a certain type of wavelet namely “db3” belongs to Daubechies wavelets group. The approximation image is applied to thresholding for binarization. It was chosen as constant value of 0.18 as the thresholding value for all images. Dilation and erosion operations are applied to binary image for outlier and unnecessary points. The schematics of image-processing algorithm can be seen in Figure 5.

Dilation operation is applied to binary image, and the space between the defective regions are filled up and interconnected. Remaining noises are removed via the erosion operation. The defect detection procedure is summarized in Figure 5 and is repeated for each fabric image frame.

Texture feature extraction

The defect classification is as important as the detection in the fabric-inspection process. The detected defects are classified according to their type, and then recorded with their names in a manual inspection process. There are various fabric defect types and some of them are similar to each other, resulting in different decisions for similar defects in classification process.

Statistical and spatial approaches are used for measuring the texture properties. The statistical approach is based on the statistical properties of the intensity histogram. It is the most frequently used method for texture analysis and classification purposes. The spatial approach is based on the Fourier spectrum. It is generally suited for describing the directionality of periodic patterns in an image. The properties are extracted and feature vector is formed for each fabric image during application. The first-order statistical properties consist of average gray level, average contrast, smoothness, third moment, uniformity, and entropy. They are derived from the intensity histogram of the gray-level image [23]. In this work, average and variance levels are obtained from the gray-level image.

Some of the second-order statistical properties are energy, contrast, correlation, variance, inverse difference moment, sum average, sum variance, sum entropy, entropy [24 –26]. They are obtained from GLCM of images by using the methodology proposed by Haralick [25]. We have used contrast, correlation, energy, and homogeneity features for classification. In Figure 6, GLCM is shown that has been observed from texture image. Figure 6 shows the number of neighborhoods of the case where situation is both the reference pixel and target pixel are equivalent to 1. For instance, the case of [2,4] indicates a neighborhood number where is the reference pixel is 5 and the target pixel is 7. Two of them are written in a matrix table.

Figure 6.

Obtain co-occurrence matrix.

GLCM of the examined fabric contrast in the image indicates the difference between the reference pixel and the target pixel, which is equivalent to the reference and target pixels

Contrast = \sum_{i = 0}^{G - 1} \sum_{j = 0}^{G - 1} Co (i, j) (i - j) 2

(1)

GLCM diagonals describe the contrast value computed as in equation (1).

Energy is a measure of the uniformity of a fabric. It can be measured as high value in regular order and can be calculated as in equation (2).

Energy = \sum_{i = 0}^{G - 1} \sum_{j = 0}^{G - 1} Co (i - j) 2

(2)

Homogeneity is called as a reverse torque difference. Analyzing the homogeneity equation, unlike contrast weights, homogeneity weights are decreased exponentially with distance from the diagonal. Equation (3) shows the calculation of homogeneity.

Homogeneity = \sum_{i = 0}^{G - 1} \sum_{j = 0}^{G - 1} \frac{1}{1 + (i - j) 2} Co (i, j)

(3)

The Correlation measures the linear dependency of gray levels of neighboring pixels. It can be calculated for successively larger window sizes as in equation (4).

Correlation = \sum_{i = 0}^{G - 1} \sum_{j = 0}^{G - 1} \frac{{ixj} xP (i, j) - {μ_{X} x μ_{y}}}{σ_{X} x σ_{y}}

(4)

In this study, the identified defects are automatically classified according to their texture features by using the KNN method. The texture features are assessed as the network input values and the defect class is obtained as the output.

Classification

KNN is a trained algorithm and its purpose is to cluster on an existing learning data when a new instance occurs. The algorithm determines the new instance clusters by looking at its closest K neighborhood [27]. Document and training document vectors are used in order to classify a new vector with KNN algorithm. After the vector expression and classification of all training documents, these vectors are evaluated by using KNN algorithm.

For classification process, KNN is one of the most important nonparametric algorithm [28]. The training samples are used for classification rules without any additional data. It predicts the test sample class according to the K training values. When the chosen k value is small, the similar recordings are put in the same cluster, while choosing the k value too big may cause misclassification of similar recordings. Figure 7 shows the case for choosing the k value as 3 in two-dimensional space [29].

Figure 7.

Classification with k-nearest neighbor (k = 3).

The process of KNN algorithm for sample X is [30]:

C1, C2, … , Cj are training categories. The sum of them, becomes m-dimension feature vector after feature reduction, are N.

For all training samples, X is a feature vector of the form (X1, X2, … , Xm).

The similarities are calculated between all training samples and X. The similarity SIM(X, d_i) is calculated as equation (5)

SIM (X, d_{i}) = \frac{\sum_{j = 1}^{m} X_{j} . d_{ij}}{\sqrt{(\sum_{j = 1}^{m} X_{j}) 2} \sqrt{(\sum_{j = 1}^{m} d_{ij}) 2}}

(5)

K samples are chosen, which are larger than N similarities of SIM(X, d_i), (i = 1, 2, … , N), and are treated as a KNN collection of X. The probability of X corresponding to each category is calculated as in equation (6).

P ({X, C}_{j}) = \sum_{d} SIM (X, d_{i}) \times y (d_{i}, C_{j})

(6)

In equation (6), y(d_i, Cj) is a category attribute function described as

y (d_{i}, C_{j}) = {1, d_{i} \in C_{j} 0, d_{i} AB \notin C_{j}}

(7)

Sample X is judged and it describes the category that has the largest P(X, C_j). In this study, four defect types; hole, nope, tear, and foreign yarn were classified by using KNN algorithm. Thirty sample images have been used during training for classified. Then, the classification process is based on the faults that were predefined.

Results and discussion

A user interface was prepared in MATLAB® for real-time defect detection application. The Figure 8 shows the interface for detection process. During the fabric flow, the user interface helps to detect the defective areas from the fabric surface, allowing the detection of position and properties of the defective zones were examined. Also the class of the error type was determined. The user interface in Figure 8 consists of load video, stop, take frame, and trial video buttons. Also there are list boxes for showing instant frame number, defective position of fault, and defective type. On the selected defective area, the defect can be seen as a binary format. In the parameter adjustment part, user could set contrast and brightness values of the image. For all types of defects, corresponding defect values were saved for future use.

Figure 8.

Detection of nope fault.

The experimental results for all kinds of defects determined by the image processing apply to pictures are shown in Figure 9. The proposed image-processing algorithm produces promising results. Nope, hole and tear faults were detected distinctly with the image-processing algorithm, only foreign yarn fault was detected roughly. However, the algorithm can be tuned by changing the threshold value and morphological operations to improve identification of the defect more accurately.

Figure 9.

Detection of all types of defects.

The performance of the proposed algorithm was evaluated statistically with 385 sample involving different type of fabric images. Thirty of the image features were used during training. Six samples for nope, hole, and foreign yarn defects, 12 sample for tear fault were selected as a training set. The remaining 355 samples were used for algorithm test performance. As shown in Table 1, applications with four different defects, 96.15% classification rate was obtained. Developed processing algorithm and the new method yielded successful results.

Table 1.

Evaluation of the defect detection algorithm performance.

	Nope	Tear	Hole	Foreign yarn
Number of images	40	151	154	40
True detect	37	143	150	40
False detect	3	8	4	0
Rate (%)	92.5	94.7	97.4	100

In Figure 10, the visual results of the classification process are located. Classification results are plotted for average gray level and gray-level variance values. Foreign yarn errors are classified correctly. Some of the errors that belong to unexpected type of defects were classified wrongly.

Figure 10.

Classification results with KNN.

Conclusion

In this study, a new thermal-based method has been developed for identification of defects in textile fabrics. Basic woven pattern has been used as a fabric material. Thermal camera is used for image capture. Thus, the proposed system can be easily adapted to the output of the ram machine eliminating the need for a quality control machine.

Fabric defect identification software has been developed in MATLAB. For classification purposes, features of the defective images are extracted by using GLCM. Then defective images are classified by KNN algorithm with the help of these features. In the image-processing step, a wavelet-based algorithm has been used for the defective area. The developed defect identification system is expected to provide time and cost benefits. Statistical results about the defect type and place can be obtained as additional data.

According to the experimental results, the proposed algorithm produces 96% accuracy rates on average. It is expected that the proposed system that could be applied to various types and color of fabrics in further studies. Further studies could be in the way of determination of different fabric defect types in various fabric structures.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study is supported by Marmara University Scientific Research Projects Management Unit with the name “Detect Fabric Defects with Image Processing and Data Mining Techniques” and the project number is FEN-C-DRP-141112-0339.

References

Anagnostopoulos

Vergados

Kayafas

. A computer vision approach for textile quality control. J Visual Comp Anim 2001; 12: 31–44.

Ralló

Millán

Escofet

. Wavelet based techniques for textile inspection. Opt Eng 2003; 26: 838–844.

Karayiannis YA, Stojanovic R, Mitropoulos P, et al. Defect detection and classification on web textile fabric using multiresolution decomposition and neural networks. In: Electronics, Circuits and Systems, 1999 Proceedings of ICECS'99 The 6th IEEE International Conference on IEEE, 1999, pp. 765–768.

Mak KL and Peng P. Detecting defects in textile fabrics with optimal Gabor filters. Int J Comput Sci 2006; 1(4): 274–282.

Çelik H, Dülger L and Topalbekiroğlu M. Development of a machine vision system: real-time fabric defect detection and classification with neural networks. J Text Inst 2014; 105: 575–585.

Tsai

. Fabric inspection based on best wavelet packet bases. Text Res J 2000; 70: 662–670.

Jing

. Combined fabric defects detection approach and quadtree decomposition. J Industr Text 2012; 41: 331–344.

Jing J, Yang P, Li P, et al. Supervised defect detection on textile fabrics via optimal Gabor filter. J Industr Text 2013. doi:10.1177/1528083713490002.

Jolliffe I. Principal component analysis. Wiley Online Library, 2005.

10.

Jianli

Baoqi

. Identification of fabric defects based on discrete wavelet transform and back-propagation neural network. J Text Inst 2007; 98: 355–362.

11.

Kuo

C-FJ

T-L

. Gray relational analysis for recognizing fabric defects. Text Res J 2003; 73: 461–465.

12.

Zhi YX, Pang GK and Yung NH. Fabric defect detection using adaptive wavelet. In: Acoustics, Speech, and Signal Processing, 2001 Proceedings (ICASSP'01) 2001 IEEE International Conference on IEEE, 2001, pp. 3697–700.

13.

Zhiqiang K, Chaohui Y and Qian Y. The fabric defect detection technology based on wavelet transform and neural network convergence. In: Informtion and Automation (ICIA), 2013 IEEE International Conference on IEEE, 2013, pp. 597–601.

14.

Guan S, Shi X, Cui H, et al. Fabric defect detection based on wavelet characteristics. In: Computational Intelligence and Industrial Application, 2008 PACIIA'08 Pacific-Asia Workshop on IEEE, 2008, pp. 366–370.

15.

Li Y and Di X. Fabric defect detection using wavelet decomposition. In: Consumer Electronics, Communications and Networks (CECNet), 2013 3rd International Conference on IEEE, 2013, pp. 308–311.

16.

Fukunaga

. Introduction to statistical pattern recognition, San Diego, CA, USA: Academic press, 1990.

17.

López

Valiente

Baldrich

Fast Surface Grading Using Color Statistics in the Cie Lab Space. In: Marques

Blanca

NPdl

Pina

(eds). Pattern Recognition and Image Analysis, Berlin Heidelberg, Springer, 2005, pp. 666–673.

18.

López

Valiente

J-M

Prats

J-M

Surface Grading Using Soft Colour-Texture Descriptors. In: Sanfeliu

Cortés

(eds). Progress in Pattern Recognition, Image Analysis and Applications, Berlin Heidelberg, Springer, 2005, pp. 13–23.

19.

Chan

C-h

Pang

. Fabric defect detection by Fourier analysis. Industry applications, IEEE transactions on 2000; 36: 1267–1276.

20.

Sonka M, Hlavac V and Boyle R. Image processing, analysis, and machine vision. Thomson Toronto, 2008.

21.

Gonzalez RC and Woods RE. Digital image processing. Prentice Hall Upper Saddle River, NJ, 2002.

22.

Simon

. Adaptive filter theory. Prentice Hall 2002; 2: 478–481.

23.

Gonzalez RC, Woods RE and Eddins SL. Digital image processing using MATLAB. Pearson Education India, 2004.

24.

FahimIrfanAlam

. Optimized calculations of haralick texture features. European Journal of Scientific Research 2011; 50: 543–553.

25.

Haralick

. Statistical and structural approaches to texture. Proceedings of the IEEE 1979; 67: 786–804.

26.

Haralick

Shanmugam

Dinstein

. Textural features for image classification. Syst Man Cybernet IEEE Trans 1973; 3: 610–621.

27.

Yildiz

Yildirim

Altilar

. Spam Filtering with Parallelized Knn Algorithm. In: Akademik

Bilişim

(ed.) 2008.

28.

Dasarathy, BV. Nearest Neighbor (Nn) Norms: Nn Pattern Classification Techniques. IEEE Computer Society Press, 1991.

29.

Duda

Hart

Stork

. Pattern classiﬁcation 2001; Vol 10, New York: John Wiley, Section, pp. l–l.

30.

Suguna

Thanushkodi

. An improved k-nearest neighbor classification using genetic algorithm. Int J Comput Sci Issues (IJCSI) 2010; 7: 18–21.