A defect detection method for unpatterned fabric based on multidirectional binary patterns and the gray-level co-occurrence matrix

Abstract

A new texture-feature description operator, called the multidirectional binary patterns (MDBP) operator, is proposed in this paper. The operator can extract the detailed distribution of textures in local regions by comparing the differences in the gray levels between neighboring pixels. Moreover, the texture expression ability is enhanced by focusing on the texture features in the linear neighborhood of the image in multiple directions. The MDBP operator was modified by introducing a “uniform” pattern to reduce the grayscale values in the image. Combining the “uniform” MDBP operator and the gray-level co-occurrence matrix, an unpatterned fabric-defect detection scheme is proposed, including texture-feature extraction and detection stages. In the first stage, the multidirectional texture-feature matrix of a nondefective fabric image is extracted, and then the detection threshold is determined based on the similarity between the feature matrices. In the second stage, the defect is detected with the detection threshold. The proposed method is adapted to various grayscale textile images with different characteristics and is robust to a wide variety of image-processing operations. In addition, it is invariant to grayscale changes, performs well when representing textures and detecting defects and has lower computational complexity than other methods.

Keywords

multidirectional binary patterns gray-level co-occurrence matrix texture-feature extraction fabric-defect detection

The ability to detect defects in fabric is important for quality monitoring in the process of textile dyeing. It has been reported that the value of the fabric will be reduced by 45–60% if it has defects.¹ In fact, traditional manual detection methods are not only time consuming but also make it difficult to ensure the inspection quality due to experience and visual fatigue. According to statistics, the error rate of manual detection is generally 35–55%,² and the missing rate is higher than 60% when the width of the fabric is more than 2 meters and the detection speed is greater than 30 meters per minute.³ Therefore, fabric-defect detection urgently requires more intelligent and computerized methods.

In general, fabric is woven with warp and weft threads, which create various patterns and texture features.⁴ In computer graphics, the texture of textile images is composed of many pixels with different gray levels. The key to fabric-defect detection is to extract and distinguish texture features, because the defects destroy the original structure. Currently, mainstream texture-feature extraction methods for detecting defects can be divided into five types: statistical-, model-, spectral-, structure- and learning-based methods.⁵

The statistical-based methods mainly include the gray-level co-occurrence matrix (GLCM),^6–9 the autocorrelation function¹⁰ and the Elo-rating algorithm (ER).^11,12 Haralick et al.⁶ proposed the GLCM method for texture classification based on the generalized co-occurrence matrix (GCM).⁷ Later, the GLCM was widely used in the field of defect detection and achieved a demonstrably good effect, but high dimensionality also resulted in high computational complexity.^8,9 The ER algorithm was first applied in fabric-defect detection by Tsang et al.¹¹ Kang and Zhang¹² presented a defect detection algorithm for various types of fabrics that applied the ER method to the basics of integral images. However, it had a low success rate with yarn-dyed fabric with a complex background texture.

Typical model methods used for defect detection include the Markov random field (MRF)^13–15 and the one-dimensional autoregressive.¹⁶ Ozdemir and Ercil¹⁴ used the MRF model to achieve good results in detecting defects in plain, unpatterned fabric. Zhang et al.¹⁵ proposed a feature-field model by combining the MRF and wavelet method to segment jacquard warp-knitted fabric images. Although this method is not used for fabric-defect detection, the improvement in the MRF method is a good reference for research in this field.

The spectral-based methods mainly include Fourier analysis,¹⁷ Gabor filters^18,19 and wavelet transforms.^20,21 Chan and Pang¹⁷ used a Fourier transform to obtain the center spatial spectrum to detect defects. However, it has a poor detection performance in the local area. Arivazhagan et al.²⁰ proposed a method combining a Gabor filter and a wavelet transform to achieve better detection results, but it was sensitive to noise and was not suitable for fabrics with inhomogeneous textures. Zhu et al.²² improved the discriminant measure of the wavelet transform, which performed well in seam detection for fabrics with irregular patterns.

Local binary patterns (LBP), a texture description operator proposed by Ojala et al.,²³ is a common structure-based method. Tajerpour et al.² were the first to apply the LBP operator to fabric-defect detection. Subsequently, many researchers have proposed modified LBP operators with stronger representation abilities, which are also widely used in fabric texture classification.^24–29 The LBP operator can be used for more than just grayscale images; Li et al.³⁰ converted the fabric image from the RGB color space to the CIELAB color space and combined the energy-based feature images with the LBP operator to detect defects; however, there was no obvious improvement in the structure of the LBP operator.

Recently, defect detection methods based on machine learning have received extensive attention.^31–35 Zhou and Wang³² proposed a new detection method, applied learning adaptive dictionaries. Wei et al.³⁴ presented a faster regional-based convolutional network structure, which is effective in fabric-defect detection. Mei et al.³⁵ proposed an unsupervised learning-based automated detection method. The advantage of this method is that it can be trained with only a small number of samples, but the detection accuracy and stability for some fabric textures with complicated patterns still need to be improved.

Most defect detection methods for unpatterned fabric have limitations based on the fabric material and cannot adapt to changes in the detection environment. Therefore, the key to defect detection is to find an effective texture-feature extraction method and to analyze the differences in the texture features of the defective area and the nondefective area.

In this paper, we propose the multidirectional binary patterns (MDBP) operator: a texture representation operator that can extract the detailed distribution of textures in local regions by comparing the gray-level differences between neighboring pixels. Moreover, the ability of texture expression is enhanced by focusing on the texture features in the linear neighborhoods of the image in multiple directions. This paper further modified the MDBP operator with a “uniform” pattern to reduce the gray levels of the image. Finally, the texture characteristics of the fabric image are described globally in combination with the GLCM and its statistical features. This paper further proposed a defect detection method for unpatterned fabric, which is divided into two stages: a texture-feature extraction stage and a fabric-defect detection stage.

The experiments show that the proposed detection schema can adapt to various grayscale textile images with different characteristics and is robust to a wide variety of image-processing operations. In addition, it is invariant to grayscale changes and has a good performance in representing textures and detecting defects and has a lower computational complexity than other methods.

The structure of this paper is organized as follows. In the next section, the original LBP operator is introduced first, and then the texture representation MDBP operator and the modified version are described. In the third section, the GLCM method and the combined MDBP and GLCM texture-feature extraction algorithm are introduced. In the fourth section, the defect detection schema for unpatterned fabric and its effectiveness are described. In the fifth section, the partial fabric images in the dataset of the experiment are shown. In the sixth section, the parameters are analyzed, and the detection results of the four methods are compared. A conclusion follows in the last section.

Original local binary patterns

LBP is a common texture representation operator that characterizes the gray-level difference between the central pixel and neighboring pixels in the neighborhood. The new gray value of the center pixel can be obtained by encoding pixels as either 0 or 1 and transforming the binary distribution to an integer. The LBP operator can be described by the following equation

{LBP}_{P, R} = \sum_{i = 0}^{P - 1} s (g_{i} - g_{c}) \times 2^{i}

(1)

s (g) = \begin{matrix} {\begin{matrix} 1, & g \geq 0 \\ 0, & g < 0 \end{matrix} \end{matrix}

(2)

In Equation (1), P is the number of neighboring pixels with respect to the internal pixels, R is the radius, g_c is the gray value of the center pixel and g_i describes the gray value of other pixels. In Equation (2), s(g) is the label of the encoded pixels. The gray value of the center pixel will be updated to ${LBP}_{P, R}$ by assigning the corresponding binomial coefficient to each encoded pixel.

The neighborhood usually adopts a square and circular neighborhood, which can increase with increasing radius. Figure 1 illustrates the operational process of the LBP in a square neighborhood, where P = 3 and R = 8.

Figure 1.

The process of the local binary patterns operator in a square neighborhood.

The original LBP operator is simple to calculate, but it has certain disadvantages. Firstly, the gray differences between the neighboring pixels are ignored. Secondly, as the radius increases, only the outermost pixels of the neighborhood are considered, which causes the loss of texture information. Finally, the LBP operator insufficiently considers the direction. To solve the above problems, the MDBP operator is proposed in this paper to describe the texture of the textile image.

Multidirectional binary patterns

The MDBP operator adopts a linear neighborhood and considers the grayscale differences between neighboring pixels. Not only can the central pixel be included in the gray-level representation, but also the internal texture features can be extracted when the neighborhood is large. Therefore, the MDBP operator has a strong ability to describe the detailed distribution of textures in local regions.

In the linear neighborhood, a pixel is labeled as either 1 or 0 by comparing it with the next pixel. Naturally, the last pixel in the current neighborhood is compared with the first pixel in the next linear neighborhood. The binary encoding process of the MDBP operator can be described by the following equation

s_{i} (p_{i} - p_{i + 1}) = \begin{matrix} {\begin{matrix} 1, & p_{i} - p_{i + 1} \geq 0 \\ 0, & p_{i} - p_{i + 1} < 0 \end{matrix}, (i = 0, 1, \dots, N - 1) \end{matrix}

(3)

In Equation (3), p_i and $p_{i + 1}$ describe the gray values of neighboring pixels, s_i is the label of the encoded pixels and N is the number of pixels in the linear neighborhood.

The MDBP operator in one direction is defined by the following equation

{MDBP}_{N} = \sum_{i = 0}^{N - 1} s_{i} \times 2^{(N - 1 - i)}

(4)

The gray value of the first pixel in the linear neighborhood is updated to ${MDBP}_{N}$ in the same way as that of the LBP operator. So, the maximum gray value after applying the MDBP operator is $2^{N}$ . The MDBP operator has grayscale invariance because it characterizes the gray difference between pixels. The calculation of the MDBP operator is shown in Figure 2.

Figure 2.

The calculation process of the multidirectional binary patterns operator.

The MDBP operator defines linear neighborhoods in four directions: 0, 45, 90 and 135 degrees. On the one hand, it conforms to the textile structure; on the other hand, it has an enhanced texture representation ability by extracting more gray information in local areas. The fabric image is processed into four result images after applying the MDBP operator, as shown in Figure 3. The MDBP operator enhances the texture expression in the direction perpendicular to the neighborhood direction, so it can achieve a good effect in distinguishing between defects and nondefects.

Figure 3.

The calculation process of applying the multidirectional binary patterns operator. Where the source image is the image with warp-lacking defects, III_D_a_0018 (Figure 11).

Uniform multidirectional binary patterns

The resulting images of the MDBP operator need to be analyzed to obtain the statistical grayscale information of the texture. However, as the number of pixels in the neighborhood increases, the grayscale of the image increases significantly, which results in high computational complexity. Thus, grayscale dimensionality reduction is a necessary step to improve the detection efficiency of the algorithm.

To solve this problem, the MDBP operator was modified. A parameter U for measuring uniform patterns is introduced to limit the number of hops between 0 and 1 in the binary sequence. Then, the number of occurrences of label 1 is taken as the new gray value of the first pixel when the number of hops in binary sequences does not exceed U. Otherwise, the pixel is labeled as N + 1, which is regarded as a “nonuniform” pattern. In this paper, the value of U is set to 2. The MDBP operator with a uniform pattern is described by the following equations

\begin{matrix} {MDBP}_{N}^{U} = \begin{matrix} {\begin{matrix} \sum_{i = 0}^{N - 1} s_{i}, & if (U ({MDBP}_{N}) \leq U) \\ N + 1, & else \end{matrix} \end{matrix} \end{matrix}

(5)

U ({MDBP}_{N}) = \sum_{i = 0}^{N - 2} | s (p_{i} - p_{i + 1}) - s (p_{i + 1} - p_{i + 2}) |

(6)

In Equations (5) and (6), $U ({MDBP}_{N})$ calculates the number of hops between 0 and 1 in the binary sequence, so the grayscale of the resulting images applied for the MDBP operator will reduced to N + 2.

The proposed MDBP operator has the following advantages. Firstly, the MDBP operator considers the difference in the grayscale values between neighboring pixels, which allows the detailed texture features in a narrow area to be extracted and the ability of texture representation is enhanced. Secondly, the texture features of the fabric image are described in multiple directions by applying the linear neighborhood so that the texture expression is more directional. Finally, the texture information inside the neighborhood can be extracted even if the number of pixels is large, which makes texture-feature extraction more comprehensive.

Gray-level co-occurrence matrix

The GLCM is defined as the probability distribution of pixel pairs appearing in the entire grayscale image, which can reflect the global texture features of an image in different directions and for different distances.

The dimension of the matrix is the grayscale of the image. The relationship between a pair of pixels for constructing the GLCM is shown in Figure 4, where θ is the direction of the gray-level statistics, that is, the angle between the pixel pair with a gray value of $(i, j)$ , and the horizontal direction is the distance between pixels.

Figure 4.

The relationship between a pair of pixels for constructing the gray-level co-occurrence matrix, where $(u, v)$ and $(u + du, v + dv)$ are the coordinates of the pixels in the entire image.

The combination of the uniform multidirectional binary patterns and the gray-level co-occurrence matrix

To obtain the texture characteristics with a strong representation ability, the algorithm first applies the uniform MDBP operator to extract the local texture features of the textile image in four directions and then calculates the GLCM in the corresponding direction to characterize the texture features globally.

The GLCM cannot describe texture features directly. In this paper, five statistical features of the GLCM are calculated to represent the texture features, which are described by the following equations.

Energy. ENE reflect the degree of uniformity of texture, If the texture is simple, ENE will be large

ENE = \sum_{i = 0}^{L - 1} \sum_{j = 0}^{L - 1} P (i, j)^{2}

(7)

Entropy. ENT reflect the thickness of texture. If the texture is simple, ENT will be small

ENT = \sum_{i = 0}^{L - 1} \sum_{j = 0}^{L - 1} P (i, j) log (P (i, j))

(8)

Contrast. CON describes the sharpness of the texture; the value is large when the texture of the image is clear

CON = \sum_{i = 0}^{L - 1} \sum_{j = 0}^{L - 1} (i - j)^{2} P (i, j)

(9)

Homogeneity. HOM reflects the homogeneity of the texture; the value of HOM is large when the local texture is uniform

HOM = \sum_{i = 0}^{L - 1} \sum_{j = 0}^{L - 1} \frac{P (i, j)}{1 + (i - j)^{2}}

(10)

Correlation. COR reflects the linear dependence of the grayscale values. If there is a horizontal texture in the image, the COR of the horizontal matrix is greater than that of the other matrices:

COR = \sum_{i = 0}^{L - 1} \sum_{j = 0}^{L - 1} \frac{ijP (i, j) - μ_{x} μ_{y}}{σ_{x} σ_{y}}

(11)

μ_{x} = \sum_{i = 0}^{L - 1} \sum_{j = 0}^{L - 1} iP (i, j)

(12)

μ_{y} = \sum_{i = 0}^{L - 1} \sum_{j = 0}^{L - 1} jP (i, j)

(13)

σ_{x} = \sum_{i = 0}^{L - 1} \sum_{j = 0}^{L - 1} (i - μ_{x})^{2} P (i, j)

(14)

σ_{y} = \sum_{i = 0}^{L - 1} \sum_{j = 0}^{L - 1} (j - μ_{y})^{2} P (i, j)

(15)

where

P (i, j)

is the initial item value of pixel pairs with gray values of i and j in the GLCM and L is the maximum gray level of the image.

μ_{x}

μ_{y}

σ_{x}

σ_{y}

are the mean and standard deviations, respectively.

By extracting the feature vectors of the GLCM in four directions, a multidirectional texture-feature matrix M of size D × F is obtained, where D represents the number of directions and F is the number of GLCM features. The matrix M contains both the local texture information described by the MDBP operator and the global texture features counted by the GLCM, so it performs well in representing the texture. Table 1 illustrates the feature extraction procession of the textile image combining “uniform” MDBP and the GLCM.

Table 1.

The process of feature extraction combining the uniform multidirectional binary patterns (MDBP) and the gray-level co-occurrence matrix (GLCM)

Four directions:	0 degrees	45 degrees	90 degrees	135 degrees
Resulting image from the MDBP
Relationship between pixel pairs
GLCM results	$[\begin{matrix} 0.88 & 0.95 & \dots & 0.12 \\ 0.90 & 1.39 & \dots & 0.49 \\ : & : & \dots & : \\ 0 & 0.50 & \dots & 21.03 \end{matrix}]$	$[\begin{matrix} 0.07 & 0.23 & \dots & 0.23 \\ 0.19 & 0.89 & \dots & 0.74 \\ : & : & \dots & : \\ 0 & 0.72 & \dots & 48.31 \end{matrix}]$	$[\begin{matrix} 1.27 & 0.94 & \dots & 0 \\ 0.94 & 1.45 & \dots & 0.44 \\ : & : & \dots & : \\ 0 & 0.44 & \dots & 19.46 \end{matrix}]$	$[\begin{matrix} 0.14 & 0.24 & \dots & 0 \\ 0.25 & 0, 89 & \dots & 0.65 \\ : & : & \dots & : \\ 0.22 & 0.63 & \dots & 44.70 \end{matrix}]$
Feature vector	$V_{0} = (\begin{matrix} 0.07 \\ 3.04 \\ 8.67 \\ 0.63 \\ 31.29 \end{matrix})^{T}$	$V_{45} = (\begin{matrix} 0.17 \\ 2.57 \\ 10.61 \\ 0.66 \\ 38.71 \end{matrix})^{T}$	$V_{90} = (\begin{matrix} 0.05 \\ 3.24 \\ 7.01 \\ 0.62 \\ 25.61 \end{matrix})^{T}$	$V_{135} = (\begin{matrix} 0.20 \\ 2.46 \\ 9.96 \\ 0.69 \\ 41.29 \end{matrix})^{T}$
Texture-feature matrix M	$M = [\begin{matrix} 0.07 & 3.04 & 8.67 & 0.63 & 27.66 \\ 0.17 & 2.57 & 10.61 & 0.66 & 38.71 \\ 0.05 & 3.24 & 7.01 & 0.62 & 25.61 \\ 0.20 & 2.46 & 9.96 & 0.69 & 41.29 \end{matrix}]$

The defect detection method

In the algorithm of this paper, fabric-defect detection is mainly carried out according to feature matching of textures between the textile image without defects and the test image. The process of texture-feature extraction was introduced in the previous sections. The detection method is divided into two stages: a texture-feature extraction stage and a fabric-defect detection stage, as shown in Figure 5.

Figure 5.

The defect detection method based on multidirectional binary patterns (MDBP) and the gray-level co-occurrence matrix (GLCM). The defect-free image is III_D (in Figure 10) and the test image is III_D_a_0018 (in Figure 11).

Image preprocessing

Due to the interference of the production environment and the image capture equipment, there is often some noise in the collected fabric images, so it is necessary to preprocess them before extracting the texture features. The preprocessing of texture images mainly includes Gaussian filtering, grayscale conversion, histogram equalization and high contrast retention; the results are shown in Figure 6.

Figure 6.

The preprocessing results of some textile images. The nondefective images are III_D and VI_H (Figure 10), the image with warp-lacking defects is III_D_a_0018 (Figure 11) and the image with shredded defects is VI_H_c_0009 (Figure 11).

The texture-feature extraction stage

The appearance of defects destroys the normal textile texture, so the characteristics of the defects must be quite different from those of the standard features. Therefore, the goal of the first stage is to obtain the detection threshold from the texture features of the nondefective image. The steps of texture-feature extraction are as follows.

Step 1: preprocess the nondefective fabric image.

Step 2: extract the multidirectional texture-feature matrix $M_{F}$ of the entire image by using the texture-feature extraction algorithm combined the uniform MDBP and the GLCM.

Step 3: divide the image into nonoverlapping subimages of size $W_{d} \times W_{d}$ , and extract the texture-feature matrix $M_{i} (i = 0, 1, \dots, H)$ for each subimage, where H is the number of subimages.

Step 4: calculate the similarity S_i between M_i and M_F

S_{i} = \frac{\sqrt{\sum_{x = 0}^{D - 1} \sum_{y = 0}^{F - 1} (m_{xy}^{i} - m_{xy}^{F})^{2}}}{\sqrt{\sum_{x = 0}^{D - 1} \sum_{y = 0}^{F - 1} m_{xy}^{F}}}, (i = 0, 1, \dots, H - 1)

(16)

where

m_{xy}^{i}

and

m_{xy}^{F}

describe the value of the features in the texture-feature matrixes.

Step 5: take the maximum value of S_i as the detection threshold T

T = Max {S_{0}, S_{1}, \dots, S_{H - 1}}

(17)

The larger S_i is, the larger the distance between the two texture-feature matrices and the more dissimilar the two images. T is related only to the type of fabric and the detection environment. The algorithm can automatically determine the detection threshold in different environments. Therefore, the proposed method is adapted to various gray textile images with different characteristics and is robust to a wide variety of image-processing operations.

The fabric-defect detection stage

In the second stage, the defect is detected with the detection threshold T. The multidirectional texture-feature matrix of the entire nondefective fabric image is also needed in this stage.

The steps of the fabric-defect detection stage is as follows.

Step 1: preprocess the textile image to be detected.

Step 2: select the detection window that has the same size as the subimages in the first stage, and adopt the overlap sliding window method from left to right and from top to bottom according to the fixed step size until the detection is completed.

Step 3: extract the multidirectional texture-feature matrix M_t of the detection window W_t by using the texture-feature extraction algorithm combined the uniform MDBP and the GLCM.

Step 4: calculate the similarity S_t between M_t and M_F, according to Equation (16).

Step 5: compare the similarity S_t with the detection threshold T obtained by the first stage, and detect the current detection window W_t according to the comparison result, as shown in Equation (18)

W_{t} = {\begin{matrix} defect, & S_{t} > T \\ undefect, & S_{t} \leq T \end{matrix}

(18)

Step 6: if the detection of the test image is not complete, return to Step 2 and move the detection window.

A diagram of the defect detection method for unpatterned fabric based on MDBP and the GLCM is shown in Figure 7.

Figure 7.

Diagram of the defect detection method for unpatterned fabric based on multidirectional binary patterns (MDBP) and the gray-level co-occurrence matrix (GLCM).

The validity of the detection method

To indicate the validity of the detection method, this paper selected a nondefective fabric image and obtained the detection threshold (T = 0.60). Then, an image of the same fabric but with warp-lacking defects was divided into nonoverlapping subimages. These subimages were assigned unique labels from top to bottom and from left to right. Figure 8 shows the assignment of labels to partial subimage windows.

Figure 8.

The assignment of labels to partial subimage windows of an image with warp-lacking defects (III_D_a_0018, Figure 11). The nondefective image is III_D (Figure 10).

The similarity values of all the subimages and the defect-free fabric image were calculated, as shown in Figure 9.

Figure 9.

The similarity values of all subimages.

In Figure 9, the horizontal axis is the labels of subimages, the vertical axis represents the similarity values between each of the subimages and the defect-free image and the horizontal line represents the detection threshold of T = 0.60. The subimages whose similarity values are larger than T are marked with a rhombus. For example, the similarity values of the defective subimages labeled 88, 89, 104 and 105 are 0.71, 0.67, 1.02 and 0.66, respectively (Figure 8). We observe that all of the marked subimages are defective. Therefore, the threshold is effective for detecting defects.

Dataset

The unpatterned fabric images used in the experiment were collected with a Dalsa industrial linear scanning camera (PX-HC-16K04T) and Dalsa frame grabbers (Xtium-CLHS PX8) in a fabric production workshop. The textures of the fabric are classified as tabby, twill, high-density twill, high-elastic textile, linen, flannelette and woolen textiles, which are labeled in order from I to VII, as shown in Figure 10. The size of the fabric image collected by the camera is 1024 × 1024, the image patch detection method is adopted in the actual detection system, the size of images used in the detection method is 256 × 256 and the size of the subimages and detection window is 16 × 16.

Figure 10.

Examples of the unpatterned, nondefective fabric images included in the experiment, where I–VII represent the material of the fabric, which is tabby, twill, high-density twill, high-elastic textile, linen, flannelette, and woolen textiles, respectively. A–I are the color labels.

For each type of fabric, the defect-free images are selected as the standard images used in the first stage, and the other images are used as test images. The defects can be divided into five types: warp-lacking, weft-lacking, shredded, oil and scuffed defects; they are labeled with the letters a–e in order (Figure 11).

Figure 11.

Examples of the unpatterned, defective fabric images included in the experiment. The letters a–e represent the types of defects (warp-lacking, weft-lacking, shredded, oil and scuffed defects, respectively). The figure label is the image identifier. For example, image III_D_a_0018 has warp-lacking defects, and the corresponding defect-free image is III_D (Figure 10).

Experimental analysis and results

Three main parameters affect the detection result in the defect detection method: the number of pixels N in the linear neighborhood of the MDBP operator, the distance d in the GLCM and the sliding step length s in the detection stage. To choose appropriate parameter values, the influence of each parameter value will be discussed. Subsequently, four different detection methods are compared.

The detection accuracy rate C is taken as the parameters for evaluation

C = \frac{TD + TU}{TD + FD + TU + FU}

(19)

In Equation (19), TD represents the number of defective windows detected correctly and TU is the correct number of nondefective windows. FU is the number of defective windows that are detected as nondefective and FD is the number of nondefective windows that are also detected incorrectly. A higher value for C indicates a better detection effect.

The number of pixels N in the linear neighborhood

The number of pixels N in the linear neighborhood of the MDBP operator should adapt to the thickness of the fabric. If N is small, the MDBP operator reveals only the texture features in a small area, which can easily cause misclassifications. If N is too large, the number of MDBP operator-based patterns will increase, resulting in a high computational complexity. Therefore, this section analyzes the detection results of two images in the cases of N = 3, N = 5 and N = 8, as shown in Figure 12 (d = 4 and b = 3).

Figure 12.

The defect-detection results of the multidirectional binary patterns operator with different neighborhoods, where the nondefective images are III_D and VI_H (Figure 10) and the test images are III_D_a_0018 and VI_H_c_0009 (Figure 11).

The results show that when N = 3, a missed detection occurred in the defect area, so the accuracy rate is low. When N = 5 and N = 8, the operator shows good results in the defect region, but the accuracy rate is higher when N = 5. Therefore, the value of N in this algorithm is set to 5. Table 2 shows the detection accuracy rate C of the MDBP operator with different neighborhoods for the five types of defects.

Table 2.

Detection accuracy rate of multidirectional binary patterns (MDBP) operators with different neighborhoods

MDBP operator	Detection accuracy rate C
N	(a)	(b)	(c)	(d)	(e)
3	96.43	96.36	96.15	96.40	94.28
5	97.27	97.79	96.50	97.10	95.80
8	96.89	96.90	96.25	96.80	95.18

In Table 2, (a) represents an image with warp-lacking defects, (b) represents an image with weft-lacking defects, (c) represents an image with shredded defects, (d) represents an image with oil defects and (e) represents an image with scuffed defects.

The results show that the detection effects of the MDBP operator with N = 5 are better than those of the other two neighborhoods, with an average detection accuracy rate of 97%.

The distance d in the GLCM operator

In the process of constructing the GLCM, the distance d directly affects the grayscale statistics of the texture. If d is small, the runtime will be long. The accuracy rate of the texture extraction procedure is reduced when d is large, which will result in a weak detection ability.

The defective subimages of the textile image with warp-lacking defects (III_D_a_0018, Figure 11) are obtained according to the analysis process of the validity of the detection results (Figure 8); then, the mean values of the GLCM features in the four directions are calculated. The feature value changes under distances between d = 1 and d = 12 are obtained, as shown in Figure 13 (N = 5 and b = 3).

Figure 13.

Each feature value change under different values of d.

As shown in Figure 13, for each feature, the value changes significantly when d changes from 1 to 3. When d is between 4 and 7, the change in the feature values is relatively stable, which indicates that d has less influence on the texture features of the defective area in this interval than in other intervals, so the algorithm performance is better. When d is greater than 7, the value of the features begins to change gradually. Therefore, d is set to 4 in this algorithm.

Step b for sliding detection

It is important to select a reasonable sliding step size b in the defect detection stage. If b is small, the computational complexity of the method will be high. If b is large, missed detections will occur in a partial area.

For the two defective images, the detection effects for values of b ranging from 1 to 16 are analyzed. Figure 14 shows the result images only when b is even (N = 5 and d = 4).

Figure 14.

The detection results of defective images at different sliding step sizes b: (a) detection results for the image with warp-lacking defects (III_D_a_0018, Figure 11); (b) detection results for the image with shredded defects (VI_H_c_0009, Figure 11).

According to the detection results, when b is small, the defective area can be accurately detected, but a small part of the normal area is also detected as defect, which reduces the detection accuracy rate. As b increases, the detection effect significantly weakens, although the detection speed increases. For the sake of comprehensive consideration, b = 4.

The detection accuracy rates C for the five types of defects under different b values are shown in Figure 15. C increases gradually when b is small, but decreases significantly when b is greater than 4. Therefore, it is reasonable to set b to 4.

Figure 15.

Detection accuracy rate for the five types of defects under different step sizes.

Comparison of different detection methods

The detection results of four methods are compared in this section: the LBP-based detection method, the neighbor-based binary pattern (NBP)-based detection method, the GLCM-based detection method and the MDBP/GLCM detection method proposed in the paper.

In general, the four detection methods have similar processes, except the steps of extracting the texture features and calculating the similarity. In the LBP-based detection method, the original LBP operator with a circular neighborhood is applied to describe the texture features of the fabric image, and the similarity between the gray-level histograms is calculated to detect defects. In the NBP-based detection method, the texture description operator is the NBP, in which the binary sequence is obtained by considering the gray correlation information between neighboring pixels in a square neighborhood.²⁷ The encoding mode of the NBP operator is similar to that of the MDBP operator, but somewhat different, especially the selection of the neighborhood. In the GLCM-based detection method, the texture-feature vector is obtained by adopting the GLCM method, and the Euclidean distance between the vectors is calculated as the similarity value of the textile images. Figures 16 and 17 show the detection results of these four methods.

Figure 16.

Comparison of the four detection methods. The defective images include an image with warp-lacking defects (III_D_a_0018), an image with weft-lacking defects (I_B_b_0061) and an image with shredded defects (VI_H_c_0009) (Figure 11). LBP: local binary patterns; NBP: neighbor-based binary pattern; GCLM: gray-level co-occurrence matrix; MDBP: multidirectional binary patterns.

Figure 17.

Comparison of the four detection methods. The defective images include an image with oil defects (IV_F_d_0005), an image with scuffed defects (V_E_e_0078) and an image with weft-lacking defects (II_B_a_0025) (Figure 11). LBP: local binary patterns; NBP: neighbor-based binary pattern; GCLM: gray-level co-occurrence matrix; MDBP: multidirectional binary patterns.

Figures 16 and 17 illustrate that the detection effect of the proposed method is better than those of the other three detection methods. There is a missed detection in the LBP-based method because that the LBP operator insufficiently considers the direction, which makes it difficult to describe the detailed features of the texture. The NBP operator is sensitive to some textures in nondefective areas, so the misjudgment rate is high. For the GLCM-based detection algorithm it is difficult to extract features of fine textures, so the detection accuracy rate is lower. The MDBP/GLCM detection method describes both the local and global information of the texture in four directions, so it achieved significant detection results.

Some of the fabric defects collected by the camera are not obvious and are directional, such as the image with weft-lacking defects in Figure 17. In the LBP-based method, only a few defective pixels fall in the neighborhood, so the defective pixels have a smaller weight when calculating the LBP value. The GLCM-based method statistics measure the global distribution of the gray level and the values of the defective pixel pairs in the matrix are smaller than those normal pixels, so the extracted texture features have no obvious ability to distinguish between defects and nondefects. Therefore, these two methods cannot detect weak weft-lacking defects (Figure 17). However, the proposed MDBP operator uses the linear neighborhood to encode the image in multiple directions, which shows a good detection ability by greatly enhancing the texture expression of the defects. For the weft-lacking defects (Figure 17), the linear neighborhood within 90 degrees of the MDBP operator has an obvious texture enhancement effect and better detection ability than the circular neighborhood of the LBP operator, as illustrated in Figure 18.

Figure 18.

The resulting images from the multidirectional binary patterns (MDBP)-based method at 90 degrees and from the local binary patterns (LBP)-based method. The image with weft-lacking defects is II_B_a_0025 (Figure 17).

The average detection accuracy rate C of each method for the velvet chiffon fabric images of the dataset is shown in Table 3, where the average accuracy rate C is the mean value of the five types of defects.

Table 3.

Average detection accuracy rate of each method for one type of fabric

	Images in the dataset			Detection results
Type of textile	Type of defects	Defective	Number	Value	LBP	NBP	GLCM	MDBP/GLCM
flannelette	Warp-lacking	Yes	360	TD	316	329	312	351
		Yes	360	FU	44	31	48	9
		No	140	TU	115	120	127	138
		No	140	FD	25	20	13	2
		Evaluation		C (%)	86.20	89.80	87.80	97.80
	Weft-lacking	Yes	350	TD	293	302	290	343
		Yes	350	FU	57	48	60	7
		No	150	TU	123	129	128	145
		No	150	FD	27	21	22	5
		Evaluation		C (%)	83.20	86.20	83.60	97.60
	Shredded	Yes	330	TD	283	291	279	320
		Yes	330	FU	47	40	51	10
		No	170	TU	155	154	160	163
		No	170	FD	15	16	10	7
		Evaluation		C (%)	87.60	89.00	87.80	96.60
	Oil	Yes	380	TD	320	334	311	375
		Yes	380	FU	60	46	69	5
		No	120	TU	106	102	111	112
		No	120	FD	14	18	9	8
		Evaluation		C (%)	85.20	87.20	84.40	97.40
	Scuffed	Yes	270	TD	204	229	206	260
		Yes	270	FU	66	41	64	10
		No	230	TU	212	204	201	223
		No	230	FD	18	26	29	7
		Evaluation		C (%)	83.20	86.60	81.40	96.60
	Average accuracy rate C (%)				85.08	87.76	85.00	97.2

LBP: local binary patterns; NBP: neighbor-based binary pattern; GCLM: gray-level co-occurrence matrix; MDBP: multidirectional binary patterns. TD: the number of defective images that are detected correctly; FU: the number of defective images that are detected as nondefective; TU: the number of nondefective images that are detected correctly; FD: the number of nondefective images that are detected as defects.

Figure 19 shows the average detection accuracy C of the four detection methods for all types of fabrics in the dataset. Obviously, the detection method combining the MDBP operator and the GLCM can successfully detect more than 97% of the defective images in the dataset.

It is worth mentioning that for some fabric images, there are certain difficulties for defect detection. This is related to fabric texture and the causes and size of defects. Figure 20 shows some of the failure cases of applying the proposed method for defect detection in Table 3.

Figure 19.

The average detection accuracy C of the four detection methods for all types of fabrics in the dataset. LBP: local binary patterns; NBP: neighbor-based binary pattern; GCLM: gray-level co-occurrence matrix; MDBP: multidirectional binary patterns.

Figure 20.

The failure cases of applying the proposed method for defect detection. There are scuffed defects in these two textile images.

The reason why the detection method is ineffective for these two images is that the texture of the fabric is more complicated and uneven and, what is more, the texture of the defect is similar to the normal texture of the fabric image, resulting in the similarity between the multidirectional texture-feature matrix of the defective area and the matrix of the nondefective area being less than the detection threshold. These problems will be studied in the follow-up research work.

Conclusion

To overcome the disadvantages of the original LBP operator, the MDBP, a texture description operator, was proposed in this paper. This operator has enhanced texture expression abilities by focusing on the gray-level distribution between the linear neighboring pixels in an image in multiple directions. The modified MDBP operator with a uniform pattern is further proposed to reduce the grayscale of the textile image; then, the features of the GLCM statistical method are combined to describe the global texture of the textile image.

This paper proposed a defect detection schema for unpatterned fabric based on the above texture-feature extraction process. In the texture-feature extraction stage, the multidirectional texture-feature matrix of the nondefective fabric image is extracted first, and then the detection threshold is determined based on the similarity between the feature matrices. In the detection stage, the defect is detected with the detection threshold calculated in the first stage.

This paper also analyzed the influence of three important parameters on the defect detection results. In addition, the detection results of the proposed method are compared with those of the LBP-based method, the NBP-based detection method and the GLCM-based method, which shows that the proposed algorithm has a better detection ability than the other two methods.

The proposed method is adapted to various grayscale textile images with different characteristics and is robust to a wide variety of image-processing operations. In addition, it is invariant to grayscale changes, performs well in representing textures and detecting defects and has lower computational complexity than the other methods. So, this research has good applicability and adaptability to the detection of defects in different types of fabrics under different production environments, especially for the detection of small defects, which will greatly improve the quality and detection efficiency of the fabric.

Footnotes

Declaration of conflicting interests

The authors declare no potential conflicts of interest with respect to the research, authorship and/or publication of this article.

Funding

The authors disclose receipt of the following support for the research, authorship, and/or publication of this article: This work was supported by the National Key R&D Program of China (grant no. 2017YFB0309800), the Graduate Student Innovation Fund of Donghua University (grant no. GSIF-DH-M-2019011) and the Intermittent Dyeing and Finishing Green Smart Factory Project initiated by the China Ministry of Industry and Information Technology’s Intelligent Manufacturing Integrated Standardization and New Mode Application project.

References

Stojanovic

Mitropulos

Koulamas

, et al. Real-time vision-based system for textile fabric inspection. Real Time Imag 2001; 7: 507–518.

Tajeripour

Kabir

Sheikhi

. Fabric defect detection using modified local binary patterns. EURASIP J Adv Signal Process 2008; 2008: 1–12.

Cho

Chung

Park

. Development of real-time vision-based fabric inspection system. IEEE Trans Ind Electron 2005; 52: 1073–1079.

Ralló

Millán

Escofet

. Referenceless segmentation of flaws in woven fabrics. Appl Optics 2007; 46: 6688–6699.

Li PF, Zhao Z, Zhang L, et al. The real-time vision system for fabric defect detection with combined approach. In: international conference on image and graphics (ed Yu-Jin Zhang), Tianjin, China, 13–16 August 2015, paper no. LNCS 9219, pp.460–473. Cham: Springer.

Haralick

Shanmugam

Dinstein

. Textural features for image classification. IEEE Trans Syst Man Cybern 1973; 3: 610–621.

Davis

Johns

Aggarwal

. Texture analysis using generalized co-occurrence matrices. IEEE Trans Pattern Anal Mach Intell 1979; 1: 251–259.

Asha

Bhajantri

Nagabhushan

. GLCM-based chi-square histogram distance for automatic detection of defects on patterned textures. IJCVR 2011; 2: 302–313.

Arnia F and Munadi K. Real time textile defect detection using GLCM in DCT-based compressed images. In: international conference on modeling, simulation and applied optimization (eds Ismail Kucuk, Faruk Yigit and Aydin Yesildirek), Istanbul, Turkey, 27–29 May 2015, paper no. ICMSAO.2015.7152203, pp. 1–6. Turkey: IEEE.

10.

Latif-Amet

Ertüzün

Erçil

. An efficient method for texture defect detection: sub-band domain co-occurrence matrices. Image Vis Comput 2000; 18: 543–553.

11.

Tsang

CSC

Ngan

HYT

Pang

GKH

. Fabric inspection based on the Elo rating method. Pattern Recognit 2016; 51: 378–394.

12.

Kang

Zhang

. A universal defect detection approach for various types of fabrics based on the Elo-rating algorithm of the integral image. Text Res J 2019; 0: 1–28.

13.

Cohen

Fan

Attali

. Automated inspection of textile fabrics using textural models. IEEE Trans Pattern Anal Mach Intell 1991; 13: 803–808.

14.

Ozdemir S and Ercil A. Markov random fields and Karhunen-Loeve transforms for defect inspection of textile products. In: proceedings of the 1996 IEEE conference on emerging technologies and factory automation (ed Hubert Wo), Kauai, USA, 18–21 November 1996, paper no. ETFA.1996.573989, pp. 697–703. Piscataway: IEEE.

15.

Zhang

Jiang

Yao

, et al. Intelligent segmentation of jacquard warp-knitted fabric using a multiresolution Markov random field with adaptive weighting in the wavelet domain. Text Res J 2014; 84: 28–39.

16.

Hajimowlana SH, Muscedere R, Jullien GA, et al. 1D autoregressive modeling for defect detection in web inspection systems. In: Midwest symposium on circuits and systems (ed Joseph Daigle), Notre Dame, USA, 9–12 August 1998, paper no. MWSCAS.1998.759496, pp. 318–321. Piscataway: IEEE.

17.

Chan

Pang

GKH

. Fabric defect detection by Fourier analysis. IEEE Trans Ind Appl 2000; 36: 1267–1276.

18.

Kumar

Pang

GKH

. Defect detection in textured materials using Gabor filters. IEEE Trans Ind Appl 2002; 38: 425–440.

19.

Beirão CL and Figueiredo MAT. Defect detection in textile images using Gabor filters. In: 1st international conference on image analysis and recognition (eds Aurélio Campilho and Mohanmed Kamel), Porto, Portugal, 29 September–1 October 2004, paper no. LNCS 2312, pp.841–848. Berlin: Springer.

20.

Arivazhagan

Ganesan

Bama

. Fault segmentation in fabric images using Gabor wavelet transform. Mach Vis Appl 2006; 16: 356–363.

21.

Kuo

CFJ

Shih

Huang

, et al. Image inspection of knitted fabric defects using wavelet packets. Text Res J 2015; 0: 1–8.

22.

Zhu

Liu

Pan

, et al. Seam detection of inhomogeneously textured fabrics based on wavelet transform. Text Res J 2015; 85: 1381–1393.

23.

Ojala T, Pietikäinen M and Mäenpää T. Gray scale and rotation invariant texture classification with local binary patterns. In: 6th European conference on computer vision (ed David Vernon), Dublin, Ireland, 26 June–1 July 2000, paper no. LNCS 1842, pp. 404–420. Berlin: Springer.

24.

Liao

Law

MWK

Chung

ACS

. Dominant local binary patterns for texture classification. IEEE Trans Image Process 2009; 18: 1107–1118.

25.

Guo ZH, Zhang L, Zhang D, et al. Rotation invariant texture classification using adaptive LBP with directional statistical features. In: 2010 IEEE international conference on image processing (eds Bonnie Law), Hong Kong, China, 26–29 September 2010, paper no. ICIP.2010.5652209, pp. 285–288. Piscataway: IEEE.

26.

Guo

Zhang

. A completed modeling of local binary pattern operator for texture classification. IEEE Trans Image Process 2010; 19: 1657–1663.

27.

Hamouchene I and Aouat S. A cognitive approach for texture analysis using neighbors-based binary patterns. In: 13th IEEE international conference on cognitive informatics and cognitive computing (eds Patel S, Wang Y, Kinsner W, et al.), London, UK, 18–20 August 2014, paper no. ICCI-CC 6921447, pp. 94–99. Piscataway: IEEE.

28.

Fieguth

Guo

, et al. Local binary features for texture classification: taxonomy and experimental study. Pattern Recognit 2017; 62: 135–160.

29.

Shakoor

Boostani

. Radial mean local binary pattern for noisy texture classification. Multimed Tools Appl 2018; 77: 21481–21508.

30.

Xue

Cheng

. Intelligent detection of defects of yarn-dyed fabrics by energy-based local binary patterns. Text Res J 2012; 82: 1960–1972.

31.

Kumar

. Neural network based detection of local textile defects. Pattern Recognit 2003; 36: 1645–1659.

32.

Zhou

Wang

. Fabric defect detection using adaptive dictionaries. Text Res J 2013; 83: 1846–1859.

33.

Kang ZQ, Yuan CH and Qian Y. The fabric defect detection technology based on wavelet transform and neural network convergence. In: 2013 IEEE international conference on information and automation (eds Gu J, Hu C and Li B), Yinchuan, China, 26–28 August 2013, paper no. ICInfA 6720367, pp. 697–703. Piscataway: IEEE.

34.

Wei B, Hao KR, Tang XS, et al. Fabric defect detection based on faster R-CNN. In: international conference on artificial intelligence on textile and apparel (ed Wai Keung Wong), Hong Kong, China, 3–6 July 2018, paper no. AISC volume 849, pp. 45–51. Cham: Springer.

35.

Mei

Wang

Wen

. Automatic fabric defect detection with a multi-scale convolutional denoising autoencoder network model. Sensors 2018; 18: 1064–1064.