Abstract
Detection of yarns in fabric images is a basic task in real-time monitoring in fabric production processes since it relates to yarn density and fabric structure estimation. In this paper, a new detection method is proposed that can automatically and efficiently estimate the locations as well as the numbers of both weft and warp yarn in fabric images. The method has three sequential phases. First, the modulus of discrete partial derivatives at each pixel is projected onto the weft and warp directions to generate the accumulated histograms. Second, for each histogram, a monotone hypothesis of a nonparametric statistical approach is applied to segment the histogram. Third, according to the segmentation result, the locations of each weft and warp yarn are adaptively determined, while the fabric structure is also obtained. Numerical results demonstrate that, compared with classical yarn detection methods, which are based on image smoothing, the proposed method can estimate yarn locations and fabric structures with more accuracy, but also reduce the influence of yarn hairiness.
Image-processing technology is being used in the textile industry to gradually develop methods involving automation and intelligence. The technology is widely used in fabric design1–3 and fabric quality inspection, such as estimating yarn densities4–7 and identifying weave patterns of woven fabrics.8–10 Here, yarn density is the numbers of warp/weft yarns in a unit length, and weave pattern is determined by yarn locations and the crossing rule of yarns. Both of these are the main structure parameters of woven fabrics and are related to the appearance and some physical properties of woven fabrics. When detecting yarn density and weave pattern, they refer to determining the locations of warp and weft yarns. Since it is laborious to count yarn manually, textile researchers are focused on automatic methods for estimating yarn locations in fabric images.
The methods for detecting yarns can be divided into the following two categories. (a) Frequency domain methods.4,5,8,11–15 As a woven fabric image has a periodic structure, Fourier transform4,5,8,11,12 and its inverse are used to reconstruct fabric images containing only weft or warp yarns, and then yarn locations are determined by using image analysis methods. Similarly, a wavelet transform13–15 can also be a way to extract the boundaries of weft and warp yarns and to estimate yarn locations, where the information on yarn locations is represented by horizontal and vertical high frequency coefficients. Compared with the Fourier transform, the wavelet transform has lower computation complexity and is more efficient. (b) Spatial domain methods. Gray projection6,16–20 is the main spatial domain method used to detect yarn locations. After projecting pixels in a fabric image along the weft or warp directions, yarns and interstices are located on the local maximum and local minimum of the gray projection.
These methods can successfully detect yarns in different fabric images. However, some procedures in these methods are relatively complicated and easily influenced by random yarn texture noise 21 caused by yarn hairiness. To reduce the influence of texture noise, smoothing-based methods10,22 are proposed to smooth fabric textures. Yet the methods may excessively smooth the yarns in fabric images and they may be unsuitable for fabric images with dark-color yarns. 22 To overcome this difficulty, in this paper we propose a non-smoothing yarn detection method based on histogram segmentation. Our method can estimate the number as well as the locations of both weft and warp yarns in fabric images automatically. It is worth noting that, originally, the histogram segmentation algorithm proposed by Delon et al.23–25 is used to achieve image segmentation and color extraction. In this paper, we generalize the algorithm to deal with the yarn detection problem. First, instead of applying the rgb2gray function in MATLAB, we convert a color fabric image into its grayscale image using the method presented by Lu et al., 26 which can extract boundaries of yarn with greater accuracy. Second, the modulus of discrete partial derivatives of the gray intensity at each pixel is accumulated along the horizontal/vertical directions to form a horizontal/vertical statistical histogram. Here, the horizontal/vertical statistical histogram indicates all horizontal/vertical boundaries of yarns in the original fabric image. Then, based on a so-called monotone hypothesis on histograms, we estimate all yarn locations in the fabric image by partitioning the histograms. Compared with the smoothing-based method, our method can better estimate yarn locations.
Related work
Fourier transform- and wavelet transform-based methods
In Fourier transform-based methods, Xin and Yu 4 apply peak filtering to extract peak points in Fourier spectra of fabric images, and obtain relatively accurate yarn locations for fabrics with clear yarn boundaries. However, as their method is based on a one-sided imaging system, information about woven fabrics are usually lost, which makes the method hard to use where yarn boundaries are blurred. To overcome this drawback, Xin et al. 8 further proposed a dual-side scanning method by using the technique of the active grid model. Following the work of these authors, 8 Zhang et al. 5 used radon transform for image registration in dual-side scanning, which can enhance the accuracy of yarn location detection. Similarly, Pan et al.11,12 apply Fourier transforms for constructing images containing only warp or weft yarns, and yarn locations are determined by thresholding segmentation. Furthermore, for high-tightness fabric images, a so-called structure relation of yarns is inspected to estimate yarn locations. 12
Based on multi-resolution analyses, the discrete wavelet transform (DWT) methods can be also be used to extract the information on warp and weft yarns. Feng and Li 13 apply a two-dimensional DWT to extract the frequency of yarn boundaries and automatically locate yarns in fabric images. Considering that a non-adaptive wavelet is difficult to suit to different yarn scales in fabric images, He et al. 14 introduced an adaptive wavelet decomposition filter to estimate yarn boundaries. Additionally, Jing 15 proposed a yarn detection method based on multiscale DWT and morphology processing. The decomposition and reconstruction filters generated from spline biorthogonal wavelet 3.7 are applied to obtain and to locate outlines of yarns via the Bernsen algorithm and smooth processing.
As color patterns 16 influence the frequencies of yarns, Fourier transform- and wavelet transform-based methods have the drawback that yarn locations in color fabrics cannot be well detected. Thus, these methods are mostly used for solid-color woven fabrics. To avoid the drawback, in this paper we focus on spatial domain methods to detect yarns in fabric images.
Gray projection methods
Gray projection methods are constructed in the spatial domain rather than in the frequency domain. The main principles of these methods are to detect skew angles of yarns and to find local maxima/minima of projection curves. Pan et al. 16 proposed that skew angles of yarns can be detected by a Hough transform, and grayscale values of fabric images can be projected along skew angles to obtain projection curves. Their method is suitable for solid-color fabrics, but not for fabric images with more than two colors. To overcome this drawback, Zhang et al. 6 proposed using the Otsu projection algorithm to inspect yarn locations. In their method, an optical system is used to generate transmission images from fabric images, which can reduce the influence of color on yarn detection. Pan et al. 17 first segmented yarn-dyed fabric images into different homogeneous color regions by using a fuzzy clustering method (FCM). Then, the gray projection method of their earlier work 16 is used to detect yarn locations in each region. However, the method is unsuitable for fabric images in the following cases: first, when a homogeneous region is disconnected; and second, the differences of colors in different regions are small. In addition, nonlinear deformations of yarns are not considered during the detection process of skew angles in either of Zhang et al.’s or Pan et al.’s methods. To deal with these problems, Zhang et al. in further work 18 designed a new skew-angle detection method for nonlinear deformation using two-dimensional discrete Fourier transform. Zheng et al. 7 proposed using a structure-and-texture decomposition method to detect yarn skewness, and then lightness-intensity gradients in fabric images are projected along yarn skewness to obtain projection curves. Following Zheng’s work, Aldemir et al. 19 proposed using a Gabor filter to extract textures and generate projection curves by gray line profiles of textures.
Classical gray projection methods are also sensitive to yarn random texture noise. 21 To reduce the noise, smoothing-based detection methods are proposed.10,22 A color bilateral filter and steering filter are used to smooth fabric random textures as well as to enhance boundary information of yarns. Then, yarns are detected in the enhanced images by using gray projection methods. However, in smoothing-based methods, parameters are not easily determined, which may cause yarns in fabric images to be over-smoothed. Furthermore, the reflection theory 27 of gray projection methods is not suitable for dark-color yarns. To solve these problems, in this paper a non-smoothing method based on histogram segmentation is proposed, and the boundary information of fabric images is used to generate statistical histograms. These histograms can better reflect periodic structures of fabric images than projection curves. Thus, our method can eliminate the influence of yarn hairiness and dark-color yarns and achieve yarn detection for various fabric images.
Our method
This paper proposes a yarn detection method based on histogram segmentation; the flow chart of our method is shown in Figure 1. The method includes four steps: the conversion of color fabric images, histogram generation, histogram segmentation and the estimation of yarn locations. After converting a color fabric image into a grayscale image, the boundary information of the fabric image is extracted, and the values are projected along the horizontal/vertical direction on an accumulated histogram. Here, as the histogram can reflect periodic structures of the fabric image, yarn locations can be estimated by histogram segmentation. Then, in the histogram segmentation step, the histogram is divided into several unimodal parts by the monotone hypothesis, introduced later. Finally, according to segmentation results, yarn locations are estimated by the peak in each part. More details of our method are introduced in the following sections.
Flow chart of the proposed method.
Accumulated partial derivatives histograms
As the conversion method of Lu et al.
26
can preserve more detail from the color images than the rgb2gray function in MATLAB, we use it to convert a color fabric image into its grayscale image
For example, Figure 2 shows a fabric image and its accumulated partial derivative histograms in the warp and weft directions. Both histograms exhibit several peaks at some intervals, and indicate periodic structures of the fabric images. Hence, yarn locations can be estimated according to the correspondence between a fabric image and its accumulated partial derivative histograms.
The accumulated partial derivatives histograms of a plain weave fabric image: (a) the original image (256 × 256 pixels); (b) and (c) the accumulated partial derivative histograms in the warp and weft directions, respectively.
Additionally, both gradient magnitudes and the modulus of the discrete partial derivatives can extract the information of weft/warp yarn boundaries, but the modulus of the discrete partial derivatives can better reflect the periodic structures. As shown in Figure 3, in the vertical accumulated partial derivatives histogram, the periodicity is more obvious.
The accumulated histograms of a plain weave fabric image in warp and weft directions: (a) the original image; (b) the accumulated gradient magnitude histograms; (c) the accumulated partial derivative histograms.
Histogram segmentation
Ideally, in an accumulated partial derivative histogram of a fabric image, peaks (local maxima) correspond to yarn boundaries, and valleys (local minima) correspond to yarn centers. This means that the value of the accumulated partial derivative decreases from yarn boundary to yarn center, while it increases from yarn center to yarn boundary. Thus, because of the periodic structures of the fabric image, the accumulated partial derivative histogram can be divided into several unimodal parts, with each part consisting of a monotonically increasing part and a monotonically decreasing part.
However, influenced by yarn hairiness and color pattern, the accumulated partial derivatives histogram is “statistically unimodal.”
23
This means the histogram is likely unimodal in some intervals. Hence, not all local maxima and local minima indicate yarn locations. For example, in Figure 4(a), the red part is the accumulated partial derivative from one yarn center to another, and corresponds to the red part in Figure 4(b). The red histogram is likely to be unimodal, that is its shape is similar to the green dotted line in Figure 4(c). There are several peaks and valleys in the histogram, such as the yellow circles and black circles in Figure 4(d), but the boundaries and centers of yarns are indicated by the abscissa of black circles instead of yellow circles. Thus, to find the right peaks and valleys, the monotone hypothesis
23
is used to achieve histogram segmentation.
The correspondence between a fabric image and its accumulated partial derivatives histogram: the red part in (a) corresponds to the red parts in (b); (c) and (d) show the local enlarged part of (a).
The monotone hypothesis in
23
is derived from a parameter-free method proposed by Desolneux et al.,
28
and the number of false alarms (NFA)28–31 is the test statistic. Here, NFA is an indicator of the “meaningfulness” of an event, and its definition and specific function are shown in the Appendix. Under an assumption that the distribution of an object is monotonically decreasing/increasing, if the NFA is more than a given parameter ɛ, the objects can be grouped together. For example, Figure 5 shows the segment result of the red part in Figure 4(a) obtained by the monotone hypothesis. As shown in Figure 5(b), the accumulated partial derivative in the red part is monotonically increasing, and the one in the green part is monotonically decreasing. The black star in Figure 5(b) corresponds to the red boundary in Figure 5(c). Therefore, yarn locations in fabric images can be estimated by the monotone hypothesis.
The result of the monotone hypothesis: (a) is the accumulated partial derivative histogram in Figure 4(c); (b) is the result of (a) obtained by the monotone hypothesis; and the black star in (b) corresponds to the red boundary in (c).
The details of the monotone hypothesis
Let
Assume that P(M) is the space of normalized probability distributions on {1, 2, … , M} and I(M) is the space of increasing densities on {1, 2, … , M}. The monotone hypothesis H0 is that, in the interval [xa, xb] (1 ≤ a, b ≤ M), the accumulated partial derivative histogram
The relative entropy between two Bernoulli distributions of parameters s(a, b) and p(a, b) is
Hence, given a parameter
In addition, the relative entropy The Grenander estimator of the histogram obtained by the pool adjacent violators algorithm (PAVA): (a) the accumulated partial derivative histogram of a plain weave fabric image; (b) the Grenander estimator of (a).
After performing the monotone hypothesis on all intervals of the histogram Nonparametric histogram segmentation for yarn detection Perform monotone hypothesis on the interval [si, si+3] (i = 1, 2, … , length( Update Stop when Algorithm 1
Numerical results
In this section, we inspect the yarn locations in several typical fabric images to illustrate the effectiveness of our proposed method and compare our method with the smoothing-based method. Both methods are implemented in MATLAB R2017a on a 1.86-GHz Intel dual-core computer. All testing fabric images are provided from the GAMA Lab of the Hong Kong Polytechnic University, including twill, basket weave and plain weave fabric images with RGB true color mode.
Automatic selection of parameters
As shown in Figure 7, the value of the parameter ɛ can influence detection results in a fabric image. Let The influence of parameter e on yarn detection: (a) the original image (256 × 256 pixels); (b)–(d) the results of yarn detection with different e. (b) ewarp = 28, ewetf = 1; (c) ewarp = 28, ewetf = 10; (d) ewarp = 28, ewetf = 17.
Detection of the warp and weft yarn
Several fabric samples are used to illustrate the effectiveness of our automatic method. Figure 8 displays the detection results in a basket weave fabric image, obtained by our method and the smoothing-based method, where (a) shows the input image, (b) shows the segmentation results of the accumulated partial derivative histograms in vertical and horizontal directions, and (c) and (d) show the locations of each warp and weft yarn obtained from the two methods.
The location of each yarn obtained from different methods: (a) the input fabric image (256 × 256 pixels); (b) the segmentation results of the accumulated partial derivative histogram; (c) and (d) the yarn locations obtained from the smoothing-based method (c) and our method (d).
However, as shown in Figure 9, there are some misjudgments and missed judgments of yarn locations by the smoothing-based method, while our yarn detection method gets the correct estimation results for the fabric images.
Comparison of yarn detection results obtained from different methods: (a) and (b) the smoothing-based method; (c) and (d) our method.
Furthermore, we estimate the locations of the warp and weft yarns in six fabric images, and the detection results are shown in Figure 10, where samples 1 and 2 are the twill fabric images, samples 3 and 4 are the basket weave fabric images and samples 5 and 6 are the plain weave fabric images. The numbers of yarns detected by the smoothing-based method and our method are listed in Table 1. Compared with the smoothing-based method, our method generates better results.
The detection results of different fabrics obtained from our method, where (a)–(f) are samples 1–6. Vertical and horizontal yarn number of different fabric images
The accuracy of yarn detection
In addition, to verify the robustness of our method in yarn position detection, we add Gaussian noise to the fabric images. In all the experiments, the mean m of the noise is 0 and the variance v is the parameter controlling the noise. For example, Figure 11(a)–(c) show the detection results in a noise-added fabric image with different variance v. All yarn locations have been correctly recognized. This is because the accumulated partial derivative histogram reduces the influence of the noise on yarn detection. The periodic structures of the fabric images are still reflected in the accumulated partial derivative histograms. Therefore, experiments demonstrate that our method is feasible and robust.
The detection results of yarns in a noise-added image with different variance v. (a) v = 102/2562 (b) v = 202/2562 (c) v = 302/2562.
Detection of the fabric structure
Our detection method can be applied to detect the fabric structure with multiple color effects. After estimating the yarn locations in fabric images, the edge-intensity-based cross-point classification
8
is used to determine the type of each yarn patch, and then the fabric structure is obtained. For example, Figure 12 displays the structure detection result for a basket weave fabric image using our method. Additionally, Figure 13(b) shows the representative color for each yarn patch, obtained by calculating the average color of the patch. More examples are shown in Figure 13, which further confirm the effectiveness of our method.
The detection results of a basket weave: (a) the fabric boundary grid; (b) the representative color for each yarn patch; (c) the detection result. The structure detection results in different fabric images including basket weave and plain weave fabric images.

Conclusion
In this paper, a new automatic yarn detection method is proposed based on histogram segmentation. Yarn locations in fabric images are inferred from a statistical analysis of accumulated partial derivative histograms. Specifically, accumulated partial derivative histograms are segmented by a nonparametric monotone hypothesis to estimate yarn locations. The benefits of using this hypothesis in yarn detection is manifold. First, compared with classical detection methods, the monotone hypothesis can reduce the influence of yarn random texture noise on detection results. Second, compared to the smoothing-based methods, more yarn boundary details are well preserved. Hence, for fabric images with yarn hairiness and dark-color yarns, yarn locations can be accurately detected. Numerical results demonstrate that the accuracy of our method is better than the smoothing-based method in terms of the detection of yarn locations. Moreover, our method can also be used to locate yarn floats and identify fabric structures.
Footnotes
Declaration of conflicting interests
The authors declared no potential conflicts of interest with respect to the research, authorship and/or publication of this article.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported in part by the National Natural Science Foundation of China (Grant Nos. 61872429, 61972264, 61772343, 61402290 and 61472257); in part by the Natural Science Foundation of Guangdong (Grant No. 1714050003822); and in part by the Natural Science Foundation of Shenzhen (Grant No. JCYJ20180305125521534). A short version of the paper was reported in the 2nd China pattern recognition and computer vision Conference (prcv2019).
