Automatic recognition method for the repeat size of a weave pattern on a woven fabric image

Abstract

This paper proposes a seven-characteristic method called Local Feature Similarity (LFS) to recognize weave pattern repeat automatically. The LFS method includes yarn feature acquisition, yarn marking, yarn marking subsequence segmentation, yarn marking subsequence similarity calculation and weave pattern repeat size, which recognizes the repeat size by the texture of the fabric image without considering the color influence. LFS can be applied to the three primary weave patterns and their expansions, such as warp rib, weft rib and panama, and basket weave patterns. Furthermore, six classical adaptive methods are introduced to increase the recognition rate based on the LFS method. As a result, the pre-steps of the recognition of the weave pattern repeat size are improved. Experiments proved the high degree of applicability and robustness of the LFS method. Compared with state-of-the-art methods, the LFS method has the highest recognition rate (99.62%) for the whole sample and excellent performance in average recognition time (6.97 ms). Experiments revealed that the higher recognition rate of the LFS method is helpful for a higher rate of weave pattern repeat recognition and weave pattern recognition (our source code and all experimental results are publicly available: https://github.com/spring2001/automatic-recognition-method-for-the-repeat-size-of-weave-pattern-on-woven-fabric-image).

Keywords

local Feature Similarity repeat size recognition weave pattern fabric image recognition

Most fabrics, in which wefts and warps are woven periodically, can be deciphered as one or multiple types of weave pattern repeats. The weave pattern repeat expresses the plane distribution of warp and weft floats,¹ which repeats over the whole area of the fabric and ignores the physical characteristics of each yarn. Weave pattern repeats are tiled in the warp and weft orientation periodically on fabrics. As a key parameter, the repeat size of a weave pattern plays an important role in the design and redesign, production design and control, performance analysis, texture analysis, etc., of woven fabrics.^2–4

An automatic recognition method for the repeat size of weave patterns is an important step to recognize weave pattern repeat and improve the accuracy of weave pattern recognition. Presently, the methods of recognizing a weave pattern with computer vision can be categorized as database-based^5,6 and adaptive algorithm-based.^7,8 Moreover, the latter methods can be classified as image feature-based methods working on the fabric images^9–12 and weave pattern-based methods, using the pre-recognized weave pattern.^7,13–15

For the database-based recognition methods, matching or classification algorithms are used to match fabrics to different preset templates or categories of weave pattern repeats. Salem and Nasri¹⁶ extracted the Gabor wavelet, LBP (Local Binary Pattern), or GLCM (Grey Level Co-occurrence Matrix) feature from a fabric image, respectively, then used the SVM (support vector machine) on each feature to classify the pattern of the fabric image as plain, twill or satin. Bao et al.¹⁷ proposed an improved KIII simulation model based on the ONN (Olfactory Neural Network), which can recognize the textures of plain weaves, twill weaves, satin weaves and complex weaves. Kuo et al.⁵ used back-propagation (BP) neural networks to classify the features extracted from the pre-recognized weave pattern of the three basic weave patterns based on a black–white co-occurrence matrix, then the repeat size of the weave pattern was obtained by the identified three basic weaves. Obviously, those classification methods are only applicable to the basic weaves. Pan et al.^6,18 built a standard pattern database storing the basic weave pattern repeats, and similar coefficients were calculated between a pre-recognized weave pattern and the standard pattern to classify the weave pattern repeats with this database. In Pan’s work, a BP neural network was also used to match the structure features of a pre-recognized weave pattern with the standard pattern.¹⁹ However, the performances of those matching methods all depend on database capacity.

The weave pattern repeat recognition methods based on the image feature were proposed in 1999. Kang et al.²⁰ calculated the general autocorrelation in the weft and warp directions of a fabric image to determine the pattern repeat unit size. In 2015, Jing et al.⁹ applied the IDMF (Improved Distance Matching Function) to fabric images and obtained the IDMF curve. The location of the maximum value in the second derivative of the curve is the size of the corresponding weave pattern repeat. Kang and Jing’s methods are only suitable for those fabric images that have similar thickness in the warp and weft. Zheng¹⁰ detected the corners from textural information by the Harris corner detector, then searched for the adjacent corners in the horizontal or vertical direction to identify the repeat size of the weave pattern. This method can only process fabric images with clear texture, horizontal weft and vertical warp. Moreover, the repeat size of the weave pattern obtained by the three methods mentioned above is described by the number of the pixels rather than the yarns in a weave pattern repeat. Lim and Kim⁸ calculated the average difference in the color among the rectangular regions in the center of the fabric image and its four same-size neighboring regions, then obtained the size of the weave pattern repeat by expanding the dimension of those regions until the minimum average color difference was calculated. Lim’s method can only be used for a yarn dyed fabric image with uniform illumination. Xiao et al.¹¹ extracted each yarn’s feature vector by combining all feature vectors of weave points in the corresponding yarn, then calculated the correlation coefficient of the features of the first yarn with that of the other yarns. The pattern repeat size could be obtained based on the peak of the correlation coefficient sequence. However, it is difficult for Xiao et al.’s method to estimate the correct peak of the correlation coefficient sequence automatically when the repeat size is over four yarns or the texture is not clear.

The state-of-the-art research focuses more on the weave pattern-based methods^21–23 because these are close to the definition of weave pattern repeat. In Ajallouian et al.’s²⁴ paper, to obtain the pattern repeat size, each row and column of the pre-recognized weave pattern are put in a separate matrix sequentially until the first repeated row and column are found. Zhong et al.^25,26 calculated the largest independent group of the rows and columns of the pre-recognized weave pattern respectively to determine the size of the repeat units. Ajallouian and Zhong’s methods are similar in principle, which is similar to some extent to the approach in Xiao’s method. However, the two methods are only suitable for the recognized weave pattern without error or unrecognized weave point. Wang et al.^13,27 calculated the two-dimensional (2D) autocorrelation matrix of the pre-recognized weave pattern to detect the size of the pattern repeat. Zhou et al.^14,28 proposed an EL (Extraction and Location) method to detect the size of the repeat unit from the pre-recognized weave pattern with errors and unrecognition. Yet, the accuracy of Wang and Zhou’s methods have good performance only when the repeat size of the weave pattern is less than six. Zhang et al.⁷ divided the pre-recognized weave pattern into sub-matrices according to different specified repeat sizes along the horizontal or vertical direction. The similar degree between the same-size sub-matrices is calculated to find the size of the repeat units. Zhang’s method only works when the size of the weave pattern repeat is less than the smaller number, between six and a quarter of the quantity of weft or warp threads on the fabric image. Schneider and Merhof²⁹ segmented the recognized weave pattern into weave pattern repeats manually. In the current automatic weave pattern-based methods, a general challenge is the rapid increase in the rate of false recognition when the repeat size of the weave pattern is over six.

It is necessary to develop a systematic, comprehensive and modular method of recognizing the repeat size of a weave pattern that can improve the recognition rates and adaptive capacity of the weave pattern repeat size recognition method. In this paper, a seven-characteristic method is proposed to recognize the weave pattern repeat, making full use of the geometric and texture characteristics of the weave pattern repeat, and achieving the best experimental result in comparison with the current methods. This paper is organized as follows: firstly, seven characteristics of the weave pattern repeat are deduced; then an automatic recognition method for the repeat size of the weave pattern, that is, the Local Feature Similarity (LFS) method, is proposed; all the steps of the LFS method and six classic weave pattern repeat recognition methods are modified to prove the validity of the theory; the proposed method is compared with the improved six classic methods to verify the efficiency; the conclusions are given in the last section.

Recognition algorithm on the weave pattern repeat size

Analysis of the characteristics of weave pattern repeats

To precisely recognize the repeat size of weave patterns, the characteristic of the weave pattern repeat must be analyzed comprehensively. There are three basic types of weave patterns and their expansions. As the structure is extended on a plane, weave pattern information can be extracted from a single-layer woven fabric image. Figure 1 shows the three basic patterns of woven fabric and their weave pattern repeats via multi-angles. Figures 1(a)–(c) show the original three basic pattern images obtained by a microscopy camera. Figures 1(d)–(f) show the performances of each weave pattern repeat on these three images. Figures 1(g)–(i) show how the weave pattern repeats are distributed on the weave pattern of the three basic patterns. Figures 1(j)–(l) illustrate the weave pattern repeats of the three basic patterns.

Figure 1.

Three basic patterns of woven fabric and their weave pattern repeats: (a) original plain fabric image; (b) original twill fabric image; (c) original satin fabric image; (d) weave pattern repeats in the plain fabric image; (e) weave pattern repeats in the twill fabric image; (f) weave pattern repeats in the satin fabric image; (g) weave pattern repeats in the plain weave pattern; (h) weave pattern repeats in the twill weave pattern; (i) weave pattern repeats in the satin weave pattern; (j) a plain weave pattern repeat; (k) a twill weave pattern repeat; (l) a satin weave pattern repeat.

The definition of the weave pattern repeat is that the smallest unit represents the distribution of warp and weft floats and appears repeatedly over the whole area of the fabric, as shown in Figure 1. The weave pattern repeats on a woven fabric with a simple weave pattern have the following characteristics.

Each weave pattern repeat is a rectangular area constituted by weft floats and warp floats.

The repeat size of the weave pattern is determined by the number of wefts and warps in the rectangle.

All weave pattern repeats on a woven fabric have the same sizes.

Warps or wefts in the weave pattern repeat are classified into different types according to different yarn structures, that is, a different arrangement of weft floats and warp floats along a yarn. The structural type of warp or weft in a weave pattern repeat is the same as the structural type of the corresponding warp or weft in any other weave pattern repeats on a woven fabric.

On a woven fabric, different types of warp or weft in all weave pattern repeats are arranged in a fixed order from left to right or from top to bottom.

The two nearest weave pattern repeats on a woven fabric are adjacent. Furthermore, the weave pattern of a woven fabric is formed by tiling the same weave pattern repeats on a plane.

For two adjacent weave pattern repeats on a woven fabric, one is located along the horizontal or vertical direction with respect to the other one.

These seven characteristics are not only adapted to the basic and simple weave pattern but also adapted to the extended and most complex weave patterns.

As well as the above-mentioned characteristics, the weave pattern repeats have multiple kinds of structures and types due to the various arrangements of weft and warp floats. The database-based methods are only able to match fabrics with preset templates or categories of weave pattern repeats, which may limit the flexibility to recognize various weave pattern repeats. The image feature-based methods and weave pattern-based methods are considered here. The generic representation of the weave pattern repeat is shown in Figures 1(j)–(l). Obviously, it is more intuitive to extract the weave pattern repeat size from the weave pattern shown in Figures 1(g)–(i), so weave pattern-based methods are studied more widely. However, those methods are modeled on the ideal weave pattern without the inevitable weave point errors and defects. Beyond that, the pre-recognized weave pattern is stored in the computer as a matrix whose elements are 0 (for warp floats) and 1(for weft floats). Sometimes, –1 is used to represent the unrecognizable weave points, which means that there are not enough redundant features to ensure the accuracy of weave pattern repeat recognition. The possible errors or defects and the lack of redundant features in the weave pattern may limit the accuracy of weave pattern recognition. Therefore, the image feature-based methods seem to be the best choice to recognize the repeat size of weave pattern.

Compared with the current image feature-based methods, Xiao et al.’s method¹¹ is more suitable for the above-mentioned weave pattern characteristics. The statistical method is also used in Xiao’s method to avoid possible extreme situations. Nevertheless, Xiao et al.’s method cannot work well when the target image has a large weave repeat or unclear texture, and for these types of woven fabric images, some help in the form of manual operation is needed to increase the recognition accuracy. To overcome the shortcomings of low recognition rate and manual operation, an automatic method is proposed in this paper, which is inspired by Xiao et al.’s method and the seven characteristics. The necessary pre-steps and the whole steps of the LFS method are described as the following sections.

Pre-steps: image pre-processing, deviation correction, weave point segmentation and weave point features extraction for characteristics (a) and (b)

The fabric images taken by microscopy camera are red, green, blue (RGB) color images. To eliminate the influence of color and keep the geometric detail on the fabric, images are transformed into hue, saturation, value (HSV) color space, as shown in Equation (1). Then the V component, that is, the illumination images, will be used in the subsequent processing, which has been proven to be effective for weave pattern recognition.³⁰ Then median filtering and eroding are deployed to enhance the value image

I_{V} = max {I_{R}, I_{G}, I_{B}}

(1)

where

I_{R}, I_{G}, I_{B}

represent the R, G, B components in the RGB color space, respectively, and I_V is the V component in HSV color space.

To extract each yarn and weave point, and rearrange them into a rectangular matrix as characteristics (a) and (b), deviation correction and weave point segmentation are as described in Xiao et al.’s paper,¹¹ and its improvement by statistical methods is proposed in this paper. For fabric images with horizontal and straightened weft yarns, weft yarns are segmented from the image based on the curve troughs of the horizontal illumination projection first with a Gaussian filter. Then the inclination angle of warp yarns is calculated by counting the average deviation between the corresponding troughs of the top and bottom wefts, and all warp yarns are corrected vertically according to the inclination angle. Finally, warp yarns are segmented based on the curve troughs of the vertical illumination projection with the Gaussian filter. The segmented weft yarns and warp yarns are combined by interlacing to extract each weave point.

When projecting the illuminated images in the horizontal direction, some redundant curve troughs may exist in the projecting curve that will interfere with the yarn segmentation. In this paper, the thickness of yarn is taken into consideration to remove the redundant curve troughs. Every curve trough is regarded as the boundary between two neighboring yarns, so the interval between two neighboring troughs is regarded as the thickness of the corresponding yarn. Wefts or warps in a woven fabric have the same thickness, that is, each interval between neighboring troughs in a projection curve has similar length. Here, we calculate the average length of all intervals between neighboring troughs on the horizontal projection curve of the illuminated image. If the length of one interval is less than 50% of the average length, the interval will be merged with the smaller neighboring interval, and then the trough between the merged neighboring intervals will be removed. The remaining troughs are regarded as the correct boundaries of wefts. This procedure of curve screening is repeated when segmenting the warps. The number of warps is defined as $N_{warp}$ and the number of wefts is defined as $N_{weft}$ . The weave point combination is regarded as a weave point matrix.

Figure 2 illustrates the result of pre-processing, deviation correction and weave point segmentation. To demonstrate the validity of the proposed method, we use the woven satin fabric image to display the whole procedure. Compared with plain fabrics and twill fabrics, the recognition of satin fabrics is more difficult.

Figure 2.

Illustration of image pre-processing, deviation correction and weave point segmentation: (a) original fabric image; (b) the illuminated image of (a); (c) the result of deviation correction; (d) the result of weave point segmentation.

According to characteristic (a), the weave points need to be divided into two groups, that is, warp floats and weft floats. To distinguish between the two different types of weave points, a type of effective feature should be used to represent warp floats and weft floats. The gradient orientation histogram feature proposed by Xiao et al. has already been proved to meet the above conditions.

Firstly, we set the direction of horizontal-right as the 0-degree angle and the clockwise rotation of the positive direction of angles. Then, the gradient orientations of all pixels on each weave point image are calculated, which are distributed in the interval $(- 180^{\circ}, 180^{\circ}]$ . The interval $(- 180^{\circ}, 180^{\circ}]$ is divided into 16 bins at intervals of 22.5° equally and the gradient orientations are sorted at each weave point into 16 bins. The gradient orientation histogram at each weave point is computed after histogram normalization. Here the gradient orientation histogram for the mth row and nth column weave point is set as vector $V_{m, n}, m = 1, 2, 3, \dots, N_{weft}, n = 1, 2, 3, \dots, N_{warp}$ . The gradient orientation histograms of a weft float and a warp float from Figure 2(c) are shown in Figure 3.

Figure 3.

Example for the gradient orientation histograms of a weft float and a warp float on a fabric image: (a) a weft float and a warp float on a fabric image; (b) gradient orientation histograms of the weft float; (c) gradient orientation histograms of the warp float.

The pre-steps are important for the subsequent steps and can improve the recognition performance. The first pre-step makes the proposed method available to colored yarn fabric or dying fabric by calculating the illumination component of RGB image, which can remove the color information and retain the geometric information of the texture on fabric images. The subsequent steps are designed based on the geometric characteristics of yarns, which makes the proposed method work well on different materials with the same geometric characteristics, for example, cotton, silk, linen, or even glass, carbon, etc.

It is noteworthy that the inclination angle is defined as the angle between the warp yarn and the vertical direction. Theoretically, if all yarns have the same thickness, Xiao et al.’s deviation correction method can correct warp yarns for which the inclination angle is up to 30° (from –30° to 30° by taking the rightward direction as positive). It is not difficult to control the inclination angle of warp yarns within this range when capturing the fabric images. In fact, the thicknesses between weft yarns and warp yarns may be different. For example, according to different weft yarns thicknesses, the upper bounds of the inclination angles that the proposed method can tolerate are from 6.20° to 39.81° for different fabric images in the database. However the max warp yarn inclination angle of all fabric images in this dataset is only 5.87° (from –4.36° to 5.87° on different fabric images).

What is more, by the pre-steps, the proposed method can be used to recognize fabrics with different yarn thicknesses. The deviation correction, weave point segmentation and weave point feature extraction can automatically adapt to different yarn thicknesses, and then all weave points with different yarn thicknesses are transformed into feature vectors with the same length; finally, the following steps only work on the features of these weave points, irrespective of the yarn thickness.

The image pre-processing, deviation correction weave point segmentation and weave point feature extraction are necessary and generic steps to analyze the texture of the fabric images. They are not included in the LFS method since the task of the LFS method is identifying the number of wefts and warps in the rectangular area that represent a weave pattern repeat as characteristic (b). All the steps of the LFS method will be discussed latter.

Yarn feature acquisition and yarn marking for characteristics (d) and (g)

The recognition between warps and wefts in the weave pattern is only in the operational direction. The process to recognize the warp number of the weave pattern repeat is given as follows. In addition, the weft number of the weave pattern repeat can be obtained by repeating the steps of warp number recognition with the change of direction.

According to characteristics (d) and (g), each column in a weave pattern repeat can be extended to a column in the weave point matrix, which is the same for each row. Therefore, the number of warps in a weave pattern repeat is equal to the cycle period of the warp in a specified float arrangement mode. The feature of each warp is obtained by combining the features of weave points in the corresponding column. The nth warp feature U_n is obtained as follows

U_{n} = (V_{1, n}, V_{2, n}, V_{3, n}, \dots, V_{N_{weft}, n}), n = 1, 2, 3, \dots, N_{warp}

(2)

The size of U_n is a $16 \times N_{weft}$ matrix. Obviously, U_n contains the whole information of the float arrangement mode in the nth warp.

To reduce the dimension of each warp and obtain the cycle period of a specified float arrangement warp, one warp will be selected as the standard warp to calculate the correlation coefficient of these features related to all warps. If two warps are in the same float arrangement mode, then their features are more alike, which means their correlation coefficient is close to 1; otherwise, their correlation coefficient is close to 0. As illustrated in Figure 4, since human attention and lamplight usually focus on the center of the fabric, which influences the layout and image definition when scanning fabric images, the middle warp of a fabric image has better geometric information. Therefore, the middle column $Midw$ and its warp feature $U_{Midw}$ are regarded as the standard warp feature, and $Midw$ is obtained by Equation (3)

Midw = ⌊ N_{warp} / 2 + 1 / 2 ⌋

(3)

where

⌊ ⌋

is the mathematical operation to round down to the nearest integer.

Figure 4.

Middle warp in Figure 2(c).

The correlation coefficient ${Rw}_{n}$ between U_n and $U_{Midw}$ from $n = 1$ to n = $N_{warp}$ is calculated. Obviously, the value of ${Rw}_{n}$ can reflect the float arrangement mode of the nth yarn. The ${Rw}_{n}$ values of yarns with the same float arrangement are closer than those with different float arrangements. ${Rw}_{n}$ of each yarn in Figure 2(c) is illustrated in Figure 5. The sequence ${{Rw}_{1}, {Rw}_{2}, {Rw}_{3}, \dots, {Rw}_{N_{warp}}}$ constitutes a warp-marking sequence $Rw$ , which can be drawn as a warp-marking curve.

Figure 5.

All yarns in Figure 2(c) and the curve of their marking numbers, ${Rw}_{n}$ . (Color online only.)

Yarn marking subsequence segmentation according to characteristics (c), (f) and (g)

According to characteristic (d), the number of warps in each weave pattern is equal to the distance between two adjacent peaks of the warp-marking sequence. However, it is difficult to extract all the correct peaks from the warp-marking curve, as marked by red circles in Figure 5. The correct peaks represent the warps, which have the same float arrangement mode as the standard warp, but there exist some extra peaks with interference in the warp-marking sequence. General peak extraction algorithms, such as the threshold method, extremum method or maximum method, are not suitable for the marking-sequence curve with multiple correct peaks or interference peaks.

As a typical example, Xiao’s method calculates the repeat size by locating the yarns that have a strong correlation coefficient with the first yarn. In this paper, there are some differences from Xiao et al.’s paper, in that Xiao et al.’s method attempts to be reworked by using the extremum points in the warp-marking curve where the correlation coefficient is over 0.6. To some extent this threshold can filter out the error peaks. However, due to the varied yarn features, it cannot find the correct peaks in the cases that two yarns have different structures but similar yarn features, or two yarns have the same structure while obvious different yarn features. Both cases can lead to the complicated distribution and varied value of the peaks in the warp-marking curve, except the relative relation in the warp-marking curve, that is, the shape of the warp-marking curve.

To avoid the instability in calculating the weave pattern repeat size based on the peaks of the warp-marking curve, an innovative automatic detection method based on the shape of the warp-marking curve is proposed so as to avoid the limitation of peak extraction. With characteristics (c), (f) and (g), different yarn structures of warps are considered to be different types, and warps with different types in every weave pattern repeat are arranged in a fixed order, which means that the curve of warp-marking numbers in every weave pattern repeat has a similar shape. For example, the repeat size of the weave pattern in the fabric image in Figure 2(c) is 5 × 5, so the warp repeat cycle is 5. As can be seen from Figure 6, three groups of adjoining equal-length subsequences are segmented from the warp-marking sequence curve based on different subsequence lengths.

Figure 6.

Correlation performances for three groups of adjoining equal-length subsequences. The groups are segmented from the warp-marking sequence curve in Figure 2(c) based on subsequence lengths 3, 5 and 10 respectively.

When the subsequence length is 5 (the real warp repeat cycle) or 10 (a double length), the correlation coefficient between two equal-length continuous subsequences is very high (0.98 and 0.96, respectively, in Figure 6). However, when the subsequence length is 3 (neither the real warp repeat cycle nor its integer multiple), the correlation coefficient between the two equal-length continuous subsequences is very low, for example, 0.23 in Figure 6. Given a warp-marking sequence $Rw$ , it is segmented into different groups of subsequences, and all the subsequences in each group have the same length, while the subsequence lengths of different groups are different. Then, the average correlation coefficient among subsequences in each group is calculated to find the groups whose elements have very strong similarity. The minimum subsequence length in the groups with very strong similarity is the repeat size of the required weave pattern.

Here, l is denoted as the length of subsequences and S_l as the number of subsequences in length $l$ . The relationship between l and S_l is denoted in Equation (4)

S_{l} = ⌊ N_{warp} / l ⌋

(4)

where

l = 2, 3, \dots, ⌊ N_{warp} / l ⌋

. The minimum value of l is limited to 2 for the reason that the smallest weave pattern repeat comes from plain weave. In addition, the minimum value of S_l is set as 2 as the correlation coefficient needs to be computed between at least two subsequences. The geometrical information in the central area of the fabric image is better than those in other areas, so all the subsequences segmented from the warp-marking sequence with specific length are symmetrical to the center of the marking sequence. For subsequence length l, the elements in the ith subsequence are from the

A (l, i)

th to the

B (l, i)

th in the warp-marking sequence, where

i = 1, 2, 3, \dots, S_{l}

A (l, i)

and

B (l, i)

are computed as follows

A (l, i) = ⌊ (N_{warp} \mod l) / 2 ⌋ + l \times (i - 1) + 1

(5)

B (l, i) = ⌊ (N_{warp} \mod l) / 2 ⌋ + l \times i

(6)

where mod is the modular arithmetic operator. Figure 7 illustrates the subsequences segmentation result of warp-marking sequence (in Figure 5) with different subsequence lengths l. Each row in Figure 7 indicates a subsequence length l. The ith subsequence with length l extracted from the warp-marking sequence is written as

D_{l, i}

. So,

D_{l} = {D_{l, 1}, D_{l, 2}, D_{l, 3}, \dots, D_{l, S_{l}}}

is written as the l-length subsequence set and shown in the right-hand column in Figure 7. By comparing the curves of all subsequences in each l-length subsequence set (l = 2,…,15), we can provide further evidence that when l is 5 (the real size of weave pattern repeat), 10 or 15 (integral multiple of the real weave pattern repeat size), any two subsequences in the corresponding set are strongly correlative. The next step is using the correlation coefficient between any two subsequences in each l-length subsequence set to obtain the warp number of the weave pattern repeat automatically.

Figure 7.

Subsequence segmentation result of the warp-marking sequence shown in Figure 5 with each subsequence length l. Columns from left to right: the length of subsequence will be segmented from the warp-marking sequence, the segmentation way of warp-marking sequence shown in Figure 5 and all subsequences obtained by segmentation with the corresponding subsequence length. Rows from top to bottom: the lengths of subsequences are from 2 to 15. (Color online only.)

Similarity calculation of yarn-marking subsequences and weave pattern repeat size obtained by characteristic (e)

After the warp-marking sequence segmentation, the average correlation coefficient among subsequences in each l-length subsequence set will be calculated. It is unnecessary to compute the correlation coefficients between every subsequence and the rest, which needs $[(S_{l} - 1) \times S_{l}] / 2$ times of calculation for an l-length subsequence set. If the correlation coefficients are computed between a selected standard subsequence and the rest in each l-length subsequence set, the amount of calculation will be greatly reduced. As the middle warp of a fabric image has better geometric information, the middle subsequence is regarded as the standard subsequence. In Figure 7, the middle subsequences in all l-length subsequence sets are marked with a red rectangle. For each l-length subsequence set, in which the number of elements is S_l, the ordinal values for the middle subsequence are ${Midc}_{l}$ , that is

{Midc}_{l} = ⌊ S_{l} / 2 + 0.5 ⌋

(7)

A selected $D_{l, {Midc}_{l}}$ is taken as the standard subsequence to calculate the average correlation coefficient.

$r_{l, i}$ is denoted as the correlation coefficient between $D_{l, {Midc}_{l}}$ and the ith subsequence $D_{l, i}$ , where $i = 1, 2, 3, \dots, S_{l}$ . $r_{l, {Midc}_{l}}$ is 1 when the correlation coefficient is computed between the standard subsequence and itself. The average correlation coefficient among all elements of the l-length subsequence set is r_l, that is

r_{l} = (\sum_{i = 1}^{S_{l}} r_{l, i} - 1) / (S_{l} - 1)

(8)

All r_l of the l-length subsequence are set for $l = 2, 3, 4, \dots, N_{warp} / 2$ , which forms a subsequence similarity sequence R_c, where $R_{c} = {r_{2}, r_{3}, \dots, r_{N_{warp} / 2}}$ . The curve of R_c calculated from Figure 7 is shown in Figure 8.

Figure 8.

Subsequence similarity sequence calculated from Figure 7. (Color online only.)

To extract the strong correlation value from R_c, a relative threshold T is set. If the value of correlation between two subsequences is T or more, then those two subsequences are considered to be strongly correlative. Here, the threshold of T is adopted to isolate the first l₀-length subsequence set with strong correlation from all the l-length subsequence sets, where l₀ is the warp number of the weave pattern repeat. In this paper, an absolute threshold T is set to determine the first l-length subsequence set with a strong correlation. As shown in Figure 8, the absolute threshold is marked with a red-dot horizontal line.

However, because of the influence of irregularity in the fabric image, subtle texture features or inaccuracy in the previous steps, etc., not all R_c include elements greater than or equal to T. For example, Figure 9 shows two fabric images. The wefts of Figure 9(a) are not horizontal or straight, while the texture feature of Figure 9(b) is unclear. As a result, for the R_c calculated from Figures 9(a) or (b), the maximum element is less than T. However, the peak of R_c, calculated from Figures 9(a) or (b), which represents the warp number or the weft number of the weave pattern repeat, is significantly higher than the others. In this paper, a relative threshold to determine the first l-length subsequence set with very strong correlation is designed to refer to the T rule. The relative threshold is equal to T times the maximum value of R_c. In Figures 9(e) and (f), the relative threshold is marked with a red horizontal dotted line.

Figure 9.

Two examples of fabric images with irregularities in the image and subtle texture features: (a) original fabric image whose wefts are not horizontal or straight; (b) original fabric image whose texture feature is insignificant; (c) weave point segmentation of (a); (d) weave point segmentation of (b); (e) warp-marking subsequence similarity sequence of (c); (f) weft-marking subsequence similarity sequence of (d). (Color online only.)

The value of T plays an important role, which affects the recognition result. If T is too low, the recognized repeat sizes of weave pattern for a fabric may be smaller than the real value; otherwise, the recognized repeat sizes of the weave pattern may be larger than the real results. According to the strong correlation coefficient theory and experiment results, this paper sets $T = 0.8$ .

All the steps to extract the warp number of the weave pattern repeat from warp-marking subsequence similarity sequence R_c based on an absolute threshold and relative threshold are summarized in Algorithm 1.

Algorithm 1 Weave pattern repeat size obtained based on absolute threshold and relative threshold
Input: Subsequence similarity sequence: $R_{c} = {r_{2}, r_{3}, \dots, r_{N_{warp} / 2}}$ Output: Warp or weft number of the weave pattern repeat: $NS$
1: Set $T = 0.8$ . 2: Set $ind = 1$ . 3: If $r_{ind} < T$ , go to Step 4; otherwise, go to Step 5. 4: If $ind = ⌊ N_{warp} / 2 ⌋$ , let $T = 0.8 \times max (R_{c})$ , return to Step 2; otherwise, $ind = ind + 1$ , return to Step 3. 5: Output $NS = ind$ as the warp or weft number of the weave pattern repeat.

The weft number of the weave pattern repeat can be recognized by repeating the above process from yarn feature acquisition to the weave pattern repeat size obtained. The only difference is that the processing object changes to wefts. Image pre-processing, deviation correction, weave point segmentation and weave point feature extraction are also necessary steps. Therefore, only the steps from yarn feature acquisition to weave pattern repeat size acquisition are called LFS in this paper. Figure 10 demonstrates the result of the recognition of the weave pattern repeat size from Figure 2(a) based on LFS. The LFS method can recognize a weave pattern repeat whose sizes are not larger than one half of the number of warps and wefts on the fabric image.

Figure 10.

The result of weave pattern repeat size recognition from Figure 2(a) based on Local Feature Similarity.

Improvements to comparison methods based on Local Feature Similarity

Experimental framework and dataset

In this section, all the steps of the LFS method are illustrated and six current, adaptable recognition methods for weave pattern repeat size are modified based on the theory behind LFS. Xiao et al.’s method and the six modified methods are used for comparison. Algorithm 2 sums up all the LFS steps, Xiao et al.’s method and the six modified methods.

Algorithm 2 Overall algorithm of the LFS method and comparative methods
Input: High-resolution color fabric image. Output: Repeat size of weave pattern.
1: Pre-step of the LFS method and comparative methods. 1.1: Image pre-processing. 1.2: Deviation correction. Output deviation corrected illu mination image. 1.3: Weave point segmentation. Output yarn segmentation position. 1.4: Weave point feature extraction. Output each weave point feature. 1.5: Weave point classification by Fuzzy C-Means Algorithm (FCM). Output pre-recognized weave pattern matrix. 2: LFS method to recognize the repeat size of weave pattern. Input all weave point features obtained from Step 1.4. Loop over warp-to-weft direction: 2.1: Yarn feature acquisition. 2.2: Yarn marking. 2.3: Yarn marking subsequence segmentation. 2.4: Yarn marking subsequence similarity calculation. 2.5: Weave pattern repeat size acquisition. Output the repeat size of weave pattern. 3: Comparative methods to recognize the repeat size of the weave pattern. 3.1: Modified IDMF method and modified gray autocorrel ation method. Input deviation corrected illumination image obtained in Step 1.2 and yarn segmentation position obtained in Step 1.3. Output the repeat size of weave pattern. 3.2: Xiao et al.’s method. Input yarn segmentation position obtained in Step 1.3 and each weave point feature obtained in Step 1.4. Output the repeat size of weave pattern. 3.3: Ajallouian’s method modified, modified pattern autocorrelation method, modified EL method and Zhang’s method modified. Input pre-recognized weave pattern matrix obtained in Step 1.5. Output the repeat size of weave pattern.

Algorithm 2 Overall algorithm of the LFS method and comparative methods

Input: High-resolution color fabric image. Output: Repeat size of weave pattern.

1: Pre-step of the LFS method and comparative methods. 1.1: Image pre-processing. 1.2: Deviation correction. Output deviation corrected illu mination image. 1.3: Weave point segmentation. Output yarn segmentation position. 1.4: Weave point feature extraction. Output each weave point feature. 1.5: Weave point classification by Fuzzy C-Means Algorithm (FCM). Output pre-recognized weave pattern matrix. 2: LFS method to recognize the repeat size of weave pattern. Input all weave point features obtained from Step 1.4. Loop over warp-to-weft direction: 2.1: Yarn feature acquisition. 2.2: Yarn marking. 2.3: Yarn marking subsequence segmentation. 2.4: Yarn marking subsequence similarity calculation. 2.5: Weave pattern repeat size acquisition. Output the repeat size of weave pattern. 3: Comparative methods to recognize the repeat size of the weave pattern. 3.1: Modified IDMF method and modified gray autocorrel ation method. Input deviation corrected illumination image obtained in Step 1.2 and yarn segmentation position obtained in Step 1.3. Output the repeat size of weave pattern. 3.2: Xiao et al.’s method. Input yarn segmentation position obtained in Step 1.3 and each weave point feature obtained in Step 1.4. Output the repeat size of weave pattern. 3.3: Ajallouian’s method modified, modified pattern autocorrelation method, modified EL method and Zhang’s method modified. Input pre-recognized weave pattern matrix obtained in Step 1.5. Output the repeat size of weave pattern.

The six methods selected are the IDMF method,⁹ the gray autocorrelation method,²⁰ Ajallouian’s method,²⁴ the pre-recognized weave pattern autocorrelation method (pattern autocorrelation method for short),¹³ the EL method¹⁴ and Zhang’s method.⁷

Four of the six methods are based on pre-recognized weave patterns. Thus, the weave points must be categorized into warp floats and weft floats or, sometimes, as unrecognized floats. This paper classifies these weave points by clustering the weave point features based on the improved FCM algorithm in Xiao et al.’s paper. The weave point features will be divided into two groups, that is, warp floats (marked with 0) and weft floats (marked with 1). Each weave point in the weave point matrix corresponds to 0 or 1 to form a $N_{weft} \times N_{warp}$ matrix, that is, a pre-recognized weave pattern matrix.

A fabric image dataset is used in this paper, which was provided by Xiao. All the images are in RGB color space and they comprise 205 fabric images with plain weave, 34 with twill weave and their simple expansions and 26 with satin weave. On each image all the yarns are straightened as much as possible; light is focused on the center of fabric; the wefts are almost horizontal; and the warps are tilted to some extent in the vertical direction when scanning. Samples with different colors, different yarn thicknesses, defocus blurs, irregular textures and cotton yarns are included in this dataset, which can be processed by the proposed method robustly, as this method is based on the geometric characteristics rather than the color information of yarns on fabric images. The size of each image in this dataset is 640 × 480 pixels. On each sample, the number of weft yarns is from 6 to 32, the number of warp yarns is from 11 to 45 and the thickness of each yarn is from 13 to 94 pixels, which provides enough geometric information of yarns for LFS and the comparison methods.

As LFS and the comparison methods depend on the geometric information of the yarns, the scanning resolution in the experimental dataset, which is enough to obtain the necessary information, has little impact on the analysis in this paper. Although the higher the resolution the better, it is better to ensure the thickness of every yarn over 5 pixels to keep the different geometric information between the warp float and weft float for LFS.

As the IDMF method, the gray autocorrelation method, Ajallouian’s method, the pattern autocorrelation method, the EL method and Zhang’s method used different ideal data in their respective papers, the performances of these methods on Xiao et al.’s dataset are poor. In order to test and compare these algorithms on a unified dataset, each algorithm was improved simply based on LFS. However, Xiao et al.'s method is an exception because it has been proved to be suitable for this dataset in Ziao et al.¹¹ The codes and dataset used in this paper is available on the authors’ webpages (https://github.com/spring2001/automatic-recognition-method-for-the-repeat-size-of-weave-pattern-on-woven-fabric-image).

Comparison methods I and II: improved IDMF method and improved gray autocorrelation method

The principles of the IDMF method and the gray autocorrelation method are similar. According to characteristics (b)–(d), (f) and (g), both methods are based on the periodic cycle of texture on the fabric image. It is assumed that each warp on the image has the same thickness as all others, and the same for wefts. Another assumption is that different pixel values can represent the two weave point types directly. Therefore, the IDMF and gray autocorrelation methods try to search for the minimum cycle of the pixel value, which is also the pixel number of each weave pattern repeat in the horizontal or vertical direction. In this paper, average yarn thickness is introduced to ensure that the outputs of the IDMF and gray autocorrelation methods are the numbers of yarns.

Firstly, improve the IDMF method. Let F be the 2D image matrix with the size of W × H obtained by deviation correction. The vector represents all the increasing horizontal coordinates of the split lines between warps, and $G_{weft}$ represents all the increasing vertical coordinates of the split lines between wefts after weave point segmentation. The number of elements in $G_{warp}$ is $N_{warp} + 1$ . Warps are taken as an example.

The first step is to calculate the average warp thickness ${Th}_{warp}$ as follows

W_{warp} = [G_{warp} (N_{warp} + 1) - G_{warp} (1)] / N_{warp}

(9)

where

G_{warp} (N_{warp} + 1)

is the last element of

G_{warp}

and

G_{warp} (1)

is the first element of

G_{warp}

The second step is to calculate the horizontal IDMF. As the smallest weave pattern repeat, the plain fabric limits the initial periods p_i to be larger than an average warp thickness. Let $p_{i} = 1.25 \times {Th}_{warp}$ and, therefore, the horizontal IDMF is computed as Equation (10)

{IDMF}_{Hor} (p_{c}) = \sum_{h = 1}^{H} \sum_{w = 1}^{W - p_{i} - p_{c}} [F (h, w) - F (h, w + p_{i} + p_{c})]^{2}

(10)

The third step is to extract the location of the maximum value $O_{max}$ in the second derivative of the horizontal IDMF. The warp number of the weave pattern repeat ${Npr}_{warp}$ will be obtained according to Equation (11). The weft number of the weave pattern repeat can be obtained based on vertical IDMF, similar to the above process

{Npr}_{warp} = (O_{max} + p_{i}) / {Th}_{warp} + 0.5

(11)

Secondly, improve the gray autocorrelation method. The element index of the gray autocorrelation function is unconventional according to Kang et al.’s paper.²⁰ Moreover, it is difficult to extract the correct peaks representing the cycle period of the weave pattern repeat. In this paper, we try to improve the gray autocorrelation method referencing the improved IDMF method. As for the example of warps, the only difference is in computing the horizontal autocorrelation curve using corrected Equation (12) in the second step and extracting the minimum value location of the autocorrelation curve to obtain the warp number of the weave pattern repeat

{AC}_{Hor} (p_{c}) = \sum_{h = 1}^{H} \sum_{w = 1}^{W - p_{i} - p_{c}} F (h, w) F (h, w + p_{i} + p_{c})

(12)

As the IDMF method and gray autocorrelation method process each pixel on the fabric image, the time complexities for these two methods are higher than that of LFS.

Comparison methods III: improved Ajallouian’s method

In Ajallouian et al.’s paper,²⁴ the method to recognize the size of the weave pattern is to find the distance between two adjacent rows and columns with the same arrangement of warp and weft floats from the pre-recognized weave pattern by expanding a separate matrix. Ajallouian et al.’s method only meets characteristics (a), (b), (d), (f) and (g), and assumes that each weave point is recognized as a weft float or warp float accurately. However, the assumption gives a lack of robustness to this method. To overcome this shortcoming, the limit of the separate matrix is left out and a tolerance of error is introduced to search for two adjacent rows and columns with a similar arrangement of warp and weft floats.

Let E_t represent the error tolerance. The value of E_t is determined by the requirement, and it is set as 20.00% in this paper. The higher the E_t value, the more robust is the performance of Ajallouian et al.’s method, but the smaller is the size of the weave pattern repeat that can be recognized. All the steps of Ajallouian et al.’s method with modifications are described in Algorithm 3.

Algorithm 3 Ajallouian et al.’s method improved to extract the warp number of the weave pattern repeat
Input: Pre-recognized weave pattern. Output: Warp number of the weave pattern repeat.
1: Set $E_{t} = 0.20$ , ${ind}_{0} = 1$ , ${ind}_{1} = {ind}_{0} + 2$ . 2: Compare the ${ind}_{1}$ th column with the ${ind}_{0}$ th columns. If the number of elements different between the ${ind}_{1}$ th column and the ${ind}_{0}$ th column is less than $E_{t} \cdot N_{warp}$ , go to Step 5; otherwise, go to Step 3. 3: If ${ind}_{1} = N_{warp}$ , go to Step 4; otherwise, ${ind}_{1} = {ind}_{1} + 1$ , return to Step 2. 4: If ${ind}_{0} = N_{warp} - 2$ , output $N_{warp}$ as the warp number of the weave pattern repeat; otherwise, ${ind}_{0} = {ind}_{0} + 1$ , ${ind}_{1} = {ind}_{0} + 2$ , return to Step 2. 5: Output ${ind}_{1} - {ind}_{0}$ as the warp number of the weave pattern repeat.

Algorithm 3 Ajallouian et al.’s method improved to extract the warp number of the weave pattern repeat

Input: Pre-recognized weave pattern. Output: Warp number of the weave pattern repeat.

1: Set

E_{t} = 0.20

{ind}_{0} = 1

{ind}_{1} = {ind}_{0} + 2

. 2: Compare the

{ind}_{1}

th column with the

{ind}_{0}

th columns. If the number of elements different between the

{ind}_{1}

th column and the

{ind}_{0}

th column is less than

E_{t} \cdot N_{warp}

, go to Step 5; otherwise, go to Step 3. 3: If

{ind}_{1} = N_{warp}

, go to Step 4; otherwise,

{ind}_{1} = {ind}_{1} + 1

, return to Step 2. 4: If

{ind}_{0} = N_{warp} - 2

, output

N_{warp}

as the warp number of the weave pattern repeat; otherwise,

{ind}_{0} = {ind}_{0} + 1

{ind}_{1} = {ind}_{0} + 2

, return to Step 2. 5: Output

{ind}_{1} - {ind}_{0}

as the warp number of the weave pattern repeat.

The steps to recognize the weft number of the weave pattern repeat are similar. With the introduction of error tolerance, the recognition rate of Ajallouian et al.’s method increases from 33.21% (88 out of 265) to 81.51% (216 out of 265).

Comparison methods IV: improved pattern autocorrelation method

The pattern autocorrelation method proposed in Wang et al.’s paper²⁷ is another way of applying the autocorrelation algorithm, which matches characteristics (a)–(d), (f) and (g). The element index is evidently also out of the matrix boundary when describing the autocorrelation function. Here C represents the 3 × 3 tiling of weave float matrix instead of just the weave float matrix, and it is used to recognize the size of the weave pattern repeat by the autocorrelation method.

Comparison methods V: improved EL method

The EL method was proposed by Zhou in 2014,¹⁴ and it used the characteristics (a)–(c) and (e)–(g). Moreover, the EL method’s design was based on the assumption that the pre-recognized weave pattern matrix must possess at least one column and a row in which each weave point corresponds to the right type. If there is no exact identification in any column or row of the pre-recognized weave pattern matrix, the EL method will fail to function. To overcome the limitation of assumption and to increase the recognition accuracy, error tolerance E_t and statistical theory are added to the EL method.

An example of recognition is taken for the warp number of the weave pattern repeat. Weft number recognition can be obtained in the same way. In the original EL method, each row of the pre-recognized weave pattern matrix is tried in turn until the following condition is achieved: the smallest interval (represented by ${In}_{0}$ ) exists under the condition that all elements in row in ${In}_{0}$ intervals are put into one group. That means the row is divided into ${In}_{0}$ groups, and all elements in each group are either 0 or 1. Thus, ${In}_{0}$ is the warp number of the weave pattern repeat.

An improved EL method is proposed in this paper. The smallest intervals ( ${In}_{i}$ ) of the ith rows are computed under the following condition ( $i = 1, 2, \dots, N_{weft}$ ): in the ith row, elements in ${In}_{i}$ intervals are put into a group, then the corresponding row is divided into ${In}_{i}$ groups. The number of fewer elements (0 or 1) is counted in every group, such that the sum of the fewer elements in all groups is not more than $E_{t} \times N_{warp}$ . If ${In}_{i}$ cannot meet the condition in this step, let ${In}_{i} = N_{warp}$ . Then the values of ${In}_{i}$ in all rows are counted to find the number that occurs most often in $In = {{In}_{1}, {In}_{2}, \dots, {In}_{N_{weft}}}$ , or the smallest one if more than one number occurs most often in $In$ . This number is the warp number of the weave pattern repeat.

In experiments, the recognition rate of the original EL method is 80.00% (212 out of 265), and the recognition rate of the improved EL method is 88.30% (234 out of 265), which proves the validity of the improved method. Owing to the process that each row is divided into at least $N_{warp} / 2$ groups and each column is divided into at least $N_{weft} / 2$ groups, the EL method can recognize the weave pattern repeat where sizes are not larger than half of the number of warps and wefts.

Comparison methods VI: improved Zhang’s method

Zhang et al.’s method⁷ is the closest one to the LFS method proposed in this paper in principle, meeting all seven characteristics. By testing the dataset used in this paper, Zhang et al.’s method recognizes the satin, and the twill and its expansion at a rate of only 72.08% (191 out of 265). With more analysis on the process, the main reason for the low recognition rate is that the size of the weave pattern repeat from the similarity curve (represented by S as in Zhang et al.’s paper) of repeat units cannot be found. Based on Zhang et al.’s idea, the steps to detect the weave pattern repeat from S are modified into two steps instead of the original three steps.

Algorithm 4 Improved Zhang et al.’s method to extract the repeat size of weave pattern
Input: Pre-recognized weave pattern. Output: Warp number of the weave pattern repeat.
1: If there exist j₀ such that at least one number in $j = 1, 2, \dots, ⌊ L_{s} / 2 ⌋$ to makes $S_{i \times j_{0}}$ greater than the mean value of S, where $S_{i \times j_{0}} = max ([S_{(i - 1) \times j_{0} + 1}, S_{(i + 1) \times j_{0} - 1}])$ for all $i = 1, 2, \dots, ⌊ L_{s} / j_{0} ⌋$ , and L_s is the length of S, then the weave pattern repeat is the first j₀; otherwise, go to Step 2. 2: Find all k₀ to make $S_{k_{0}} = max (S)$ . The weave pattern repeat is the first k₀.

Algorithm 4 Improved Zhang et al.’s method to extract the repeat size of weave pattern

Input: Pre-recognized weave pattern. Output: Warp number of the weave pattern repeat.

1: If there exist j₀ such that at least one number in

j = 1, 2, \dots, ⌊ L_{s} / 2 ⌋

to makes

S_{i \times j_{0}}

greater than the mean value of S, where

S_{i \times j_{0}} = max ([S_{(i - 1) \times j_{0} + 1}, S_{(i + 1) \times j_{0} - 1}])

for all

i = 1, 2, \dots, ⌊ L_{s} / j_{0} ⌋

, and L_s is the length of S, then the weave pattern repeat is the first j₀; otherwise, go to Step 2. 2: Find all k₀ to make

S_{k_{0}} = max (S)

. The weave pattern repeat is the first k₀.

With the improved steps above, the recognition rate of Zhang et al.’s method has been improved to 97.74% (259 out of 265). However, restricted by Step 1 in Algorithm 4, Zhang et al.’s method can only recognize the weave pattern repeat well where the size is not larger than a quarter of the number of yarns on the fabric image.

Results and discussion

Theoretical comparison

The LFS method is compared with all six improved methods theoretically in Table 1. The improved Ajallouian et al.’s method, the improved pattern autocorrelation method, the improved EL method and the improved Zhang et al.’s method recognize the weave pattern repeat size from the pre-recognized weave pattern, which means that an incompletely recognized weave pattern has a significant effect on these methods. However, it does not affect the LFS method, the Xiao et al.’s method, the improved IDMF method and the improved gray autocorrelation method, as these methods recognize the weave pattern repeat size based on the feature vectors extracted from the fabric image directly, which is an operation step before weave pattern recognition. The LFS method and the improved Zhang et al.’s method make full use of the weave pattern repeat characters to realize higher recognition rates; the improved Zhang et al.’s method has the lowest utilization rate for fabric image and the improved IDMF method has the highest utilization rate.

Table 1.

Theoretical comparison of Local Feature Similarity, Xiao et al.’s method and six improved methods

Method	Category	Pattern repeat characteristics used	Mathematical theory used	Gettable size of weave pattern repeat recognized	Optimum size of weave pattern repeat recognized
S method	Image feature-based	(a)–(g)	Statistics Similarity	Warp: 2 to $N_{warp} / 2$ Weft: 2 to $N_{weft} / 2$	Warp: 2 to $N_{warp} / 2$ Weft: 2 to $N_{weft} / 2$
Xiao et al.’s¹¹ method	Image feature-based	(a)–(d), (f) and (g)	Statistics Similarity	Warp:2 to $(N_{warp} - 1)$ Weft: 2 to $(N_{weft} - 1)$	Warp: 2 to $(N_{warp} - 1)$ Weft: 2 to $(N_{weft} - 1)$
Improved IDMF method	Image feature-based	(b)–(d), (f) and (g)	Statistics Similarity	Warp: 2 to $N_{warp}$ Weft: 2 to $N_{weft}$	Warp: 2 to $N_{warp}$ Weft: 2 to $N_{weft}$
Improved gray autocorrelation method	Image feature-based	(b)–(d), (f) and (g)	Statistics Similarity	Warp: 2 to $N_{warp} / 2$ Weft: 2 to $N_{weft} / 2$	Depend on the texture of each fabric image
Improved Ajallouian et al.’s²⁴ method	Weave pattern-based	(a), (b), (d), (f) and (g)	Statistics Error tolerance	Warp: 2 to $(N_{warp} - 2)$ Weft: 2 to $(N_{weft} - 2)$	Warp: 2 to $(N_{warp} - 2)$ Weft: 2 to $(N_{weft} - 2)$
Improved pattern autocorrelation method	Weave pattern-based	(a)–(d), (f) and (g)	Statistics Similarity	Warp: 2 to $N_{warp} / 2$ Weft: 2 to $N_{weft} / 2$	Warp: 2 to $N_{warp} / 2$ Weft r: 2 to $N_{weft} / 2$
Improved EL method	Weave pattern-based	(a)–(c) and (e)–(g)	Statistics Error tolerance	Warp: 2 to $N_{warp} / 2$ Weft: 2 to $N_{weft} / 2$	Warp: 2 to $N_{warp} / 2$ Weft: 2 to $N_{weft} / 2$
Improved Zhang et al.’s⁷ method	Weave pattern-based	(a)–(g)	Statistics Similarity	Warp: 2 to $N_{warp} / 2$ Weft: 2 to $N_{weft} / 2$	Warp: 2 to $N_{warp} / 4$ Weft: 2 to $N_{weft} / 4$

IDMF: Improved Distance Matching Function; EL: Extraction and Location.

The LFS method is a general algorithm for the size recognition of weave pattern repeats. The inputs of the LFS method are the weave point positions obtained by segmentation and the corresponding weave point features obtained by feature extraction. The segmentation method and feature calculation method are not limited to those used in this paper, which means that the LFS method can be used as a part of various weave pattern recognition methods with some adaptation. Moreover, LFS is not only applicable for the basic weave pattern and simple expansion of twill in the test dataset, but also for more complex derivatives of basic weaves, such as warp rib, weft rib, panama and basket weave pattern. These complex derivatives of basic weaves cannot be identified by other methods in theory. Moreover, when recognizing these complex derivatives with large repeat sizes, the only limitation for LFS is the image size.

Recognition results of the LFS method and discussion

To evaluate the versatility, applicability, robustness and accuracy of the LFS method proposed in this paper, eight methods are tested in the dataset on the platform built by using MATLAB with an I7-4710HQ CPU and 16 GB RAM. The experiment included three parts: Part 1 is the recognition of the LFS method for each fabric image, to prove the effectiveness of all steps from inputting the original fabric image to outputting the weave pattern repeat size; Part 2 is the comparison of weave pattern repeat size recognition among the LFS method and other improved methods; Part 3 compares the LFS method and other improved methods in its application to weave pattern repeat recognition and weave pattern recognition.

Figure 11 illustrates the recognition results of the LFS method for 18 fabric images in the dataset as three groups. Figures 11(a)–(c) show the recognition results for plain fabric, twill and its expansion fabric and satin fabric. All the weave pattern repeats are mapped out in the original color images with deviation correction. The area in each rectangular frame (red dotted line) is represented as a weave pattern repeat. From the color fabric image, the different colors, different yarn thickness, different clarity, different weave patterns and irregular textures are intuitionistic. The inclination angle of warp yarns before the deviation correction in each image can be deduced backward from the width of each shown image, as the deviation correction will remove the horizontal boundary pixels from the sample images with the same size. All the above factors may affect the result of weave pattern size recognition. Figure 11 can prove directly the high suitability and robustness of the LFS method.

Figure 11.

Repeat size recognition results of the weave pattern using the Local Feature Similarity (LFS) method for 18 fabric images in the dataset: (a) the result of the LFS method for plain; (b) the result of the LFS method for twill and its expansion; (c) the result of the LFS method for satin. (Color online only.)

Comparison of methods for the recognition of the weave pattern repeat size

The recognition rate and average runtime of the eight methods are listed in Table 2. The LFS method ranks the first in weave pattern repeat size recognition rate and the fourth in average recognition time. In consideration of the recognition ratio, the LFS method recognized the weave pattern repeat size for all but one of the 265 samples, which is far more advanced than other methods, except the improved Zhang et al.’s method. The reason for failure in recognizing this one satin sample was the incorrect weave point segmentation for severely twisted yarns. The average recognition time per image of the LFS method was only 6.97 ms in this experiment over the whole dataset, meeting the requirements of fast response time and high efficiency for real-time application.

Table 2.

Comparison among Local Feature Similarity (LFS), Xiao et al.’s method and the six improved methods for weave pattern repeat size recognition

Method	Recognition rate for plain	Recognition rate for twill and its expansion	Recognition rate for satin	Recognition rate for whole samples	Average recognition time
LFS method	100% (205 in 205)	100% (34 in 34)	96.15% (25 in 26)	99.62% (264 in 265)	6.97 ms
Xiao et al.’s¹¹ method	89.27% (183 in 205)	97.06% (33 in 34)	92.31% (24 in 26)	90.57% (240 in 265)	3.39 ms
Improved IDMF method	94.15% (193 in 205)	97.06% (33 in 34)	84.62% (22 in 26)	93.58% (248 in 265)	1384.10 ms
Improved gray autocorrelation method	90.24% (185 in 205)	82.35% (28 in 34)	84.62% (22 in 26)	88.68% (235 in 265)	1088.03 ms
Improved Ajallouian et al.’s²⁴ method	84.39% (173 in 205)	73.53% (25 in 34)	69.23% (18 in 26)	81.51% (216 in 265)	0.88 ms
Improved pattern autocorrelation method	72.68% (149 in 205)	82.35% (28 in 34)	88.46% (23 in 26)	75.85% (201 in 265)	7.76 ms
Improved EL method	88.78% (182 in 205)	91.18% (31 in 34)	88.46% (23 in 26)	88.68% (236 in 265)	4.95 ms
Improved Zhang et al.’s⁷ method	99.51% (204 in 205)	88.24% (30 in 34)	96.15% (25 in 26)	97.74% (259 in 265)	28.55 ms

IDMF: Improved Distance Matching Function; EL: Extraction and Location. Note. Best recognition rates and minimum average recognition time are highlighted in bold.

A further comparative analysis of the LFS method and the improved Zhang et al.’s method is made. As both of them make full use of all seven characteristics, the LFS method presented only little better results than the improved Zhang et al.’s method. Compared to the LFS method, the improved Zhang’s method spent more average runtime on each fabric image. Different from the LFS method, the error weave pattern recognition reduced the weave pattern repeat size recognition rate by using the improved Zhang et al.’s method for plain, twill and its expansion, and the insufficient number of yarns on the fabric image caused the failure to recognize the size of 4/4 twill for the improved Zhang et al.’s method, thus proving the accuracy of the LFS method.

Comparison of methods for weave pattern repeat recognition and weave pattern recognition

In the third experiment, the LFS method and six improved methods are used respectively. The recognition of the weave pattern repeat only adds one step to Algorithm 2, which uses both the results of weave point classification by FCM and weave pattern repeat size recognition. According to Xiao et al.’s paper,¹¹ the pre-recognized weave pattern matrix obtained from weave point classification is divided into sub-matrices. These sub-matrices are distributed in the pre-recognized weave pattern matrix adjacently. The size of every sub-matrix is the recognized weave pattern repeat size. Then the corresponding elements in all sub-matrices are calculated. The element values appeared most often in the corresponding position of all sub-matrices forming a matrix, which is considered as the weave pattern repeat matrix.

In this paper, because of the random nature of FCM, warp floats and weft floats are divided into two groups instead of being represented by 0 and 1 strictly. If the weave pattern repeat matrix meets the following two conditions, it is considered to be correctly identified: (a) elements in the matrix have two values, with one indicating the warp float and the other indicating the weft float, that is, the matrix will form the weave pattern by tiling in the horizontal and vertical directions on a plane; (b) the size of the matrix is the smallest to meet condition (a). According to the definition of a correctly identified weave pattern repeat, the recognition rates of the weave pattern repeat based on different repeat size methods are listed in Table 3. As the two influential factors of weave pattern repeat recognition are the repeat size and the classification effect of weave points, the recognition rate of weave pattern repeat depends on the recognition rate of weave pattern repeat size, as can be seen in Table 2.

Table 3.

Comparison of Local Feature Similarity (LFS), Xiao et al.’s method and the six improved methods for weave pattern repeat recognition

Method	Recognition rate for plain	Recognition rate for twill and its expansion	Recognition rate for satin	Recognition rate for the whole samples
LFS method	99.51% (204 in 205)	100% (34 in 34)	38.46% (10 in 26)	93.59% (248 in 265)
Xiao et al.’s¹¹ method	88.29% (181 in 205)	97.06% (33 in 34)	34.62% (9 in 26)	84.15% (223 in 265)
Improved IDMF method	93.66% (192 in 205)	97.06% (33 in 34)	34.62% (9 in 26)	88.30% (234 in 265)
Improved gray autocorrelation method	89.76% (184 in 205)	82.35% (28 in 34)	30.77% (8 in 26)	83.02% (220 in 265)
Improved Ajallouian et al.’s²⁴ method	83.90% (172 in 205)	73.53% (25 in 34)	30.77% (8 in 26)	77.74% (206 in 265)
Improved pattern autocorrelation method	72.20% (148 in 205)	82.35% (28 in 34)	30.77% (8 in 26)	69.43% (184 in 265)
Improved EL method	88.78% (182 in 205)	91.18% (31 in 34)	30.77% (8 in 26)	83.40% (221 in 265)
Improved Zhang et al.’s⁷ method	99.02% (203 in 205)	88.24% (30 in 34)	38.46% (10 in 26)	91.70% (243 in 265)

IDMF: Improved Distance Matching Function; EL: Extraction and Location.

In the current weave pattern recognition method, the weave pattern repeat size is usually used in the error correction step. The specific process replaces all the sub-matrices divided from the weave pattern matrix as the weave pattern repeat matrix recognized above. The correct identification conditions are different from the weave pattern repeat recognition: (a) each element in the weave pattern matrix corresponds with a weave point at the same position on the fabric image; (b) warp floats and weft floats are represented by two different values; (c) the matrix can reflect the weave pattern. With the above conditions, the recognition rate for the weave pattern is calculated as shown in Table 4. Obviously, the error correction step can increase the recognition ratio of a weave pattern significantly.

Table 4.

Comparison among Local Feature Similarity (LFS), Xiao et al.’s method and the six improved methods for weave pattern recognition

Method	Recognition rate for plain	Recognition rate for twill and its expansion	Recognition rate for satin	Recognition rate for the whole samples
Pre-recognition weave pattern without error correction	6.34% (13 in 205)	2.94% (1 in 34)	0% (0 in 26)	5.28% (14 in 265)
LFS method	99.02% (203 in 205)	67.65% (23 in 34)	38.46% (10 in 26)	89.06% (236 in 265)
Xiao et al.’s¹¹ method	92.68% (190 in 205)	67.65% (23 in 34)	34.62% (9 in 26)	83.77% (222 in 265)
Improved IDMF method	99.02% (203 in 205)	55.88% (19 in 34)	38.46% (10 in 26)	87.55% (232 in 265)
Improved gray autocorrelation method	99.02% (203 in 205)	58.82% (20 in 34)	34.62% (9 in 26)	87.55% (232 in 265)
Improved Ajallouian et al.’s²⁴ method	97.07% (199 in 205)	52.94% (18 in 34)	34.62% (9 in 26)	85.28% (226 in 265)
Improved pattern autocorrelation method	97.07% (199 in 205)	64.71% (22 in 34)	34.62% (9 in 26)	86.80% (230 in 265)
Improved EL method	97.56% (200 in 205)	64.71% (22 in 34)	34.62% (9 in 26)	87.17% (231 in 265)
Improved Zhang et al.’s⁷ method	98.54% (202 in 205)	61.76% (21 in 34)	38.46% (10 in 26)	87.92% (233 in 265)

IDMF: Improved Distance Matching Function; EL: Extraction and Location.

The weave pattern repeat recognition focuses on the repeat unit, while the weave pattern recognition focuses on the whole structure corresponding to the fabric image: the judgment standards of the recognition ratio are different. The influence factors of weave pattern repeat recognition and weave pattern recognition are very complicated, which include the repeat size, the classification result of weave points and the segmentation performance of weave points, and may lead to opposite results for these two recognitions based on respective judgment standards. If weave points were mis-segmented, the result of weave pattern recognition must be wrong, but that of weave pattern repeat recognition maybe right, for example, LFS and improved Zhang et al.’s methods. If an error size of the weave pattern repeat is obtained, particularly several times that of the real size, the result of the weave pattern recognition repeat maybe wrong but that of the weave pattern recognition maybe right, for example, the improved EL method.

Moreover, even if the size of the weave pattern repeat is calculated to be the integral multiple of the right size, the pre-recognized weave pattern can be corrected with high probability. As a result, compared with Table 2, the higher recognition rate for weave pattern repeat size is also helpful to the higher recognition rate for the weave pattern.

Conclusion

This paper proposes a weave pattern repeat recognition method with high versatility, applicability, robustness and accuracy based on a comprehensive review of the existing methods in the first section, and the characteristic analysis of the weave pattern in the second section. Seven characteristics of weave pattern repeat are summed up in this paper. Experiments proved that more characteristics of weave pattern repeat used in the weave pattern repeat size recognition method may lead to a higher recognition rate for the method. The methods designed with all seven characteristics have the highest recognition rate.

Further work will be devoted to the better adaptability and expansion of the LFS method for more application scenarios. For example, when recognizing the repeat size of double-yarn weave patterns, the two yarns with same weaving method can be merged into a single yarn before using the LFS method. In addition, when the correlation coefficient is not suitable for calculating the similarity between two yarns based on the extracted features some other methods, such as the Euclidean distance, Hamming distance, chi-square test etc., can be used. Another further work will be devoted to improve the weave pattern recognition rate for satin weaves. The satin weave is an unsteady structure in that the yarns of satin weaves are twisted out more easily. A more accurate weave point segmentation method, weave point features extraction method and weave point classification are needed to handle the highly variable shape of yarns. The deep learning methods, such as Convolutional Neural Network and Recurrent Neural Network, provide an option for the future.

Footnotes

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship and/or publication of this article.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was granted by Tianjin Sci-tech Planning Projects (Grant No. 14RCGFGX00846), the Natural Science Foundation of Hebei Province, China (Grant No. F2015202239), Tianjin Sci-tech Planning Projects (Grant No. 15ZCZDNC00130), Tianjin Sci-tech Planning Projects (Grant No. 17ZLZDZF00040) and Tianjin Sci-tech Planning Projects (Grant No.14JCYBJC18500).

References

Zheng

Wang

. Multi-scale density detection for yarn-dyed fabrics with deformed repeat patterns. Text Res J 2017; 87: 2524–2540.

Osborne, EF. Weaving process for production of a full fashioned woven stretch garment with load carriage capability. Patent No. 7,841,369, USA, 2010.

Akgun

. The effect of fabric balance and fabric cover on surface roughness of polyester fabrics. Fiber Polym 2013; 14: 1372–1377.

Schröder

Zinke

Klein

. Image-based reverse engineering and visual prototyping of woven cloth. IEEE Trans Visualiz Comput Graphic 2015; 21: 188–200.

Kuo

C-FJ

Shih

C-Y

C-E

et al.

Application of computer vision in the automatic identification and classification of woven fabric weave patterns. Text Res J 2010; 80: 2144–2157.

Pan

Gao

Liu

et al.

Automatic recognition of woven fabric patterns based on pattern database. Fiber Polym 2010; 11: 303–308.

Zhang

Pan

Gao

et al.

Automatic recognition of the color effect of yarn-dyed fabric by the smallest repeat unit recognition algorithm. Text Res J 2015; 85: 432–446.

Lim

Kim

. Analysis of woven fabric structure using image analysis and artificial intelligence. Fiber Polym 2011; 12: 1062–1068.

Jing

. Automatic recognition of weave pattern and repeat for yarn-dyed fabric based on KFCM and IDMF. Optik 2015; 126: 2876–2883.

10.

Zheng

. A new method for the detection and classification of weave pattern repeat. Text Res J 2014; 84: 1586–1599.

11.

Xiao

Nie

Zhang

et al.

Recognition for woven fabric pattern based on gradient histogram. J Text Inst 2014; 105: 744–752.

12.

Guarnera

Hall

Chesnais

et al.

Woven fabric model creation from a single image. ACM Trans Graph 2017; 36: 1–13.

13.

Wang

Georganas

Petriu

. Fabric texture analysis using computer vision techniques. IEEE Trans Instrum Meas 2011; 60: 44–56.

14.

Zhou

Sheng

et al.

A novel weave/color repeat extraction method with error tolerance. J Eng Fabric Fiber 2014; 9: 127–139.

15.

Fasahat

Payvandy

. A novel hybrid genetic and imperialist competitive algorithm for structure extraction of woven fabric images. J Text Inst 2017; 108: 893–905.

16.

Salem

Nasri

. Automatic recognition of woven fabrics based on texture and using SVM. Sign Image Video Proc 2010; 4: 429–434.

17.

Bao X, Ni X, Wang Y, et al. Recognition of fabric structures based on improved olfactory neural network. In: Proceedings of International Conference on Artificial Intelligence and Computational Intelligence (AICI 2010) (ed FL Wang, H Deng, J Lei), Sanya, China, 23–24 October 2010, pp. 325–328. Washington, DC: IEEE Computer Society.

18.

Pan

Gao

Liu

et al.

Automatic detection of structure parameters of yarn-dyed fabric. Text Res J 2010; 80: 1819–1832.

19.

Pan

Gao

Liu

et al.

Automatic recognition of woven fabric pattern based on image processing and BP neural network. J Text Inst 2011; 102: 19–30.

20.

Kang

Kim

. Automatic recognition of fabric weave patterns by digital image analysis. Text Res J 1999; 69: 77–83.

21.

Zhang

Wang

Pan

et al.

A computer vision-based system for automatic detection of misarranged warp yarns in yarn-dyed fabric. Part I: continuous segmentation of warp yarns. J Text Inst 2017; Aug 6: 1–8.

22.

Sabuncu

Ozdemir

Akdogan

. Automatic identification of weave patterns of checked and colored fabrics using optical coherence tomography. IEEE Photon J 2017; 9: 1–8.

23.

Sabuncu

Özdemir

. Recognition of fabric weave patterns using optical coherence tomography. J Text Inst 2016; 107: 1406–1411.

24.

Ajallouian

Tavanai

Palhang

et al.

A novel method for the identification of weave repeat through image processing. J Text Inst 2009; 100: 195–206.

25.

Zhong

Shi

Chen

et al.

Research on digital intelligent recognition method of the weave pattern of fabric based on the redundant information. Fiber Polym 2013; 14: 1919–1926.

26.

Zhong

Shi

et al.

Research on computer-aided analysis and reverse reconstruction for the weave pattern of fabric. Text Res J 2013; 83: 298–310.

27.

Wang X, Georganas ND and Petriu EM. Automatic woven fabric structure identification by using principal component analysis and fuzzy clustering. In: proceedings of 2010 IEEE international instrumentation and measurement technology conference (I2MTC2010), Austin, TX, USA, 3–6 May 2010, pp. 590–595. Washington, DC: IEEE Computer Society.

28.

Zhou

Sun

. A novel feedback error-correcting algorithm for automatic recognition of the color and weave pattern of yarn-dyed fabrics. Text Res J 2013; 83: 1673–1689.

29.

Schneider

Merhof

. Blind weave detection for woven fabrics. Pattern Anal Appl 2015; 18: 725–737.

30.

Smith

. Color gamut transform pairs. ACM Siggraph Comput Graph 1978; 12: 12–19.