ClothNet: sensitive semantic segmentation network for fabric defect detection

Abstract

When the conventional semantic segmentation method is applied to fabric defect detection, the omission factor of small size defects is relatively high, and the network model with larger depth is easy to lose the features of small size defects and has poor real-time performance. To address these problems, we propose two sensitive semantic segmentations, ClothNet based on deep feature fusion and ClothNet-tiny based on atrous spatial pyramid pooling. First, in ClothNet, deep and shallow features are fused to compensate for the information loss caused by pooling. Second, ClothNet-tiny is designed to improve the detection speed. Finally, an adaptive loss function for defect size, namely weighted dice loss is proposed. The results on the validation set show that ClothNet achieves 78.8% Mean Intersection over Union mean. Compared to fully convolutional networks, ClothNet reduces memory consumption by 28% and ClothNet-tiny by 77%.

Keywords

Fabric defect detection deep feature fusion atrous spatial pyramid pooling semantic segmentation ClothNet weighted dice (WD) loss

Fabric defects are the biggest factor affecting fabric quality, thus reducing fabric availability.^1,2 Defect detection is not only used to evaluate fabric quality, but also to infer machine operation.³ Currently, fabric defects are mainly detected manually, and human attention and proficiency are very limited, even skilled workers can only achieve about 70% detection accuracy.⁴ The process of defect detection affects productivity and results in significant labor costs.⁵

In recent years, with the rapid development of computer vision, many fabric defect detection algorithms have emerged.⁶ These algorithms can be divided into four major categories: statistical algorithms, spectral algorithms, dictionary learning, and deep learning.⁷

As fabric defects are random, statistical algorithms were first studied, in which the images were preprocessed and then statistically identified. A typical approach was the Sylvester matrix similarity estimation algorithm proposed by Kumari et al.⁸ The algorithm involved six stages: resolution matching, image enhancement using histogram specification and median mean-based sub-image-clipped histogram equalization, image registration by alignment and hysteresis processes, image subtraction, edge detection, and fault detection by rank of the Sylvester matrix. The algorithm achieved 93% accuracy on KTH-TIPS-I and KTH-TIPS-II, but the real-time performance was poor.

As the texture of fabrics was periodic, spectral algorithms were used to model the texture and used the model to detect defects. Jing et al.⁹ used a genetic algorithm to adjust the Gabor filter on a defect-free fabric to match the defect-free texture, and then used the adjusted filter to detect defects, which achieved good results on plain fabrics but could not handle color fabrics. Schneider and Merhof¹⁰ proposed a Fourier analysis-based algorithm for woven fabric detection, extracting distinct peaks to obtain texture frequencies and combining template matching and fuzzy clustering algorithms to extract warp and weft angles to obtain weave pattern information and construct a yarn map. The method obtained 97% accuracy on a dataset including 140 real fabric images, but the applicability range was too small, and the sampling requirement was high.

Mimicking the process of training manual fabric defect detection, visual inspection methods based on biological vision and dictionary learning have been proposed, which in turn were both low-rank models and thus were also referred to as low-rank decomposition. Li et al.¹¹ integrated dictionary learning and Laplace regularization into a low-rank representation model concerning the low-rankness and sparsity of biological vision. Dictionary learning reduced the noise of saliency maps, and the addition of Laplace regularization enlarged the grayscale difference between defects and background; finally, the generated visual salient maps were segmented. This method could achieve better results on fabrics with complex patterns, but the huge computational cost made it less real time. Noting that the traditional low-rank decomposition model ignored the effect of Gaussian noise, Shi et al.¹² proposed a low-rank decomposition model with noise regularization and gradient information. The algorithm used noise regularization to expand the gradient between defects and nondefects and constrained the noise terms based on the gradient information to guide the matrix decomposition, which achieved good results in recall and precision, but failed in real time.

In recent years, deep learning has become the preferred choice for many vision tasks. Neural network models can understand scenes like a human. Many researchers have applied deep learning to the fabric detection problem with satisfactory results. Liu et al.¹³ and Simonyan and Zisserman¹⁴ classified defective images based on the VGG16 neural network model and pruned the original VGG16 network model by visualizing the feature maps to maximize the model performance while reducing the network parameters. The tests on the self-constructed dataset achieved good classification performance. The disadvantage was that the model could only distinguish the presence or absence of defects, but could not obtain the size, type, and number of defects. Jing et al.¹⁵ proposed a sliding window recognition method based on classification networks, which could achieve a general judgment of defect contours. However, to obtain specific contour and classification information, intensive segmentation of the image is required, leading to an increase in computational effort, and good real-time performance cannot be obtained.

In response to the above-mentioned deep learning networks that do not reflect the location and size of defects well, object-based detection methods,¹⁶ Pixel-level segmentation and semantic segmentation have now been developed. Here is what they have done for fabric defects.

Li used the R-CNN model for identifying fabric defects,¹⁷ the previous bounding box size of R-CNN was obtained based on the clustering of fabric defect sizes. The traditional nonmaximum suppression method was replaced by a soft nonmaximum suppression method to avoid incorrect discarding of prediction bounding boxes due to stacking,¹⁸ but the large aspect ratio was not sufficient for fabric defects such as broken yarns.¹⁹

The size of the prediction bounding box does not reflect the actual size of the defect, pixel-level segmentation of defects is a better alternative. Huo et al.²⁰ used a Bayesian optimization to prune UNet++, denoise and labeling of connected domains on the feature map output of the neural network. The threshold was set to estimate a connected domain defect. The method reduced the inference time by 24% with a 1.4% reduction in Intersection over Union. The method was able initially to identify the contours of defects, but the detection process could not be performed end to end, and the defects could not be classified.

Semantic segmentation provides predictive classification for every pixel.²¹ It not only segments the defects but also identifies the classes of defects, and provides all information about the defects.²² Jing et al.²³ used the network structure of UNet, replaced backbone with MobileNetv2 which had fewer parameters, and used depthwise separable convolution to improve the inference speed, achieving 70% accuracy (mean intersection over union; MIoU) on the self-built dataset. Semantic segmentation is usually used in robotics and autonomous driving,²⁴ some classic network structures have emerged; however, most of these networks are used for semantic segmentation of complex scenes. For the fabric defect detection task, the scene semantics are relatively homogeneous, which means that these networks are too deep, meanwhile some networks have huge receptive fields, while fabric defects are generally small and narrow, thus making such small-scale features easily overlooked in these networks.

Therefore, it is necessary to redesign the networks to cope better with the task of fabric defect detection. Meanwhile, in semantic segmentation network training, the imbalance of positive and negative samples in the dataset can lead to nonconvergence or slow convergence of the model, which can be solved by a weighted loss function. The weights are often set artificially and need to be adjusted several times. We designed a new loss function that adaptively balanced the positive and negative samples and had a more reasonable weight assignment to improve the training effect for small size categories. We also used a decaying cosine annealing learning rate to prevent the model from falling into a local optimum and to reduce model performance oscillations caused by restarting the learning rate. In summary, the proposed approach has the following contributions:

WD loss with size adaptive function is designed to solve the sample imbalance problem, the size adaptation function is a nonlinear function based on the pixel occupancy of positive samples. It is proved that WD loss can converge the model better than Dice loss, and improve the mean MIoU of the model by 4.7% and the hole MIoU by 9.2%.

ClothNet is designed based on deep feature fusion, maintaining a high feature map size by dilated convolution makes small-scale defects less likely to be lost. ClothNet had a miss detection rate of 1.3% and a false detection rate of 2.0% on the test set.

ClothNet-tiny uses a tandem network structure, compared to fully convolutional networks (FCNs), ClothNet-tiny reduces memory cost by 77%. ClothNet-tiny had a miss detection rate of 2.7% and a false detection rate of 3.4% on the test set.

A decaying cosine annealing function is used to prevent the model from falling into a local optimum and to ensure the stability of the final training model.

Models

This section describes the proposed ClothNet and ClothNet-tiny network models in detail. ClothNet uses an encoding–decoding network structure, including a backbone part and an upsampling part, while using deep feature fusion to compensate for the information loss due to pooling.²⁵ ClothNet-tiny performs atrous spatial pyramid pooling after the backbone, and then concates the feature maps that form the dilation convolution. The concated feature map was sent to the upsampling section to recover the original resolution. All convolutional layers are followed by batch normalization,²⁶ and LeakyReLU is used as the activation function with $α = \frac{1}{5.5}$ in LeakyReLU.²⁷

ClothNet

Residual blocks are used to build the backbone. Networks with greater depth are generally considered to have better performance, but in practice, deeper networks degrade. Residual connection solves the problem of network degradation by establishing identity mapping. In Figure 1, the 1 × 1 convolution in the residual block not only realizes the fully connected in the channel dimension, but is also flexible for decreasing and increasing dimensions. Convolution of the dim-decreased tensor with larger kernel size N × N will reduce the calculation volume. The convolution of N × N is used to enhance the network receptive field and increase the network parameters. Figure 1(a) shows the structure when the input and output dimensions are different, and Figure 1(b) shows the structure when the input and output dimensions are the same. The residual block is widely used in backbone networks because of its excellent performance and fast speed.²⁸

Figure 1.

Residual block: (a) different input and output dimensions and (b) same input and output dimensions.

The output of the $x -$ 1 layer $F_{m}^{x - 1}$ is adjusted by 1 × 1 convolution and then maxpooling, got ${F_{m}^{x - 1'}}^{}$ . The output of the $x$ layer $F_{m}^{x}$ is added with ${F_{m}^{x - 1'}}^{}$ , the result is used as the input of the $x +$ 1 layer, thus achieving deep feature fusion.

Referring to Figure 2(a), when the input is 240 × 240, the minimum span of defects is concentrated between 8 and 15 pixels. In Figure 2(b) to (e), as the feature map size decreases, the defect contours become blurred, which also means that subsequent contour reduction becomes difficult. When the encoder downsamples the feature map to 60 × 60 size, the minimum span of defects occupies only 2–3 pixels, and 30 × 30 is the minimum size to ensure that the features are not scaled. To ensure the detection rate of defects with small size and the resolution restoration in the subsequent contour inference process, dilated convolution is selected instead of maxpooling.

Figure 2.

Downsampling size comparison: (a) image 240 × 240; (b) feature map 240 × 240; (c) feature map 120 × 120; (d) feature map 60 × 60 and (e) feature map 30 × 30.

The dilated convolution can replace the receptive field expansion brought by the maxpooling without changing the feature map size.²⁹ Dilated convolution used a kernel size of 3 × 3 and dilation rate of 2, and the end of the encoder section. Figure 3 shows the 3 × 3 convolution kernel for different dilation rates.

Figure 3.

Dilated convolution kernel with different dilation rates (Dr).

The calculation of the encoder receptive field is shown in equation (1). Where ${RF}_{n}$ is the receptive field of the layer $n$ , ${stride}_{n}$ is the stride of convolutional $n$ , ${kernel}_{n}$ is the kernel size of convolutional $n$ . Calculation of receptive field from top to bottom; the receptive field of ClothNet encoder is 42 × 42, it is enough to cover the full size of the defect.

{RF}_{n} = {(RF}_{n - 1} - 1) \times {stride}_{n} + {kernel}_{n}

(1)

The decoder part uses transpose convolution and tandem structure to form the upsampling layer, transpose convolution reduces the number of channels while restoring the resolution of the image, and convolution further extract features and infer the class to which the pixels belong. Outputs of different scales of the backbone layer are summed with the matched upsampled feature maps to achieve large span feature fusion which benefits the contour derivation. Finally, the classification probabilities of pixels are obtained by SoftMax activation function. Figure 4 shows the network structure of ClothNet.

Figure 4.

ClothNet.

ClothNet-tiny

Figure 5 shows the network structure of ClothNet-tiny. ClothNet-tiny also uses the residual block as the basic layer of the backbone. In the residual block, the input dimensions are aligned with the output dimensions, the convolution is used instead of maxpooling for downsampling. The minimum downsampling size of the backbone is 30 × 30, it is the minimum downsampling size that keeps the minimum span of the defect from scaling with higher speed. The receptive field of ClothNet-tiny encoder is 38 × 38. Dilated convolution is used to process the feature maps output from the backbone feature network, the dilation rates are 1, 2, 3, 4. The extracted feature maps are concated as inputs to the decoder network for scale diversity. Compared with ClothNet’s decoder network, ClothNet-tiny does not use the structure of large span feature fusion, and the whole decoder network is connected in tandem.

Figure 5.

ClothNet-tiny.

Comparison

According to Table 1, compared to FCN, ClothNet has 90% fewer parameters and ClothNet-tiny has 93% fewer. Even compared to PSPNet, which is more lightweight, ClothNet reduced by 44% and ClothNet-tiny reduced by 62%. Multiscale feature fusion and large span feature fusion structures are used in ClothNet, which results in better network performance, but such structures involve temporary storage of a large number of tensors, which can lead to memory usage. The higher downsampling rate of ClothNet-tiny makes it faster. As seen from Figure 6(a), for the fabric defect detection task, the large fabric width makes it necessary for the images to be segmented. Multiple images will be packaged into one batch for network prediction, which makes the video memory occupation very important. ClothNet-tiny eliminates a large number of feature fusions and the entire network is basically structured in tandem mode, making the video memory occupation much lower. In Figure 6(b), compared with FCN, ClothNet memory cost reduced by 28%, ClothNet-tiny reduced by 77%.

Table 1.

Models contrast

Model	Parameters (number of floats)	Defect-free MIoU (%)	Carrying MIoU (%)	Broken yarn MIoU (%)	Knot MIoU (%)	Hole MIoU (%)	Soiled yarn MIoU (%)
ClothNet	1.9M	99.4	79.4	72.0	77.2	71.7	72.4
ClothNet-tiny	1.3M	99.3	75.6	69.0	63.5	68.0	68.6
UNet	4.9M	99.3	74.5	65.9	62.4	47.4	64.3
FCN	19.0M	99.3	77.9	69.3	74.4	66.7	69.4
PSPNet	3.4M	99.2	75.3	62.8	67.3	63.9	67.4

MIoU: mean intersection over union; FCN: fully convolutional network.

Figure 6.

Image segmentation and video memory cost.

Training

Loss function

In semantic segmentation, positive and negative samples are unbalanced, and the number of negative samples is usually much larger than the number of positive samples. Figure 7(a) shows the distribution of positive and negative samples in the dataset; the pixel occupancy of positive samples is only $\frac{1}{6}$ of negative samples. Figure 7(b) shows the pixel occupancy of different classes in positive samples, which can visually reflect the problem of unbalanced data. Using traditional loss functions (e.g. cross-entropy loss) leads to fast convergence of negative samples and lagging or slow convergence of positive samples.³⁰ Therefore, sample balancing of the loss function is necessary.

Figure 7.

Distribution of samples: (a) number of positive and negative sample pixels and (b) number of pixels in different classes.

The Dice loss provides a good indication of the similarity between predicted results and ground truth.³¹ To measure the effect of negative samples on the loss values, a size-adaptive weighting function designed for Dice loss, denoted by $L_{WD}$ , named WD loss.

L_{dice} = 1 - \frac{2 \cdot |(GT ⋂ P)|}{|GT| + |P|}

(2)

PR = \frac{{PN}_{obj}}{PN}

(3)

L_{WD} = R_{c} {\cdot e}^{- \sqrt{P R}} L_{dice} (obj) + (1 - R_{c} \cdot e^{- \sqrt{P R}}) L_{dice} (b g)

(4)

where

GT

is the ground truth,

P

is the network prediction output,

{PN}_{obj}

is the number of positive sample pixels,

PN

is the total number of sample pixels,

R_{c}

is an adjustable constant, and

R_{c} =

0.8 during training. After equalization, the loss weight percentage of positive samples increases. The case of small size defect loss being diluted is solved, thus avoiding the problems of positive and negative sample imbalance and positive sample imbalance in each class.

Learning rate

The learning rate plays an important role in the convergence process of the network model. In general, the learning rate gradually decreases as the number of training epochs increases so that the model eventually converges. In the case where the objective function is multipeaked, such a strategy can also cause the model to fall into local optimization rather than global optimization. The cosine annealing learning rate is used to restart the learning rate by setting a progressively longer period and setting the learning rate to the initial value at the beginning of each period, thus jumping out of the local optimum.³²

The cosine annealing function also has its drawbacks: restarting the learning rate after a certain number of training iterations leads to large oscillations in model performance, slow convergence, and a decay factor is needed to constrain the learning rate.

N = \frac{{epoch}_{i}}{{30 \cdot T}_{r}^{{0.1 \cdot epoch}_{i}}}

(5)

n_{i + 1} = n_{\min} + 0.5 \cdot (n_{\max} - n_{\min}) D_{r}^{0.5 \cdot {epoch}_{i}} (1 + \cos ((N - ⌊N⌋) π))

(6)

Where

{epoch}_{i}

is the epoch value of the current batch,

T_{r}

is the growth rate,

n_{\min}

is the minimum learning rate threshold,

n_{\max}

is the maximum learning rate threshold,

and D_{r}

is the amplitude decay rate. The learning rate function is shown in Figure 8.

Figure 8.

Learning rate.

Experimental evaluation

In this section, the performance of ClothNet and ClothNet-tiny will be evaluated. All training and testing performed on an IPC with an Intel Core i7-6850K CPU, NVIDIA GeForce GTX 1080 Ti GPU, and Windows 10 OS; Adam was the optimizer for both networks, $β_{1} =$ 0.9, $β_{2} =$ 0.999. Training of ClothNet sets the batch size to 24; ClothNet-tiny sets the batch size to 64.

Dataset

Both training and testing were performed using a self-built dataset containing six classes: defect-free, carrying, broken yarn, knots, holes, and soiled yarn. The original image is acquired from a 1920 × 1440 camera, and the image is divided into a 4 × 3 grid, where each subimage is 480 × 480 and the resize to 240 × 240. The images without defects are removed, only the images with defects are kept. Each image was manually labeled at the pixel level, with a total of 1451 images and 1846 defects. Table 2 shows the number of images and defects in each class. Some images have multiple disconnected defects, and each image contains at least one defect; 1303 images were selected as the training set and 148 images were selected as the test set. All images and ground truth were 240 × 240 in size. Figure 9 shows some defect images and ground truths of the dataset.

Table 2.

Number of images and defects

Defects	Carrying	Broken yarn	Knot	Hole	Soiled yarn
Number of images	425	308	184	112	422
Number of defects	427	378	224	113	704

Figure 9.

Dataset: (a) image and (b) ground truth.

Training and verification

First, mean intersection over union, $MIoU (c)$ , an evaluation metric for semantic segmentation task, is introduced, which is used to evaluate the accuracy of classification and segmentation of semantic segmentation. For class $c$ :

MIoU (c) = \frac{1}{n + 1} \sum_{i = 0}^{n} \frac{TP (c)}{FN (c) + FP (c) + TP (c)}

(7)

In equation (7), $TP$ (true positive): the prediction is correct, the prediction result is positive, the actual is positive. $FP$ (false positive): the prediction is wrong, the prediction is positive, the actual is negative. $FN$ (false negative): the forecast is wrong, the forecast result is negative, the actual is positive. $TN$ (true negative): the prediction is correct, the prediction is negative, the actual is negative. $n$ is the total number of samples, Figure 10 shows the schematic of $TP$ , $FP$ , $FN$ , $TN$ in IoU computing.

Figure 10.

Intersection over union (IoU) schematic.

A strategy is needed for deciding when to stop training the model, avoiding overfitting due to overtraining, and saving time.^33,34 Assume that $E$ is the error function of the model, $E_{t r} (t)$ is the error on the training set at $epoch = t$ , $E_{t e} (t)$ is the error on the test set at $epoch = t$ , and $E_{opt} (t)$ is assumed to be the best validation set error obtained at $epoch = t$ :

E (t) = 1 - MIoU (t)

(8)

E_{opt} (t) = \min_{t^{'} \leq t} E_{t e} (t')

(9)

A generalization loss $GL (t)$ is defined, which describes the proportional mean of $E_{t e} (t)$ and the historical minimum error $E_{opt} (t)$ over the validation period $k$ at $epoch = t$ :

GL (t) = \frac{\sum_{t^{'} = t - k + 1}^{t} E_{t e} (t')}{k \cdot E_{opt} (t)} - 1

(10)

In equation (10), we can find that the appearance of higher generalization loss in the later stages of training (for larger value of $t$ ) symbolizes the appearance of model overfitting, but the model validation oscillation in the early stages of training (for smaller value of $t$ ), can also lead to high generalization loss. Usually, overfitting only occurs when the training error is reduced very slowly, training measure progress $MP (t)$ is used to indicate the degree of model training:

MP (t) = \frac{\sum_{t^{'} = t - k + 1}^{t} E_{t r} (t^{'})}{k \cdot \min_{t^{'} = k + 1}^{t} E_{t r} (t^{'})} - 1

(11)

Set stop threshold $α$ , when $\frac{GL (t)}{MP (t)} \geq α$ , training will stop. Both ClothNet and ClothNet-tiny choose $k$ = 20, $α$ = 3. The loss-epoch profile of ClothNet and ClothNet is shown in Figure 11.

Figure 11.

Loss-epoch: (a) ClothNet and (b) ClothNet-tiny.

ClothNet was trained for 1000 epochs to verify the improvement of WD loss. Figure 12 shows the plots of MIoU-epochs for each class in the test set, Figure 12(a) shows that the defect-free class converges quickly in the case of imbalanced training samples. In Figure 12(c) to (f), comparing with Dice loss, it is obvious that WD loss makes these classes converge earlier, especially in the hole class in Figure 12(e), WD loss converged 502 epochs in advance. WD loss makes the training progress much earlier, which indicates that WD loss plays a good role in dealing with the problem of sample imbalance. To get a good performance model, training with WD loss could save more time. In Figure 12, the detection accuracy (MIoU) of the model trained with WD loss is higher than that of Dice loss, and Figure 12(d) to (f), the MIoU of small-sized defects broken yarn, knot, and hole reached 72.0%, 77.2%, and 71.7%, indicating that the size-adaptive weighting function in WD loss has a better detection performance for small size defects.

Figure 12.

Mean intersection over union (MIoU) with Dice loss and weighted dice (WD) loss: (a) defect-free; (b) carrying; (c) broken yarn; (d) knot; (e) hole and (f) soiled yarn.

Comparison

To test the performance of ClothNet and ClothNet-tiny, we selected models that performed well in the semantic segmentation task for comparison, including UNet, FCN, and PSPNet. The performance of the models was judged using MIoU as a metric, and all models were based on the same train and test set.

As can be seen from Table 1, ClothNet achieves the best results in all classes, and ClothNet-tiny achieves a good result with many fewer parameters than other networks. Both ClothNet and ClothNet-tiny show better results in the detection of the hole class with small size. UNet does well in contour segmentation but has poor classification ability. FCN’s huge number of parameters provides better accuracy. PSPNet fails to achieve better results in accuracy, but its smaller number of parameters is its obvious advantage.

In Figure 13, the detection results of each class are shown. In general, there are no missed detections in each model, but it is obvious that ClothNet and ClothNet-tiny have better classification results and there are no wrong detections. In the detection of holes with small size, the traditional model is less effective in inferencing about the defect contour and class, while ClothNet and ClothNet-tiny have excellent results.

Figure 13.

Outputs of different models.

In terms of detail performance, in Figure 13, we can observe that ClothNet’s segmentation of contours is superior to that of the traditional network and ClothNet-tiny, due to ClothNet’s minimum downsampling size of 60 × 60 and the use of a large number of feature fusion techniques. In the detection results of the broken yarn class in Figure 13, we can better see ClothNet’s superior performance in contour segmentation.

ClothNet reaches 48FPS and ClothNet-tiny reaches 99FPS, both satisfying real-time (30FPS). Packing groups of images into batches can effectively improve the efficiency of the network, and the larger the batch size for processing quantitative images, the higher the efficiency. If memory is sufficient, larger batch sizes are encouraged in tasks with high parallelism requirements. Table 3 shows the time taken to process 1000 images with different batch sizes.

Table 3.

Time cost with different batch

Model	Number of images	Batch size	Time cost (s)
ClothNet	1000	1	20.772
		20	14.601
		50	6.656
		100	0.908
ClothNet-tiny	1000	1	10.565
		10	2.642
		50	0.592
		100	0.070

Conclusion

To achieve end-to-end fabric defect detection, ClothNet and ClothNet-tiny were designed, both of which use an encoder–decoder architecture. ClothNet uses a large span of feature fusion to enhance defect contour segmentation, and a residual connection module to avoid network degradation and reduce calculation effort. ClothNet-tiny has fewer memory costs and faster detection speed, with lower hardware requirements in practice. ClothNet and ClothNet-tiny were trained and tested on a self-built dataset, then compared with traditional models.

ClothNet has only 10% of the parameters of FCN, and even compared to UNet and PSPNet, which have fewer parameters, ClothNet has 61.2% and 44.1% fewer parameters, respectively. Compared to ClothNet, ClothNet-tiny has 31.5% fewer parameters and only 4.7% lower performance (MIoU mean).

In the actual defect distribution, the samples will have serious imbalance, which will make the network insensitive to smaller size defects. The WD loss solves the imbalance problem for positive and negative samples by calculating them separately. The weights are calculated by the size-adaptive function, the weights corresponding to small size samples are distributed larger, which can solve the problem of imbalance between samples. Comparing with Dice loss, WD loss has higher detection accuracy, improving 2.7%, 1.8%, 10.3%, 15.1%, 5.6% in five defect classes, respectively, 502 epochs earlier convergence in the hole class, and the trend of earlier convergence is also reflected on other classes.

Current defect detection methods still require manual data collection and labeling, and for printed fabrics, for example, with their numerous and frequently updated patterns, such supervised networks are not well adapted to this way of working. However, semisupervised or unsupervised trained neural networks can be deployed faster. In addition, we can achieve the semantic generation of defect-images through Generative Adversarial Network (GAN), and input the random labels and images to the semantic segmentation network for supplementation, which can solve the problem of insufficient data to some extent, and this will be our future research direction.

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Tuanshan Zhang

References

Ngan

Pang

Yung

NH.

Automated fabric defect detection – a review. Image Vision Comput 2011; 29: 442–458.

Song

Jing

Huang

, et al. EfficientDet for fabric defect detection based on edge computing. Journal of Engineered Fibers and Fabrics 2021; 16: 1–13.

Zhu

Han

Jia

, et al. Modified densenet for automatic fabric defect detection with edge computing for minimizing latency. IEEE Internet Things J 2020; 7: 9623–9636.

Zhou

Akankwasa

, et al. Fabric texture representation using the stable learned discrete cosine transform dictionary. Text Res J 2019; 89: 294–310.

Liu

Wang

, et al. Multistage GAN for fabric defect detection. IEEE Trans Image Process 2019; 29: 3388–3400.

Dlamini

Kao

C-Y

S-L

, et al. Development of a real-time machine vision system for functional textile fabric defect detection using a deep YOLOv4 model. Textile Research Journal 2022; 92: 675–690.

, et al. Fabric defect detection in textile manufacturing: a survey of the state of the art 2021; 2021: 1–13.

Kumari

Bandara

Dissanayake

MBJJoS

Sylvester Matrix-Based Similarity Estimation Method for Automation of Defect Detection in Textile Fabrics

2021; 2021: 1–11.

Jing

Yang

, et al. Supervised defect detection on textile fabrics via optimal Gabor filter. J Indust Text 2013; 44: 40–57.

10.

Schneider

Merhof

Blind weave detection for woven fabrics. Pattern Anas Appl 2015; 18: 725–737.

11.

Gao

Liu

, et al. Fabric defect detection based on biological vision modeling. IEEE Access 2018; 6: 27659–27670.

12.

Shi

Liang

, et al. Fabric defect detection via low-rank decomposition with gradient information. IEEE Access 2019; 7: 130423–130437.

13.

Liu

Zhang

, et al. Fabric defect recognition using optimized neural networks 2019; 14: 1–10.

14.

Simonyan

Zisserman

Very deep convolutional networks for large-scale image recognition. arXiv preprint 2014; arXiv:1409.1556.

15.

Jing

Zhang

HH.

Automatic fabric defect detection using a deep convolutional neural network. Colorat Technol 2019; 135: 213–223.

16.

Liu

, et al. Fabric defects detection based on SSD. In: Proceedings of the 2nd International Conference on Graphics and Signal Processing, October 6–8, 2018, NSW, Sydney, Australia, pp. 74–78.

17.

Bag of tricks for fabric defect detection based on Cascade R-CNN. Text Res J 2020; 91: 599–612.

18.

Bodla

Singh

Chellappa

, et al. Soft-NMS – improving object detection with one line of code. In: Proceedings of the IEEE International Conference on Computer Vision, October 22–29, 2017, Venice, Italy, pp. 5561–5569.

19.

Yang

Multi-evidence filtering and fusion for multi-label classification, object detection and semantic segmentation based on weakly supervised learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, June 18–22, 2018, Salt Lake City, Utah, USA, pp. 1277–1286.

20.

C-C

Chou

W-C

Deep convolutional neural network optimization for defect detection in fabric inspection. Sensors 2021; 21: 7074.

21.

Yang

Soatto

FDA: Fourier domain adaptation for semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, June 14–19, 2020, Online, pp. 4085–4095.

22.

Romera

Alvarez

Bergasa

, et al. Erfnet: Efficient residual factorized convnet for real-time semantic segmentation. IEEE Trans Intell Transport Syst 2017; 19: 263–272.

23.

Jing

Wang

Rätsch

, et al. Mobile-Unet: An efficient convolutional neural network for fabric defect detection. 2022; 92: 30–42.

24.

Watanabe

Karasawa

, et al. MFNet: Towards real-time semantic segmentation for autonomous vehicles with multi-spectral scenes. In: 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Sept 24–28. 2017, Vancouver, BC, Canada, pp. 5108–5115. IEEE.

25.

Liu

Ding

Scene classification based on two-stage deep feature fusion. IEEE Geosci Remote Sens Lett 2017; 15: 183–186.

26.

Ioffe

Szegedy

Batch normalization: Accelerating deep network training by reducing internal covariate shift. In: International Conference on Machine Learning, July 6–11, 2015. Lille France,, pp. 448–456. PMLR.

27.

Wang

Chen

, et al. Empirical evaluation of rectified activations in convolutional network. arXiv preprint 2015; arXiv:1505.00853.

28.

Zhang

Ren

, et al. Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016. June 26th–July 1st, Las Vegas, Nevada, USA, pp. 770–778.

29.

Koltun

Multi-scale context aggregation by dilated convolutions. arXiv preprint 2015; arXiv:1511.07122.

30.

Lin

T-Y

Goyal

Girshick

, et al. Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, October 22–29, 2017, Venice, Italy, pp. 2980–2988.

31.

Sun

Meng

, et al. Dice loss for data-imbalanced NLP tasks. arXiv preprint 2019; arXiv:1911.02855.

32.

Loshchilov

Hutter

Decoupled weight decay regularization. arXiv preprint 2017; arXiv:1711.05101.

33.

Prechelt

Early stopping-but when? In: Neural Networks: Tricks of the trade. Berlin, Heidelberg: Springer, 1998, pp. 55–69.

34.

Havelka

Nagy

Tunák

, et al. Testing the effect of textile materials on car seat comfort in real traffic. Journal of Industrial Textiles 2021; 51: 740–767.