Automated defect detection in nanomaterial-coated-fabrics using variational autoencoder

Abstract

This paper introduces an unsupervised method for detecting regions with a high density of nanomaterials on coated fabric using Variational Autoencoder, a generative model capable of learning dominant features of the input data and generating similar outputs. The model was applied to images of cotton fabric coated with commercial single-walled carbon nanotubes (SW-CNT) to learn their features. Morphological and electrical properties of the coated samples were initially investigated to identify high-density particle regions. Subsequently, an experiment evaluated the performance of these samples in a specific smart textile application, establishing ground truth for defect localization in each fabric image. Image post-processing techniques were then employed to accurately detect defective regions in test images. The proposed method achieved a recognition rate of 93.2% and the highest Intersection over Union (IoU) of 0.923. This study demonstrates a promising approach for defect identification in coated fabrics, advancing smart textile technology.

Keywords

Defect detection deep learning generative model variational autoencoder SWCNT smart textiles

Introduction

Detecting defects in fabric coated with nanomaterials holds significant importance across various domains due to their applications in functionality, sensing, responsive textiles, and healthcare/wearable technology. Previous research^1–4 has elucidated that fabric coated with nanomaterials, such as single-walled carbon nanotubes (SW-CNT), exhibits enhanced conductivity, enabling multifunctionality, allowing the creation of smart textiles capable of sensing and monitoring environmental changes or physiological signals.^5–10 Moreover, the utilization of SW-CNT-coated fabrics has been instrumental in the development of responsive and adaptive textiles, as demonstrated in studies by Wang et al.¹¹ and Iqbal et al.,¹² showcasing their potential in dynamic applications where adaptability and responsiveness are crucial.

Nevertheless, despite the significant strides made in leveraging fabric coated with nanomaterials for enhanced conductivity and multifunctionality, there remain persistent challenges in detecting defects within these coated fabrics, especially within densely populated particle areas. These areas, characterized by substantial conductivity discrepancies compared to the fabric’s general surface, directly impact its electrical properties. The identification of these critical regions is indispensable as they directly influence the accuracy of data collection in sensing and monitoring applications, depicted in Figure 1. This importance amplifies notably in domains like healthcare and wearable technology, where precise localization of such regions on conductive fabrics is imperative.^13–16 This precision is vital, especially in applications involving the monitoring of vital signs and the functionality of therapeutic garments.

Figure 1.

Some applications of smart garments in sensing and healthcare technology.

In addressing the intricacies of fabric defect identification, diverse methods are employed through image processing, aiming to distinguish defective and defect-free areas or the boundaries of defective regions. In a recent study, Lin et al.¹⁷ incorporated the gray-level co-occurrence matrix with redundant contourlet transform to extract texture features from segmented sub-images. These features were then employed to train a convolutional neural network for fabric defect detection. However, this method involved manual feature extraction, leading to time-consuming procedures and challenges in accurately pinpointing and extracting defects. In response to these challenges, attention has shifted toward deep learning methods, specifically generative models. In a study by Rui and Qiang,¹⁸ an enhanced GAN (Generative Adversarial Network) combining both GAN and AE (Autoencoder), was proposed to reconstruct fabric images and locate defective areas based on differences between the original and reconstructed images. While this approach achieved a high recognition rate, it still presented some drawbacks, including the use of the Mean Squared Error (MSE) loss function in the model architecture, leading to the vanishing gradient problem,^19–22 and hindering effective learning. Additionally, the manual definition of the threshold in the method proved time-consuming. Furthermore, this method employed defect-free data for model training, whereas for coated fabrics, defective patterns with highly dense particles are inevitable due to the manual nature of the coating process at the lab scale. This underscores the ongoing need for refined approaches to fabric defect detection in nanomaterial-coated fabrics.

Variational Autoencoders (VAEs) emerge as a promising alternative in navigating the intricacies of defect detection within nanomaterial-coated fabrics, such as SW-CNT. Extensively deployed across various studies, VAEs showcase their ability to detect fabric defects, even in real-time scenarios.^23–25 By encoding inputs into a latent space and reconstructing high-fidelity outputs and operating on unsupervised learning principles, they prove instrumental in precisely identifying defects within complexly structured, post-coated surfaces. This shift toward VAEs represents an advancement in the pursuit of efficient and accurate defect detection methodologies for coated fabric.

Therefore, in this study, the authors present a deep learning method using a VAE network to identify defects in fabric coated with SW-CNT. Initially collected via scanning without labels, the fabric images underwent resolution adjustment and division into equally sized patches. These patches were fed into the Variational Autoencoder model to derive the distribution of compressed input features via the encoder. Random sampling from this distribution facilitated image reconstruction through the decoder, aiming for defect-free representations of dominant features. To pinpoint defects in the test dataset, additional image processing like intensity threshold-based masking and morphological techniques were applied. The encoding-decoding process is detailed in the Materials and Methods Section. The evaluation of network performance through visualization of defective regions in the query images is presented in the Results Section, followed by a comprehensive discussion and conclusion in subsequent sections.

Materials and methods

Data preparation

In this investigation, the procedure for capturing images of coated fabric in its original state involves a two-step process: initially applying a coating to the fabric and subsequently photographing the coated samples. The selection of cotton fabric as the focal point for this research arises from its seamless integration of electronic components for smart textiles with comfort and wearability, offering a breathable, washable, and biocompatible solution for sustainable and user-friendly wearable technology.^26–29 The author utilized white cotton fabric, segmenting a large sheet into 48 samples measuring 70 by 130 mm each. These samples underwent a detailed coating process (depicted in Figure 2) where a coating of SW-CNT was applied. To confer conductivity, the 48 samples were immersed in 150 ml of commercial single-walled carbon nanotube (SW-CNT) liquid, featuring a SW-CNT-weight percentage of 0.1% with its diameter ranging between 1.1 and 1.4 nm (KORBON Co., Ltd, Korea) for precisely 10 min. Subsequently, water extraction occurred using a vertical padder machine, followed by a 10-min exposure to 150°C in a two-way drying machine. Figure 3 illustrates the sample both before and after the coating procedure.

Figure 2.

The fabrication process of the SWCNT-cotton samples.

Figure 3.

A cotton sample before (a) and after (b) the coating procedure.

For image acquisition in this study, each sample was scanned using a general scanner on one side, resulting in the generation of a 24-bit image per sample, depicted in Figure 3(b). Observing these images reveals that areas with a higher density of particles exhibit a darker color in contrast to the overall piece, indicating lower resistance. The variations in the electrical properties of this fabric can introduce inaccuracies in certain applications involving smart textiles. Consequently, the objective was to identify and extract these specific regions for further analysis.

To improve the contrast of these images and accentuate the darker regions, facilitating the VAE network in learning image features, the authors employed the Contrast Stretching method, which involves expanding the range of intensity values. Specifically, this process stretches the minimum and maximum intensity values present in the images to reach the potential minimum and maximum intensity values. The implementation of this enhancement technique follows equation (1) outlined below.

s = (r - r_{m i n}) \frac{I_{m a x} - I_{m i n}}{r_{m a x} - r_{m i n}} + I_{m i n}

(1)

In which, s is the adjusted pixel intensity value, r is the current pixel intensity value, $r_{m i n}$ and $r_{m a x}$ are the minimum and maximum intensity values present in the whole image, respectively. For a standard 8-bit grayscale image, $I_{m i n}$ is 0 and $I_{m a x}$ is 255. The images before and after applying the Contrast Stretching method are illustrated in Figure 4. As evident from this figure, the procedure enhances the contrast between the defective regions (the dark regions) and the rest of the image, thereby aiding the VAE network in learning the compressed features more effectively.

Figure 4.

The image of a cotton sample before (left) and after (right) applied the contrast stretching method.

Variational autoencoder – An overview

The Variational Autoencoder (VAE) stands as a prominent generative model for its capacity to comprehend the inherent probability distribution within a dataset and produce synthetic samples. Its architecture embodies an encoder-decoder structure: the encoder translates input data x into a latent space z, while the decoder reverses this process, translating the latent code back into reconstructed data. This dual-network system operates akin to an unsupervised autoencoder, creating a computational pipeline that facilitates the model’s functionality.

In our computational process, we employed the probabilistic encoding network $q_{ϕ} (z | x)$ as an approximation to the posterior distribution of latent variables, denoted as $p_{θ} (x | z)$ . The primary objective of a Variational Autoencoder (VAE) is to comprehend $p_{θ} (x | z)$ . This is achieved by utilizing an encoder network to approximate the true $p_{θ} (x | z)$ with the learned approximation $q_{ϕ} (z | x)$ . We assume a prior over latent variables as the centered isotropic multivariate Gaussian $p_{θ} (z) = ℕ (z; 0, 1)$ , and $p_{θ} (x | z)$ is modeled as a multivariate Gaussian distribution. The construction of the variational approximate posterior is detailed in equation (2).

q_{ϕ} (z | x^{(i)}) = N (z; μ^{(i)}, σ^{2 (i)})

(2)

As can be seen from this equation, this normal distribution has two sets of parameters: mean ( $μ^{(i)}$ ) and the square of the variance $σ^{2 (i)}$ ) respective to the i-th datapoint ( $x^{(i)}$ ). Considering the distribution difference between the input image and the reconstructed image, VAE defines a loss function as in the following equation (equation (3)).

\begin{array}{l} ℒ (θ, ϕ; x^{(i)}) = - D_{K L} (q_{ϕ} (z | x^{(i)}) | | p_{θ} (z)) \\ + E [\log (p_{θ} (x^{(i)} | z))] \end{array}

(3)

In which, $D_{K L}$ is the Kullback-Leibler divergence (KL-divergence) loss which measures the difference between the learned probability distribution – $q_{ϕ} (z | x^{(i)})$ and the predefined distribution - $p_{θ} (z)$ ; and $E [\log (p_{θ} (x^{(i)} | z))]$ is the reconstruction loss which measures the difference between the original input x and the output generated by the decoder $\hat{x}$ .

As shown in Figure 5, the $q_{ϕ} (z | x^{(i)})$ is the encoder and the directed probabilistic $p_{θ} (x^{(i)} | z)$ is the decoder. The latent variable z needs to be sampled from $q_{ϕ} (z | x^{(i)})$ , which has a normal distribution with the mean and variance calculated by the trained encoder. The overall architecture is then obtained by concatenating the encoder and decoder parts. To make the gradient descent possible, a simple trick called reparameterization is used. In which, VAE samples a standard normal distribution instead of a random variable from the normal distribution. The random variable $z ~ q_{ϕ} (z | x^{i})$ is re-parameterized by a deterministic transformation $h_{ϕ} (ε, x^{(i)})$ that $ε$ is from a standard normal distribution $N (0, 1)$ . The reparameterization trick is demonstrated in equation (4):

z = h_{ϕ} (ε, x^{(i)}) = μ^{(i)} + ε^{(l)} * σ^{(i)}, ε^{(l)} ~ N (0, 1)

(4)

where l represents the l-th noise $ε$ , $μ^{(i)}$ and $σ^{(i)} e$ are the outputs of the encoder respective to the datapoint $x^{(i)}$ .

Figure 5.

The architecture of a variational autoencoder network.

Training process

This section focuses on our Variational Autoencoder (VAE) model, tailored for pinpointing flaws within coated fabric surfaces. Our strategy integrates the VAE with a subsequent post-processing method specifically crafted to extract these regions. In the training phase, unlabeled images are segmented into patches and fed into the model to refine its parameters. During testing, the trained model employs the original image to extract compressed features, reconstructing an image based on these features. By comparing the original and reconstructed images, post-thresholding and morphological processing allow us to detect defects by analyzing the discrepancies between the two.

Encoder

The encoder incorporates a 2D convolutional layer to facilitate the learning of input features, specifically focusing on the probability distribution. The efficacy of the 2D convolutional layer in learning image features lies in its ability to capture hierarchical information, provide translation invariance, and enable efficient parameter sharing. This, in turn, allows for effective feature extraction and the establishment of a hierarchical representation within convolutional neural networks (CNNs).

The encoder network essentially constitutes a 2D CNN, where the input layer acts as a passive recipient of input data. The dimensions of this layer are $(640 \times 360 \times 1)$ , with $(640 \times 360)$ representing the input patch size and 1 denoting the number of channels of each patch. The entire encoder network comprises six blocks, each consisting of a 2D convolutional layer, an activation layer (employing the Rectified Linear Unit or ReLU), and a batch normalization layer. The initial 2D convolutional layer following the input patch adheres to the parameters outlined in Table 1. Within this layer, each filter executes a sequence of convolutions, and the cumulative result undergoes processing through the ReLU layer. The ReLU layer serves the purpose of determining the significance of input nodes, activating primarily the crucial nodes having values greater than 0 and assigning a value of 0 to inputs less than or equal to 0. Subsequently, the batch normalization layer is applied to standardize the output from the preceding layer. This normalization occurs across a batch of data, treating each channel independently to maintain mean output proximity to 0 and standard variation proximity to 1. Such normalization aids in expediting network training and mitigating sensitivity to network initialization. Following five analogous blocks, differing only in the parameters of the 2D convolutional layer, a fully connected layer, or Dense layer, is positioned at the network’s conclusion. This layer involves the multiplication of the input by a weight matrix, followed by the addition of a bias vector to the output. The resulting output has dimensions $((2 * l a t e n t d i m e n s i o n) * b a t c h s i z e)$ . This output is further separated into matrices of means and natural logarithm of the squared values of the variances, each with dimensions $(l a t e n t d i m e n s i o n * b a t c h s i z e)$ . These matrices are instrumental in implementing the reparameterization trick, facilitating the computation of the encoding.

Table 1.

The architecture of the proposed VAE network.

Layer name	Type of layers	Output dimensions $(h \times w \times c)$	Kernel size	Stride/Padding
input_encoder	Input	$640 \times 360 \times 1$
conv1 + relu1 + bn1	Convolution + ReLU + Batch Normalization	$320 \times 180 \times 8$	3	2/same
conv2 + relu2 + bn2	Convolution + ReLU + Batch Normalization	$160 \times 90 \times 16$	3	2/same
conv3 + relu3 + bn3	Convolution + ReLU + Batch Normalization	$80 \times 45 \times 16$	3	2/same
conv4 + relu4 + bn4	Convolution + ReLU + Batch Normalization	$40 \times 23 \times 32$	3	2/same
conv5 + relu5 + bn5	Convolution + ReLU + Batch Normalization	$20 \times 12 \times 64$	3	2/same
conv6 + relu6 + bn6	Convolution + ReLU + Batch Normalization	$10 \times 6 \times 128$	3	2/same
fc_encoder	Dense	1×1×(2*latent_dim¹)
transpose1 + relu1	Transpose Convolution + ReLU	$4 \times 3 \times 64$	7	(4, 3)/same
transpose2 + relu2	Transpose Convolution + ReLU	$16 \times 9 \times 64$	3	(4, 3)/same
transpose3 + relu3	Transpose Convolution + ReLU	$32 \times 18 \times 32$	3	2/same
transpose4 + relu4	Transpose Convolution + ReLU	$128 \times 72 \times 32$	3	4/same
transpose5 + relu5	Transpose Convolution + ReLU	$640 \times 360 \times 16$	3	5/same
transpose6	Transpose Convolution	$640 \times 360 \times 1$	3	1/same

Latent dimension.

Sampling function – Reparameterization trick

The reparameterization process in a Variational Autoencoder (VAE) is crucial for introducing stochasticity into encoding, enabling the generation of diverse samples, and facilitating end-to-end training through gradient descent. Using the mean and logarithm of the square of variance matrices from the encoder, the latent variable is sampled from a unit Gaussian distribution with mean 0 and standard deviation 1. To simplify computation during the reparameterization trick, the output matrices of the encoder consist of the mean and the natural logarithm of the square of the variance, rather than the mean and variance themselves. Hence, the sampled variable undergoes scaling and shifting per equation (4) with some modifications as indicated in equation (5).

z = h_{ϕ} (ε, x^{(i)}) = μ^{(i)} + ε^{(l)} \times e^{\frac{1}{2} \log {(σ^{2})}^{(i)}}

(5)

In which, $μ^{(i)}$ and ${(\log (σ^{2}))}^{(i)}$ are the i-th points in the two matrices outputted from the encoder, and $ε^{(l)}$ is the l-th point in the matrix of noise sampled from the standard normal distribution.

Decoder

In this phase, the decoder reconstructs data from acquired latent space representations, transforming the sampled latent variable z in equation (5) from encoded probabilistic representations into meaningful output samples. This process completes the generative cycle, yielding outputs that capture dominant input features and align with the original data distribution. The decoder model comprises six blocks, each housing a transposed 2D convolution layer and a ReLU activation layer. The transposed convolution, functioning as an up-sampling technique, is illustrated in Table 1 with input and output sizes. Similar to standard convolution layers, the transposed convolution layer is defined by padding and stride, with these parameters theoretically applied to the output for input generation. For the decoder network employed in this study, the padding parameter is set as “same,” determining the size of the generated output feature map (o) based on the input size (i) and the kernel size (k) by the following equation (equation (6)).

o = i \times k

(6)

Defining the loss function

Suppose that there is a hidden variable z that generates an observation x. The characteristics of z, p(z|x) can be inferred from x using equations (7) and (8).

p (z | x) = \frac{p (x | z) p (z)}{p (x)}

(7)

p (x) = \int p (x | z) p (z) d z

(8)

Because x is observed through z, the distribution p(x) becomes intractable. Hence, another distribution q(z|x), which is tractable, is employed to approximate p(z|x). By setting the parameters of q(z|x) to resemble those of p(z|x), it becomes possible to conduct approximate inference on the intractable distribution. This involves minimizing the KL-divergence between the two distributions, represented as M $in (K L ((q (z | x) | | p (z | x)))$ .

This can be done by maximizing equation (9).

E_{q (z | x)} [l o g p (x | z)] - K L ((q (z | x) | | p (z))

(9)

Therefore, the loss function for this VAE network consists of two terms and is defined in equation (10).

\begin{array}{l} the ℒ (θ, ϕ; x^{(i, j)}) = \frac{1}{2} {({\hat{x}}^{(I, j)} - x^{(I, j)})}^{2} - \\ \frac{1}{2} \sum_{j = 0} \sum_{i = 0} (1 + \log σ^{2 (I, j)} - μ^{2 (I, j)} - e^{\log σ^{2 (I, j)}}) \end{array}

(10)

Overall, the VAE network architecture is summarized in Table 1 and the algorithm for training the VAR is shown in Algorithm 1.

Algorithm 1. The training process of VAE.

Input: Training set: ${x^{(1)}, x^{(2)}, \dots, x^{(m}}$ of m training examples from the unlabeled scanning fabric dataset.

Output: Encoder $q_{ϕ}$ , Decoder $p_{θ}$ .

Step 1: Initialize the network parameters $ϕ$ and $θ$ with random values.

Step 2:

For i = 1 : num_epochs:

For i = j: num_batches:

1. Forward stage:

• Select k random samples from the training set.

• Input k samples into the encoder, get the mean value $μ$ and the natural logarithm of the square of variance $l o g (σ^{2})$ from its output.

• Reparametrize from the latent variable according to equation (5).

• Input the latent variable into the decoder, get the reconstructed data from its input.

2. Backward stage:

• Calculate the loss function according to equation (3).

• Update the network parameters $ϕ$ and $θ$ of the layer l-th:

{(θ, ϕ)}^{(l)} : = {(θ, ϕ)}^{(l)} - α [(\frac{1}{k} \nabla_{{(θ, ϕ)}^{l}} ℒ (θ, ϕ; x)) + λ {(θ, ϕ)}^{(l)}]

Where $α$ is the learning rate, $ℒ (θ, ϕ; x)$ is the loss function defined in equation (3), and $λ {(θ, ϕ)}^{(l)}$ is a term for regularization.

End j.

End i.

Step 3: Return the encoder $q_{\emptyset}$ , the decoder $p_{θ}$ .

Hyperparameters

As outlined in the Data Preparation section, the training dataset comprises 35 preprocessed images, from which 64 patches were randomly extracted per image, resulting in a total of 2240 patches. The validation set consists of seven preprocessed images, with 16 patches extracted from each image, totaling 112 patches. The entire network architecture was defined using MATLAB software (R2021b, Mathworks Inc., Natick, MA, USA) and fine-tuned during training. The training process utilized an initial learning rate of 1e−5, Adam optimizer, and a batch size of 64, equivalent to the number of patches extracted from each training image, conducted over 7000 iterations. Both the training stage and the inference process of the model, including post-processing, are depicted in Figure 6.

Figure 6.

The flow chart of the coated fabric defect detection.

Results and discussion

Morphology and electrical properties of the coated fabric

Morphology evaluation

The morphological characteristics of the cotton sample post-coating with SW-CNT are depicted in Figure 7. Figure 7(a) exhibits a scanning image of one side of the sample following the coating procedure. Notably, darker areas or spots are discernible on the sample’s surface, contrasting with the majority of the fabric’s surface. This variation in color arises from regions with higher nanomaterial density, as evidenced in Figure 7(b)–(e). Specifically, Figure 7(b) and (d) represent scanning electron microscopy (SEM) images of the lighter-colored area, indicative of the overall sample color. Conversely, Figure 7(c) and (e) portray SEM images of the darker-colored regions.

Figure 7.

(a) The scanning image of a SW-CNT-coated cotton sample and (b–e) the morphology of two regions in this sample.

Electrical property evaluation

In this section, we assessed the electrical properties of various dark regions within a sample by measuring their sheet resistance. Sheet resistance, a common electrical property of conductive fabric, is typically measured using a 4-probe method.³⁰ Initially, visual observations were made to identify areas with darker coloration compared to the fabric’s general appearance. These areas are delineated by the red boundaries in Figure 8(a). Subsequently, sheet resistance measurements were conducted on these delineated regions. The acquired results confirm the theory that between regions of the same area, darker regions have lower sheet resistance, establishing a correlation between sheet resistance and the darkness levels of the coated fabric. However, whether a dark region is considered defective depends on the application of the coated fabric. In this research, we focus on utilizing coated fabric for creating textile electrodes, crucial components in wearable technology for health monitoring, fitness tracking, and medical diagnostics. Among the functional and non-functional requirements for textile electrodes, electrical safety, and signal quality are paramount. Specifically, textile electrodes must meet safety standards to mitigate risks such as electrical shock or thermal injury, while also ensuring stable and reliable signal acquisition under various conditions, including moisture, sweat, and motion artifacts. Given the absence of an established standard for the optimal sheet resistance disparity to ensure fabric functionality in this application, we utilized electrical safety and signal quality as criteria to define the darkness level threshold for labeling regions as defective. A preliminary experiment was conducted to determine this threshold using fabric from Figure 8(a). Eight circular samples, each with a diameter of 20 mm, were excised, as depicted in Figure 8(b). Among these, four circles (green circles) were positioned outside the marked areas in Figure 8(a) (or from a sample with consistent color, as observed in other fabric samples), while the remaining four circles (yellow circles) encompassed progressively darker regions marked in Figure 8(a), ranging from the least (circle 1) to the most intense darkness (circle 4) within each circle. Care was taken to ensure that the area of the dark regions within each yellow circle was less than 10% of the whole circle area. This precaution was necessary because if the areas of high particle density are too large compared to the entire area of the electrode, the electrodes may not provide an effective muscle/nerve activation and even cause skin irritation or pain.³¹ Each pair of green and yellow circles served as electrodes. These four pairs of circular samples were then designated to replace a pair of adhesive electrodes during electromyographic signal acquisition, following the methodology outlined in Chun et al.¹⁵ This experimental setup aimed to evaluate the performance of textile electrodes in direct contact with the skin and their impact on signal acquisition. Post-experimentation analysis indicated that electrode pair (1) met the operational requirements for this application. Based on the experimental findings, it was concluded that the regions identified as dark in Figure 8(a) exhibited a significant sheet resistance difference compared to the entire fabric, exceeding 45%, thus qualifying them as defective regions, as shown in Figure 8(c). This determination underscores the importance of identifying and characterizing such regions to ensure the efficacy and reliability of smart textile applications. Assuming that conductive fabric with similar resistance behaves similarly, we applied this threshold to the entire test dataset to establish the ground truth for further evaluation.

Figure 8.

(a) Dark regions in a test image were defined by visual observations, (b) cutting position of circular electrodes for EMG signal acquisition experiments and sheet resistance (Ω/square) of dark regions within the electrode area, and (c) ground truth defined after evaluating the experiment’s results.

Evaluation the model with test images

Figure 9 shows some images in the test set. All images were transformed from RGB to grayscale and processed with the Contrast Stretching method. For each sample, the left image (a) indicates the preprocessed image, while the ground truths defined by the method presented in the previous section are illustrated in the right image (b).

Figure 9.

(a) Test images and (b) their corresponding ground truth.

To extract the defective regions, we first generated difference maps for each image by adding the test sample to its reconstructed image rather than subtracting them as in some previously proposed methods.^18,23,32,33 This adjustment was essential because the reconstructed image encompasses features from the non-defective regions present in the training image. Notably, the dark regions observed in the training images correspond to defects resulting from the manual coating process. These regions lack specific shapes and occupy arbitrary-size areas within each sample. Consequently, although the training dataset comprises both non-defective and defective samples, the model primarily learns the dominant features of the images, particularly those of the non-defective regions, thanks to the convolution layers. As a result, the reconstructed images predominantly contain these features. Given the grayish color of the coated fabric, subtracting the two images would result in a darker image, making it challenging to locate the defective regions, which are expected to appear blackish. This effect is evident in the histograms of the test image, its reconstructed image, and the sum image of these two in Figure 10. Therefore, by adding the test sample and its reconstructed image, the goal is to increase the brightness of the resulting image. Consequently, the pixel values in the sum images tend to be closer to 1 (after normalization), representing the non-defective area, which appear very bright due to the high pixel values, whereas the defective regions exhibit a darker color. This approach facilitates the definition of a threshold for detecting dark (defective) areas, thereby simplifying the extraction of defective areas.

Figure 10.

Histogram of the test image, reconstructed image, and sum image, respectively from left to right for each sample.

Figure 11 illustrates the binarization outcomes obtained from comparing the difference map to the original test sample. Following a sequence of procedures including intensity threshold determination and morphological processing, the defective regions were identified and masked. This binarization map was generated by eliminating all defective regions, as defined by the threshold, that encompass fewer than 70 pixels from the difference map. The connectivity of these pixels was configured as a three-dimensional and 18-connected neighborhood. The thresholds for generating the binarization map on the sum image were established in the HSV (Hue, Saturation, Value) color space. This choice was made over the RGB space due to its ability to separate color information from brightness information, facilitating the manipulation of images with indistinct color variations. Specifically, in this study, the dataset comprises grayish images with variations primarily in brightness rather than color. Thus, utilizing the value component in HSV space enables more efficient thresholding to capture these brightness variations.

Figure 11.

Some images in the test set with ground truth (left) and their corresponding binarization map (right).

The results demonstrate the proposed method’s ability to accurately localize defective regions compared to the ground truth, achieving a recognition rate of up to 93.2%. Figure 12 illustrates the close correspondence between the masked regions in the difference map and the actual defective regions.

Figure 12.

Some images in the test set with ground truth (left) and their corresponding masked region map (right). The blue-contoured regions have the darkest color, the red ones have dark and light patches, and the yellow ones have the lightest color.

Notably, in regions characterized by high particle density or darker colors, the masked regions exhibit completeness, encompassing the entire true regions delineated by the blue boundaries. However, upon examining the masked region map of the test dataset, it became evident that while the darkness level of defective regions correlates with the density of the masked regions, this relationship is not consistent across all defective regions. Some regions, marked in the figure within the red boundaries, exhibit variations in darkness within a single region. Additionally, these defective regions with both dark and light patches often result in masked regions surpassing the ground truth, indicative of over-segmentation and under-segmentation. Conversely, for lighter-colored regions (areas within the yellow boundaries), the masked regions may not fully cover the entire ground truth. Furthermore, the masked regions fail to capture variations in particle density or differences within the same defective region, primarily evident at the region edges. In some instances, the method struggles to separate adjacent regions with similar darker colors. This limitation underscores the challenges inherent in threshold-based segmentation tasks. The intersection over union (IoU) metric is employed to assess the degree of overlap between the masked regions and manually annotated regions, yielding a result of 0.923 for the blue-contoured regions, 0.795 for the red ones, and 0.563 for the yellow ones. Despite the relatively limited number of training images, the proposed method achieves a high recognition rate and an average IoU of 0.76 for the entire test dataset.

Conclusion

In conclusion, our study introduces an innovative unsupervised deep learning approach tailored for detecting defective regions on cotton fabric coated with SW-CNT, a material vital in smart textile applications. These regions, characterized by high particle density, pose distinct challenges in detection against the fabric’s background. Our proposed methodology combines the power of Variational Autoencoder, a generative model, with image post-processing techniques to precisely extract these defective regions. The results demonstrate the efficacy of the VAE model in learning compressed features from training images, even in the lack of defect-free samples. This methodology offers a promising solution for defect detection in coated fabrics, paving the way for advancements in smart textiles technology. Looking ahead, our future research endeavors will prioritize enhancing the speed and accuracy of image processing techniques for defect extraction. Additionally, we aim to extend the applicability of this deep learning model to encompass other non-conductive materials and nanomaterials, broadening its utility in diverse domains beyond the scope of this study.

Footnotes

Authors’ Note

Kim Jooyong also affiliated to Department of Materials Science and Engineering, Soongsil University, Seoul, Korea

Author contributions

As a corresponding author, Jooyong Kim was responsible for the whole structure construction, while Sehan Chun was responsible for sensor fabrication, experiment design, and data acquisition, Ngoc Tram Nguyen was responsible for experiment design, data processing, deep learning model design, and helping with experiments. The manuscript was drafted by Jooyong Kim and Ngoc Tram Nguyen. Finally, all authors read and approved the final manuscript.

Availability of data and material

Not applicable.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was partly funded by the Technology Innovation Program (or Industrial Strategic Technology Development Program-Materials/Parts Package Type) (20016038, development of textile-IT converged digital sensor modules for smart wear to monitor bio & activity signal in exercise, and KS standard) funded By the Ministry of Trade, Industry & Energy (MOTIE, Korea) and Korea Institute for Advancement of Technology (KIAT) grant funded by the Korea Government (MOTIE) (P0012770).

ORCID iD

Kim Jooyong

References

Guan

Wang

, et al. Personal thermal management by single-walled carbon nanotubes functionalized polyester fabrics. Materials 2021; 14(16): 4616.

Syduzzaman

Hassan

Anik

, et al. Nanotechnology for high-performance textiles: a promising frontier for innovation. ChemNanoMat 2023; 9(9): e202300205.

Zhang

Zheng

Mao

, et al. Effects of structural parameters on the thermal insulation properties of coated carbon fiber fabrics. Text Res J 2020; 90(13–14): 1549–1557.

Shahidi

Moazzenchi

Carbon nanotube and its applications in textile industry – a review. J Text Inst 2018; 109(12): 1653–1666.

Sajovic

Kert

Boh Podgornik

Smart textiles: a review and bibliometric mapping. Appl Sci 2023; 13(18): 10489.

Barman

Tirkey

Batra

, et al. The role of nanotechnology based wearable electronic textiles in biomedical and healthcare applications. Mater Today Commun 2022; 32: 104055.

Angelucci

Cavicchioli

Cintorrino

, et al. Smart textiles and sensorized garments for physiological monitoring: a review of available solutions and techniques. Sensors (Basel) 2021; 21(3): 814.

Zhou

Ding

Zhang

, et al. Fabrication of conductive fabric as textile electrode for ECG monitoring. Fibers Polym 2014; 15(11): 2260–2264.

Zhou

Zhang

Yang

, et al. Fabric electrode monitoring of dynamic and static ECG signal and comfort performance. Coatings 2023; 13(2): 289.

10.

Maji

Dookhy

Burke

MJ.

Electrical characterization of conductive textile based ECG recording electrodes. Measurement 2021; 15: 100044.

11.

Wang

Zhang

Textile-based strain sensor for human motion detection. Energy Environ Mater 2020; 3(1): 80–100.

12.

Iqbal

Sun

Fei

, et al. Knit architecture for water-actuating woolen knitwear and its personalized thermal management. ACS Appl Mater Interfaces 2021; 13(5): 6298–6308.

13.

Di Tocco

Raiano

Sabbadini

, et al. A wearable system with embedded conductive textiles and an IMU for unobtrusive cardio-respiratory monitoring. Sensors 2021; 21(9): 3018.

14.

Shao

Cui

, et al. Carbon-based textile sensors for physiological-signal monitoring. Materials 2023; 16(11): 3932.

15.

Chun

Kim

Human arm workout classification by arm sleeve device based on machine learning algorithms. Sensors 2023; 23(6): 3106.

16.

Kim

Human motion recognition by textile sensors based on machine learning algorithms. Sensors 2018; 18(9): 3109.

17.

Lin

Wang

Zhu

, et al. Fabric defect detection based on multi-input neural network. In: 2021 27th international conference on mechatronics and machine vision in practice (M2VIP), 2021, pp. 58–463.

18.

Rui

Qiang

Research on textile defects detection based on improved generative adversarial network. J Eng Fibers Fabr 2022; 17: 15589250221101382.

19.

Zahra

Essai Ali

Refaee

Performance functions alternatives of MSE for neural networks learning. Int J Eng Res Technol 2014; 3: 967.

20.

Karras

Laine

Aila

A style-based generator architecture for generative adversarial networks, http://arxiv.org/abs/1812.04948 (2019, accessed 25 January 2024).

21.

Larsen

Sønderby

Larochelle

, et al. Autoencoding beyond pixels using a learned similarity metric. In: Proceedings of the 33rd international conference on machine learning. PMLR, 2016, pp. 1558–1566.

22.

Ledig

Theis

Huszár

, et al. Photo-realistic single image super-resolution using a generative adversarial network. In: 2017 IEEE conference on computer vision and pattern recognition (CVPR), Honolulu, HI, USA, 2017, pp.105–114.

23.

Wei

Deng

Zeng

, et al. Real-time implementation of fabric defect detection based on variational automatic encoder with structure similarity. J Real Time Image Process 2021; 18(3): 807–823.

24.

Prifti

Buban

Thind

, et al. Variational convolutional autoencoders for anomaly detection in scanning transmission electron microscopy. Small 2023; 19(16): 2205977.

25.

Yang

Mao

Wang

, et al. Patch variational autoencoder-based industrial defect detection. In: 2022 13th Asian control conference (ASCC), 2022, pp.674–677.

26.

Bajgar

Penhaker

Martinková

, et al. Cotton fabric coated with conducting polymers and its application in monitoring of carnivorous plant response. Sensors 2016; 16(4): 498.

27.

Yang

Pan

, et al. Conductive cotton fabrics for motion sensing and heating applications. Polymers 2018; 10(6): 568.

28.

Rayhan

MGS

Khan

MKH

Shoily

, et al. Conductive textiles for signal sensing and technical applications. Signals 2022; 4(1): 1–39.

29.

Keawploy

Venkatkarthick

Wangyao

, et al. Eco-friendly conductive cotton-based textile electrodes using silver- and carbon-coated fabrics for advanced flexible supercapacitors. Energy Fuels 2020; 34(7): 8977–8986.

30.

Jee

MH.

A study on changes of sheet resistance in conductive knit fabrics using non-contact measurement method. Fibers Polym 2023; 24(5): 1873–1878.

31.

Euler

Guo

Persson

N-K.

A review of textile-based electrodes developed for electrostimulation. Text Res J 2022; 92(7–8): 1300–1320.

32.

Zhu

, et al. Unsupervised defect detection algorithm for printed fabrics using content-based image retrieval techniques. Text Res J 2021; 91(21–22): 2551–2566.

33.

Biradar

Shiparamatti

Patil

PM.

Fabric defect detection using deep convolutional neural network. Opt Mem Neural Netw 2021; 30(3): 250–256.