A novel objective fabric pilling evaluation method based on the photometric stereo algorithm

Abstract

The pilling grade is one of the important indexes to evaluate the quality of fabric; the traditional method for pilling evaluation is based on manual operation, which is subjective. In this paper, a self-developed system is established and a photometric stereo-based three-dimensional (3D) acquisition method for pilling evaluation has been demonstrated. Firstly, a 3D surface model is obtained by using photometric stereo, mapping a 3D model to a two-dimensional image; the gray value is used to represent the depth value of the fabric surface and the pilling segmentation is realized by the Otsu threshold method after filtering the fabric texture by the relative total variance algorithm. The pilling number and pilling area are used as eigenvectors for pilling classification using the support vector machine. Finally, the classification accuracy of the pilling grade was 95.9%. For pilling evaluation, the developed system and method can be considered reliable and effective according to the experimental results.

Keywords

Pilling evaluation photometric stereo three-dimensional reconstruction light reflex near-light model

Rubbing fabric during use causes loose fibers to form small pills that are anchored to the fabric surface. In addition, while destroying the fabric appearance, the pilling will also affect the mechanical properties of the fabric. Therefore, pilling evaluation is an important part of fabric quality control in the textile industry.

At present, the methods of pilling evaluation using image processing technology can be divided into two-dimensional (2D)-based analysis methods and three-dimensional (3D)-based analysis methods. In the research of fabric pilling evaluation, many scholars extract the fabric pilling in the spatial domain and frequency domain to achieve pilling evaluation.

Spatial domain-based methods for evaluating pilling began as early as the 1990 s. In 1990, Konda et al.,¹ with a nearly parallel light source from the side of a tangential irradiation fabric for gray images, the maximum ratio of inter-class variance and intra-class variance is used as the segmentation threshold for fabric hairball and texture, and then the pill area and the pill number were extracted as two characteristic parameters for the grade evaluation. Hu and Xin² concluded that the fabric pills in the image are brighter compared to the fabric texture background, so they proposed using Gaussian template matching to locate the pilling, but how to select the weight coefficients has a more significant effect on segmenting the fabric hairball. Sekulska-Nalewajko et al.³ used optical coherence tomography (OCT) to identify the pilling layer to complete hairball segmentation, and then principal component analysis was applied to select strongly correlated features, and the experimental results showed that the method was reliable.

Threshold segmentation using color information in the spatial domain can be affected by noise such as the fabric texture, leading to the inappropriate selection of threshold values, making pilling segmentation inaccurate. To overcome the influence of noise, the noise can be filtered in the frequency domain by using the difference frequency information between the noise and pilling; Fourier transform and Wavelet transform are used most often.

To filter the fabric texture, Xu⁴ used spectral analysis, assumed that most of the pills are circular, designed a circular stencil matching method to locate the pills, and selected a global threshold for pilling segmentation. In a study published by Zhang,⁵ wavelet analysis was suggested as a method for extracting pills from textures, using wavelet coefficients to construct the characteristic parameters of the complex texture, and conducting grade evaluation of pills according to their density. Deng et al.⁶ suggested that pilling images had several frequency components, such as noise, fabric texture, and pilling information. A pilling evaluation method based on a multi-scale 2D bi-plural wavelet was proposed to extract pilling information. Jing et al.⁷ proposed an integrated feature extraction technique combining 2D discrete wavelet transform (2DDWT) and local binary patterns (LBPs) for puckered fabric image recognition. Xiao et al.⁸ used Fourier transform to convert images into the frequency domain, combined with the energy algorithm, multidimensional discrete wavelet transform, and iterative thresholding method, to obtain the starting image. Finally, deep learning is used to complete the grade classification.

With the advancement of image processing technology, some researchers have tried to extract surface features of fabric pilling by 3D reconstruction to improve the accuracy of quality evaluation. With a 3D non-contact scanning system, Kim and Kang⁹ obtained highly accurate 3D surface data to describe the surface firmness of the fabric; wavelet and fractal algorithms were used to calculate the fractal dimension and the standard deviation of average curvature. Finally, analysis of fabric pilling using Bayesian classifiers, minimum distance classifiers, k-nearest neighbor classifiers, and neural networks was carried out. Ouyang et al.¹⁰ used the depth information of the fabric surface to detect pilling and extracted the fabric pilling density and other parameters. Xu et al.¹¹ investigated a system for 3D fabric surface reconstruction by using only two side-by-side images, where the pilling fabric was captured by a pair of regular digital cameras. Liu et al.¹² used self-developed stereo vision algorithms to complete the 3D reconstruction of fabric surfaces, including the structure from motion (SFM) and patch-based multi-view stereo (PMVS) algorithms. The experimental results show that the method is effective for fabric pilling evaluation.

The analysis methods based on 2D images are not universal due to factors such as lighting and texture. The appearance of the fabric pilling is 3D, and spatial information will be lost when using 2D images for analysis. In 2D imaging, complex textures and colors of fabric surfaces have a noticeable impact on pilling evaluations. However, analysis methods based on 3D reconstruction also have drawbacks: multi-view stereo cannot accurately recover the 3D model of the fabric tissues on the surface with no obvious texture details, but rather can only recover the macro contour. Many conditions must be met for laser triangulation, the scanning process takes a long time, and the equipment is expensive. Structure grating measurement will fail if applied to fabrics with complex surfaces or disordered colors because the grating coding pattern is destroyed. Most other image-based 3D reconstruction methods, such as stereo matching, are based on the features of the surface for 3D reconstruction, which means that when the surface texture is fuzzy, they will not be able to get an acceptable 3D model after reconstruction.

In this paper, a pilling evaluation method based on 3D reconstruction is proposed, as shown in Figure 1. Compared with the previous pilling evaluation based on 3D analysis method, our contributions mainly include the following:

1. The proposed image acquisition system consists of three or more light-emitting diodes (LEDs) and a camera, which is low-cost.

2. For traditional photometric stereo, a calibration of the light source intensity and direction is required, as well as the acquisition under parallel light. Parallel light requirements are challenging to meet, and the calibration procedures are time-consuming. Therefore, a method based on semi-calibrated near-light photometric stereo is proposed, which not only simplifies the calibration procedures but also relaxes the restrictions on light conditions, which allows using low-cost lighting devices such as LEDs.

3. Mapping the 3D model to a 2D image makes subsequent analysis more efficient and straightforward.

Methodology

Image acquisition system

The image acquisition system was developed based on the 3D fabric surface reconstruction method with a near-light photometric stereo vision algorithm. Multi-light source acquisition with different positions is required to acquire fabric images. Therefore, a multi-light source image acquisition device was developed, as shown in Figures 2 and 3. Eight LED lamps with 1 W power were used to illuminate the samples, which were calibrated and concentrated with a range of 15 luminous angles. A Nikon D7200 camera with an AF-S Micro NIKKOR macro lens was used in this study, which can obtain high-resolution microscopic images of the fabric surface. To minimize 3D reconstruction error due to light reflection, the device interior was painted with black matte paint.

Figure 1.
Pilling grade evaluation system. 3D: three-dimensional; 2D: two-dimensional; SVM: support vector machine.

Figure 2.
Self-assembly photometric stereo system. LED: light-emitting diode.

Figure 3.
Experimental facility. LED: light-emitting diode.

Different colors and textures of fabrics were selected so that the data would be representative. In accordance with ASTM test method D3512; Martindale pilling testers and random roller pilling machines were used to make pilling samples, then the samples were graded by experienced experts.¹³

System calibration

Camera calibration

The intrinsic matrix of the camera is indispensable information for 3D reconstruction. We obtain the internal matrix of the camera and establish the camera imaging model by Zhang’s plane-based method.¹⁴ The single-point distortion-free camera imaging model is as follows
$Z (\begin{matrix} u \\ v \\ 1 \end{matrix}) = M_{i n} M_{e x} (\begin{matrix} U \\ V \\ W \\ 1 \end{matrix})$
(1)
where $(U, V, W)$ is the physical coordinate of a point in the world coordinate system, $(u, v)$ is the corresponding pixel coordinate of the point in the pixel coordinate system, $Z$ is the scale factor, $M_{i n}$ is the camera intrinsic matrix, and $M_{e x}$ is the camera extrinsic matrix.

The intrinsic matrix can be expressed as
$K = M_{i n} = (\begin{matrix} \frac{f}{d X} & - \frac{f \cot θ}{d X} & u_{0} & 0 \\ 0 & \frac{f}{d Y \sin θ} & v_{0} & 0 \\ 0 & 0 & 1 & 0 \end{matrix})$
(2)
where $f$ is the focal length, $d X$ and $d Y$ respectively represent the physical length of a pixel in the x and y directions on the camera sensor, $u_{0}$ and $v_{0}$ indicate the coordinates of the camera sensor center in the pixel coordinate system, and $θ$ represents the angle between the horizontal and vertical edges of the photographic plate.

The images of the checkerboard were acquired, and the pixel coordinates of each corner point were obtained using the corresponding image detection algorithm, as shown in Figure 4. The size of each grid on the calibration board was known, and we could calculate the physical coordinates of each corner point under the world coordinate system, obtaining the intrinsic parameters of the camera through the constraint relationship between the pixel coordinates of each corner point and its world coordinates. The average reprojection error of this calibration was 0.3 pixels, which indicates a high calibration accuracy, and the reprojection error is obtained by calculating the distance between the original and reprojected position of the point.

Figure 4.
The 12 checkerboard positions used, marked with the detected points, checkerboard origin, and reprojected points.

Light calibration

In this paper, triangulation was used to estimate the light source position parameters.¹⁵ A pair of metal spheres are placed in the scene to generate visible highlights for each light source and we extract multiple highlight points by the thresholding method, as shown in Figure 5. Then the sphere contour was detected by the Canny operator,¹⁶ as shown in Figure 6. Finally, the light source position parameters were obtained by triangulation. The principle is shown in Figure 7. The incident rays ${\vec{L}}_{1}$ and ${\vec{L}}_{2}$ produce highlights $g_{1}$ and $g_{2}$ by specular reflection on the metal spheres, where ${\vec{R}}_{1}$ and ${\vec{R}}_{2}$ are the reflected rays, ${\vec{d}}_{1}$ and ${\vec{d}}_{2}$ are the normal vectors of the specular points on the spheres, $r$ is the radius of the spheres, $c_{1}$ and $c_{2}$ are the center points of the spheres, $s$ is the light source position, and $o$ is the error in calculating the position of the light source.

Figure 5.
Detect metal sphere highlights.

Figure 6.
Test the center and radius of the metal spheres.

Figure 7.
Principle of mirror sphere calibration.

The actual radius of the sphere, the focal length of the camera, and the physical pixel size of the camera sensor are needed to calculate the position of the light source using this method.

Firstly, the world coordinates of the highlight point $g (X, Y, Z)$ can be obtained by calculating the center position of the sphere
$g (X, Y, Z) = [\frac{- x}{f_{x}} \frac{f * r}{\ddot{f} * \dot{r}}, \frac{- y}{f_{y}} \frac{f * r}{\ddot{f} * \dot{r}}, \frac{f * r}{\ddot{f} * \dot{r}}] + r * \vec{d}$
(3)
where $Z$ is the distance of the sphere center to the camera in the z direction, $f_{x}$ and $f_{y}$ are the focal length in pixels in the x and y directions, respectively, $f$ is the focal length, $\ddot{f}$ is pixel length, $r$ is the actual radius of the sphere, $\dot{r}$ is pixel radius of the sphere, and $\vec{d}$ is the unit normal of highlight point $g_{1}$ or $g_{2}$
$\vec{d} = (g_{x} - c_{x}, g_{y} - c_{y}, \sqrt{（r^{2} - d_{x}^{2} - d_{y}^{2}）})$
(4)
where ( $c_{x}, c_{y}$ ) and ( $g_{x}, g_{y}$ ) are the pixel coordinates of the sphere center and the highlight on the sphere, respectively.

To calculate the direction of the light source with the coordinates of the obtained highlight point
$\vec{L} = 2 (\vec{d} \cdot \vec{R}) \vec{d} - \vec{R}$
(5)
where (0,0,1) is the reflection direction $\vec{R}$ after the light source irradiates the metal sphere.

After obtaining the direction of the light source, calculate the position of the light source
$p_{1} = g_{1} + \frac{({\vec{L}}_{2} \times (g_{1} - g_{2}) \cdot ({\vec{L}}_{1} \times {\vec{L}}_{2}))}{({\vec{L}}_{1} \times {\vec{L}}_{2}) \cdot ({\vec{L}}_{1} \times {\vec{L}}_{2})} * {\vec{L}}_{1}$
(6)

$p_{2} = g_{2} + \frac{({\vec{L}}_{1} \times (g_{1} - g_{2}) \cdot （{\vec{L}}_{1} \times {\vec{L}}_{2}）))}{({\vec{L}}_{1} \times {\vec{L}}_{2}) \cdot ({\vec{L}}_{1} \times {\vec{L}}_{2})} * {\vec{L}}_{2}$
(7)

$s = \frac{p_{1} + p_{2}}{2}$
(8)
where $p_{1}$ is the point nearest to ${\vec{L}}_{2}$ in ${\vec{L}}_{1}$ and $p_{2}$ is the point nearest to ${\vec{L}}_{1}$ in ${\vec{L}}_{2}$ . If the difference between $p_{1}$ and $p_{2}$ is too large, it is necessary to adjust the position of the mirror sphere to calibrate again.

Three-dimensional reconstruction of fabric pilling

Semi-calibrated near-source photometric stereo model

Fabric appearance evaluation requires the close acquisition of fabric images. Traditional photometric stereo assumes the use of parallel light sources, which is difficult to guarantee. To solve this problem, a method based on photometric stereo¹⁷ is proposed.

Suppose that $M$ images are acquired, the gray value of pixel $i$ with pixel coordinates (x, y) is related to a surface point $s ϵ R^{3}$ is given by the following equation
$p^{k} (i) = ρ (s) e^{k} m ax {l^{k} (s) \cdot n (s), 0}, k \in [1, M]$
(9)
where $ρ (s)$ is the albedo of the surface point $s$ , $k$ is the $k$ th light source, $e^{k}$ represents the light source intensity, $l^{k} (s)$ is the irradiated light vector of the surface points $s$ , and $n (s)$ is the normal vector of the surface point $s$ .

The point source model needs to calculate the attenuation due to distance. Vector $l^{k} (s)$ in Equation (9) is written as follows
$l^{k} (s) = \frac{[x_{l}^{k} - s]}{‖s - x_{l}^{k}‖} \frac{1}{{‖s - x_{l}^{k}‖}^{2}}$
(10)

The second factor represents attenuation due to distance, and $x_{l}^{k}$ is the light source position.

The conjugation relationship between $s$ and $i = (x, y)$ is shown in Equation (11), which unifies the independent variables of Equation (9)
$s (x, y) = z (x, y) K^{- 1} {[x, y, 1]}^{T}$
(11)
where $z (x, y)$ is the depth value obtained from the 3D reconstruction and $K$ is the intrinsic matrix of the camera.

The expression for the normal vector $n (s)$ in Equation (9) is obtained after some algebra
$n (s) = \frac{{[\begin{matrix} \frac{f}{d X} & - \frac{f \cot θ}{d X} & - (x - u_{0}) \\ 0 & \frac{f}{d Y \sin θ} & - (y - v_{0}) \\ 0 & 0 & 1 \end{matrix}]}^{T} [\begin{matrix} \partial_{x} l og z (i) \\ \partial_{y} l og z (i) \\ - 1 \end{matrix}]}{‖{[\begin{matrix} \frac{f}{d X} & - \frac{f \cot θ}{d X} & - (x - u_{0}) \\ 0 & \frac{f}{d Y \sin θ} & - (y - v_{0}) \\ 0 & 0 & 1 \end{matrix}]}^{T} [\begin{matrix} \partial_{x} l og z (i) \\ \partial_{y} l og z (i) \\ - 1 \end{matrix}]‖} = \frac{{J (i)}^{T} [\begin{matrix} \nabla \log z (i) \\ - 1 \end{matrix}]}{d (i; z (i))}$
(12)

Discrete model optimization

The albedo expression was modified following the method of Quéau et al.,¹⁸ which eliminates the nonlinearity caused by the denominator
$\ddot{ρ} (i) = \frac{ρ (i)}{d (i; z (i))}$
(13)

The expression of Equation (9) can be transformed into a system of nonlinear partial differential equations (PDEs)
$p^{k} (i) = \ddot{ρ} (i) e^{k} m ax {[{J (i) l^{k} (i; \log z (i))]}^{T} [\begin{matrix} \nabla \log z (i) \\ - 1 \end{matrix}], 0}, k \in [1, M]$
(14)

We denote $j = 1 \dots N$ for the corresponding pixel indices. The discrete counterpart of Equation (14) is written as
$p_{j}^{k} = {\ddot{ρ}}_{j} e^{k} m ax {[{J_{j} l^{k} ({\log z}_{j})]}^{T} [\begin{matrix} \nabla \log z_{j} \\ - 1 \end{matrix}], 0}, \forall k \in [1, M], \forall j \in [1, N]$
(15)

We jointly estimate the albedo value $ρ$ , depth value $\log z$ , and intensity $e$ using Cauchy’s estimator, assuming that Q is the set of all rank-1 N-by-M matrices, and optimize the discrete part
$\min_{\log z, θ : θϵQ} F (θ, \log z) = \sum_{j = 1}^{N} \sum_{k = 1}^{M} λ^{2} l og (1 + \frac{{(θ_{j}^{k} m ax {[{J_{j} l^{k} ({\log z}_{j})]}^{T} [\begin{matrix} \nabla \log z_{j} \\ - 1 \end{matrix}], 0\} - p_{j}^{k})}^{2}}{λ^{2}})$
(16)
where $θ_{j}^{k} \equiv {\ddot{ρ}}_{j} e^{k}$ , the nonconvex model of Equation (16) is minimized alternatively over the variables θ and $\log z$ . In each subproblem, we solve the local quadratic model of Equation (16) using the positive definite approximation of Hessian to achieve estimates of the albedo value $ρ$ , depth value $z$ , and intensity $e$ , and the normal vector is obtained by Equation (12). Refer to Figure 8 for the 3D reconstruction results.

Figure 8.
Example of three-dimensional reconstruction for a pilling sample. (a) Pilling sample; (b) albedo map; (c) normal map and (d) depth map.

Pilling binary graph acquisition

Two-dimensional depth image generation

Before further processing of the image, the 3D model needs to be transformed to a 2D grayscale image, which makes segmenting the pilling more efficient.¹⁹ The abscissa and ordinate coordinates of the point $s = (x, y, z (i))$ in the depth image are converted to the pixel coordinates of the grayscale image. Equation (17) converts the depth value $z (i)$ into the grayscale value for the corresponding pixel
$G (i) = 255 \frac{z (i) - z_{m}}{H}$
(17)
where $G (i)$ is the grayscale value of the 2D depth image, $z_{m}$ is the minimum depth value of the 3D model, and H is the range of the depth value.

Comparing the grayscale image with the 2D depth grayscale image indicates that the 2D depth grayscale image filters the interference information of the fabric texture and illumination, as shown in Figure 9.

Figure 9.
(a) Grayscale images of pilling and (b) Two-dimensional depth grayscale image of pilling.

Pilling segmentation

The information about pilling in the image are characteristics to measure the pilling level, but the recognition of pilling is affected by the texture of the fabric. To filter the fabric texture, a structure extraction method based on relative total variance (RTV)²⁰ is proposed.

This method uses a model based on the total variation, as shown in Equation (18)
$\arg \min_{h} \sum_{i} \{\frac{1}{2 λ} {(h_{i} - d_{i})}^{2} + |{(\nabla h)}_{i}|\}$
(18)
where $d$ is the input image, $i$ is the index 2D pixels, and $h$ is the resulting structure image. The second factor represents the total variation regularizer, expressed as
$\sum_{i} |{(\nabla h)}_{i}| = \sum |{(\partial_{x} h)}_{i}| + |{(\partial_{y} h)}_{i}|$
(19)

To solve the above model, Equation (18) is finally expressed as
$\arg \min_{h} \sum_{i} {(h_{i} - d_{i})}^{2} + λ \cdot (\frac{\sum_{q \in R (i)} g_{i, q} \cdot |{(\partial_{x} h)}_{q}|}{|\sum_{q \in R (i)} g_{i, q} \cdot {(\partial_{x} h)}_{q}| + ε} + \frac{\sum_{q \in R (i)} g_{i, q} \cdot |{(\partial_{y} h)}_{q}|}{|\sum_{q \in R (i)} g_{i, q} \cdot {(\partial_{y} h)}_{q}| + ε})$
(20)

Controlling the value of λ can affect the smoothness of the result, but it does not help much to separate the texture. Increasing λ will cause more blurriness, and λ = 0.02 can help to extract the pilling information better.

Here, $q$ belongs to $R (i)$ , the rectangular region centered at pixel $i$ and $g$ is the Gaussian kernel function
$g_{i, q} \propto \exp (- \frac{{(x_{i} - x_{q})}^{2} + {(y_{i} - y_{q})}^{2}}{2 σ^{2}})$
(21)

The window size is controlled by spatial parameters $σ$ for computing the windowed variations. Its value depends on the size of the texture, adjusting σ in the interval [1,5] depending on the fabric. For Equation (20), we first discuss the x direction; the y direction calculation is similar
$\sum_{q} \sum_{i \in R (q)} \frac{g_{i, q} |{(\partial_{x} h)}_{q}|}{|\sum_{q \in R (i)} g_{i, q} \cdot {(\partial_{x} h)}_{q}| + ε} = \sum_{q} \frac{{(G_{σ} * \frac{1}{|G_{σ} * \partial_{y} h| + ε})}_{q}}{|{(\partial_{y} h)}_{q}| + ε_{h}} {(\partial_{y} h)}_{q}^{2}$
(22)

In the experiments, ε and $ε_{h}$ are used to avoid the case where the denominator is 0. We set $ε$ = 0.001 and $ε_{h}$ = 0.02, because a slightly larger $ε_{h}$ can help preserve smoothly varying structures.

After the texture of the fabric is filtered by the structure extraction method based on RTV, the pills are segmented by the Otsu method, as shown in Figure 10.

Figure 10.
Pilling binary graph acquisition: (a) the fabric texture filtering image and (b) pilling segmentation.

Experiment and discussion

The severity of pilling is divided into five grades according to ASTM standards, where samples with a grade of 1 indicates very heavy pilling and a grade of 5 indicates almost no pilling. Five standard samples were selected according to ASTM standards, and manual pilling evaluation was achieved by comparing other samples with the standard samples. In this paper, photometric stereo with the Lambert model is used, which only considers the diffuse reflectance component and does not consider the effect of the specular component, so the samples are chosen from plain fabrics. Figure 11 shows the fabric images and depth images of the five standard samples.

Figure 11.
Standard pilling images. (a) Pilling grade 1; (b) pilling grade 2; (c) pilling grade 3; (d) pilling grade 4 and (e) pilling grade 5.

In this paper, 98 samples were used for fabric pilling evaluation, and three experts performed subjective evaluation of 98 samples by comparing with standard samples. The support vector machine (SVM) is an effective supervised learning model suitable for binary classification; its basic model is a linear classifier with the largest interval defined in the feature space. Due to the sample data being linearly separable, we use the linear kernel SVM as a classifier. Different characteristic parameters were selected as eigenvectors of the SVM (with a linear kernel) to complete the objective evaluation of fabric pilling, and the five-fold cross-validation method was used to avoid the overfitting problem caused by the small number of samples. Finally, comparison of subjective and objective pilling evaluation results determines the accuracy $P_{ACC}$ of objective pilling evaluation
$P_{ACC} = \frac{{N_{T} - N}_{M}}{N_{T}}$
(23)
where $N_{M}$ is the number of samples inconsistent between objective and subjective evaluation and $N_{T}$ is the total number of samples.

The pilling number, the pilling area, and the pilling coverage ratio were used as characteristic parameters for objective evaluation, respectively, as shown in Figure 12.

Figure 12.
Objective evaluation by different pilling features: (a) objective grade by the number, area, and coverage ratio of pills; (b) objective grade by the area and coverage ratio of pills; (c) objective grade by the number and area of pills; (d) objective grade by the number and coverage ratio of pills; (e) objective grade by the number of pills; (f) objective grade by the area of pills and (g) objective grade by the coverage ratio of pills.

Figure 12(a) shows the objective evaluation results using the pilling number, area, and coverage ratio as characteristic parameters with an accuracy of 93.9%; Figure 12(b) shows the objective evaluation results using the pilling area and coverage ratio as characteristic parameters with an accuracy of 87.8%; Figure 12(c) shows the objective evaluation results using the pilling number and area as characteristic parameters with an accuracy of 95.9%; Figure 12(d) shows the objective evaluation results using the pilling number and coverage ratio as characteristic parameters with an accuracy of 93.9%; Figure 12(e) shows the objective evaluation results using the pilling number as the characteristic parameter with an accuracy of 63.3%; Figure 12(f) shows the objective evaluation results using the pilling area as the characteristic parameter with an accuracy of 88.9%; Figure 12(g) shows the objective evaluation results using the pilling coverage ratio as the characteristic parameter with an accuracy of 86.7%.

In this paper, the pilling number and area were objectively evaluated as characteristic parameters, which had a higher consistency with the subjective evaluation results.

The sample images acquired under eight different light sources were used for 3D reconstruction, and the 3D models were used for the following experiments: 2D depth image generation, fabric texture filtering, pilling segmentation, and fabric pilling grade evaluation, as shown in Figure 13.

Figure 13.
Fabric pilling samples: (a) two-dimensional (2D) image of pilling; (b) three-dimensional (3D) model of pilling; (c) 2D depth image; (d) 2D depth map with texture filtered out and (e) pilling segmentation image.

Five-fold cross-validation of this experiment was performed and then the 98 samples were divided into five mutually exclusive subsets with approximately equal sample sizes. In the classification experiments, each subset was used for testing, and the remaining four subsets for training. Thus, each subset was tested exactly once. After testing of the SVM model was completed for all datasets, the test results were used to compare with the subjective evaluation results, and the comparison results were used to determine the accuracy of the classification experiments.

In order to verify the effectiveness of the SVM, seven classifiers were used for comparative experiments. As shown in Figure 14, the horizontal coordinate represents the pill number, the vertical coordinate represents the pill area, the green dots represent the misclassified samples, and the other color dots represent the subjective evaluation level of the fabric. The classification results of different classifiers are satisfactory and the accuracy is similar, proving that the method of extracting feature parameters in this system is reliable and can be adapted to different classifiers. Among them, the SVM with a linear kernel has the best result with 95.9% classification accuracy. The performance of the naive Bayesian classifier was the worst, with an accuracy of 92.9%.

Figure 14.
Comparison of different classifier results: (a) subjective evaluation, (b) support vector machine (SVM) classifier with a linear kernel, with accuracy of 95.9%; (c) SVM classifier with a Gaussian kernel, with accuracy of 94.9%; (d) k-nearest neighbor classifier, with accuracy of 93.9%; (e) decision tree, with accuracy of 94.9%; (f) naive Bayes, with accuracy of 92.9%; (g) quadratic discriminant analysis, with accuracy of 94.9% and (h) linear discriminant analysis, with accuracy of 94.9%. (Color online only.)

Conclusion

In this paper, a system based on photometric stereo was proposed, designed, and developed with the function of fabric 3D model reconstruction and objective pilling evaluation. The proposed system contains a camera and eight light sources with different locations. It is possible to generate a 3D model of fabric pilling using algorithms of near-light semi-calibrated photometric stereo. The reconstructed 3D model was used for the following experiments: 2D depth image generation, fabric texture filtering, pilling segmentation, and fabric pilling grade evaluation. For classification experiments, the pill number and pill area were selected as the eigenvectors. This experimental results of the study demonstrate the effectiveness and reliability of the system and method, with a recognition accuracy of 95.9%.The proposed pilling evaluation method does not perform well for pilling samples with complex colors, which will be supplemented in a subsequent study.

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Natural Science Foundation of China (grant 61876106).

ORCID iD

BinjieXin

References

Konda

Xin

Takadera

, et al. Evaluation of pilling by computer image analysis. J Text Mach Soc Jpn 1990; 36(3): 96–107.

Xin

Image based modeling and analysis of textile materials. Boston: Kluwer Academic Publishers, 2004.

Sekulska-Nalewajko

Goclawski

Korzeniewska

A method for the assessment of textile pilling tendency using optical coherence tomography. Sensors 2020; 20: 19.

Xu. Instrumental evaluation of fabric pilling. J Text Inst 1997; 88: 488–500.

Zhang

Objective grading of fabric pilling with wavelet texture analysis. Text Res J 2007; 77(11): 871–879.

Deng

Wang

An integrated method of feature extraction and objective evaluation of fabric pilling. J Text Inst 2011; 102: 1–13.

Jing

Zhang

Kang

, et al. Objective evaluation of fabric pilling based on wavelet transform and the local binary pattern. Text Res J 2012; 82: 1880–1887.

Xiao

Wang

Sun

, et al. Objective evaluation of fabric pilling based on image analysis and deep learning algorithm. Int J Cloth Sci Technol 2021; 33(4): 495–512.

Kim

Kang

TJ.

Automatic evaluation of fabric pilling using a 3-D non-contact scanning system. In: IEEE instrumentation & measurement technology conference, Ottawa, Canada, 16-19 May 2005, paper no. 8967788, pp. 628–632. Piscataway: IEEE.

10.

Ouyang

Wang

Fabric pilling measurement using three-dimensional image. J Electron Imag 2013; 22: 043031.

11.

Wang

Stereovision for three-dimensional measurements of fabric pilling. Text Res J 2011; 81: 631–635.

12.

Liu

Deng

Xin

, et al. Objective evaluation of fabric pilling based on multi-view stereo vision. J Text Inst 2021; 112: 1986–1997.

13.

AATCC Technical Manual. Appearance of apparel and other textile end products after repeated home laundering, 2014.

14.

Zhang

Flexible camera calibration by viewing a plane from unknown orientations. In: seventh IEEE international conference on computer vision, Kerkyra, Greece, 20-27 September 1999, paper no.6365272, pp. 666–673. Piscataway: IEEE.

15.

Ahmad

Sun

Smith

, et al. An improved photometric stereo through distance estimation and light vector optimization from diffused maxima region. Patt Recognit Lett 2014; 50: 15–22.

16.

Canny

A computational approach to edge detection. IEEE Trans Patt Anal Mach Intell 1986; PAMI-8: 679–698.

17.

Quéau

Durix

, et al. LED-based Photometric stereo: modeling, calibration and numerical solution. New York: Springer US , 2018.

18.

Quéau

Tao

Lauze

, et al. A non-convex variational approach to photometric stereo under inaccurate lighting. In: computer vision & pattern recognition, Honolulu, HI, USA, 21-26 July 2017, paper no.17355242, pp. 350–359.Piscataway: IEEE.

19.

Wang

Deng

Xin

Investigation of 3D surface profile reconstruction technology for automatic evaluation of fabric smoothness appearance. Measurement 2020; 166: 108264.

20.

Yan

Xia

, et al. Structure extraction from texture via relative total variation. ACM Trans Graph 2012; 31: 1.