Neighborhood preserving perceptual fidelity aware MSE for visual inspection of industrial flat surface products

Abstract

Automated visual inspection is becoming an important field of computer vision in many industries. The real-time inspection of flat surface products is a task full of challenges in industrial aspects that requires fast and accurate algorithms for detection and localisation of defects. Structural, statistical and filter-based approaches, such as Gabor Filter Banks, Log-Gabor filter and Wavelets, have high computational complexity.

This paper introduces a fast and accurate model for inspection and localization of industrial flat surface products: Neighborhood Preserving Perceptual Fidelity Aware Mean Squared Error (NP-PAMSE). The Extreme Learning Machine (ELM) is used for classification. ELM is found to be the perfect classifier for detecting defects. The proposed model resulted in defect detection accuracy of 99.86%, with 98.16% sensitivity, and 99.90% specificity.

These results show that the proposed model outperforms many existing defect detection approaches. The discriminant power displays the efficiency of ELM in differentiation between normal and abnormal surfaces.

Keywords

Automated visual inspection (AVI)perceptual fidelity aware mean squared error (PAMSE)extreme learning machine (ELM)

1 Introduction

Automation of defect detection in industrial flat surface products like textiles, paper, plastic sheets, lumber wood, etc. is a challenging task. Computer vision helps automation of the inspection process and has many advantages such as the inspection spead, cost and quality. A new trend in industry aims at building a digital twin [1] for the production machine to enhance product quality, product marketing and to simplify preventive maintence and enhance future versions of the production machines.

Achieving the above goals requires efficient machine vision algorithms for better defect detection.

This paper introduces a highly efficient defect detection and localization approach which relies on a measure of image Perceptual Fidelity Aware Mean Squared Error (PAMSE) which is used uptil now for Image Quality Assessment [2]. To apply PAMSE in the area of defect detection and localization; a novel algorithm has be developed to ensure efficient performance. A neighborhood preserving scanning mechanism is used to guarantee detection of any defect and localize it. Comparative analysis of the proposed approach indicates the advantage of NP-PAMSE over single block based approaches.

2 Related work

Automated visual inspection utilizes machine vision techniques for inspecting defects of industry and has received interest for the last years, particularly for inspecting of textured surfaces [3]. Studies on quality control and inspection of texture have important theoretical and practical value. Therefore, many techniques of texture analysis arise in this aspect. Generally, texture analysis approaches can be classified into the following four categories dependent on the image processing techniques that are used [4]:

2.1 Statistical approaches

These approaches employ the gray-scale distributions of image regions to designate texture properties. Statistical approaches are valuable particularly in cases of textures from nature, for example wood and ceramic. Gray-level co-occurrence matrix [5] and local descriptor-based approaches are representative statistical texture analysis methods.

2.2 Structural approaches

These approaches are specifically appropriate for textures with clear structural features. Properties of primary texture are directly affected by the visual structure effect and the pattern of the texture. Generally, structural approaches are applied to textures with repetitive patterns like fabrics and bricks [4].

2.3 Model-based approaches

These approaches characterize patterns of texture by modeling special distributions or other features with specific models. For example Gao et al. [6] presented an algorithm for spatial-frequency feature extraction utilizing smooth Itakura-Saito non-negative matrix factorization. This algorithm displayed better performance. Analogous methodologies aren’t just related to certain models with greater flexibility.

2.4 Filter-based approaches

These approaches are effective in describing textures in a transformed domain utilizing spatial transformations, filters, or filter banks. These approaches are broadly applied for texture analysis, description of texture and texture inspection. Fourier analysis [7], Wavelet transformation [8], and Gabor transform [9] are frequently utilized in analyzing textures.

The Fourier analysis/transformation is the traditional approach for analyzing of textile fabrics. Though, this approach is employed together with other approached for definite issues [10]. For example Schneider and Merhof [11] showed an automatic approach for detecting twill and plain textile fabric defects by merging fuzzy clustering, investigation of Fourier transform and matching of template. This approach is able to handle weaves, supplies, random image rotations and densities with no need to previous knowledge about the fabric. The approach utilizes information resulting from Fourier spectrum of fabric and merge them with features of local periodicities to construct a diagram, in which the trajectory of single yarns within the fabric can be approximated that enables assessing accuracy of the weave analysis. The approach showed to be robust with detection accuracy of 97%. Another unsupervised approach is used for periodic pattern textile cloth inspection by combining wavelet shrinkage and the analysis of Fourier coefficients was presented in [12]. The main strong point of this approach is that it is unnecessary to use reference image.

Wavelet transform had more interest in the field of the detection of fabric defects due to its prefect local time–frequency features [8 , 13]. Wavelet transform is alike to sub-band decomposition. Wavelet decomposition is achieved by a pyramidal algorithm that is used to compute wavelet coefficients. The two-dimensional (2D) wavelet transform is perceived as one-dimensional wavelet transform serially implemented with the vertical and horizontal axes [8].

Wavelet transform methods act good in outstanding edges defects, but unwell in flat surface defects with smooth gray-scale variances. In [14] an automated machine vision system is enhanced to inspect fabrics automatically and to automate the classification processes of the defects. The morphological operations, the two times thresholding binarization and transformation of wavelet coefficients are the base of the algorithm of detecting defects. The feed-forward neural network and the matrix of the gray level co-occurrence are the base for the approach of classifying the defects. Some types of commonly occurring defects were classified and detected, such as soiled yarn hole, lacking of wrap, lacking of weft and flowing of yarn. The detection accuracy of the fabric for the regions of the defect-free and the defectives was 93.4% and the accuracy of defect classification is 96.3%.

In [15], a novel automatic inspection system based on smart visual sensors is presented to detect defects on a warp knitting machine. It has the advantage of being of small size, low cost, and high reliability. An improved direct thresholding method based on a wavelet transform has been proposed to deal with the challenges. Satisfactory performance on the warp knitting machine has been achieved through a combination of both Wavelet Coefficients and a mathematical morphology filter. The detection rate is 98%.

Gabor filter is a linear filter utilized for analyzing textures. It is appropriate for emulating the biological properties of the eyes of human being and were applied in detecting defects in fabric. Many researches prove that Gabor filter has an ideal localization in two domains (frequency and space) through various dimensions and directions [3 , 16–18].

Recently, with the advancement of learning technology, techniques which use deep neural networks are intensively used in the field of inspecting defects in industry. Renet et al. [19] presented a supervised convolutional neural network (CNN) architecture for classifying patching of image using transfer learning. So that pixel-wise prediction is achieved with the classifier that is trained over input images. This proposed model displays efficiency in inspecting and discovering defects of wood. Weimer et al. [20] and Park et al. [21] presented a novel deep CNN architecture, which obviously increase the accuracy of recognizing defects automatically. In Li et al. [22], a Fisher criterion-based stacked denoising autoencoder (AE) system is proposed for detecting defects of textile.

In [23], a comprehensive and systematic literature review of Fabric defect detection systems and methods has conducted a survey of fabric defect detection approaches. It has been found that these approaches are classified into seven categories: Structural, statistical, spectral, model-based, learning, hybrid and comparison. Deduction of the best approach depends on many factors such as the fabric database used, the nature of fabric, the lighting conditions, image resolution, features extracted, classifier used, parameters, etc. Therefore, it is not fair/not objective to conduct comparisons without a reference standard for comparison. Therefore, our future work will focus on finding a benchmark as a ground for fair comparison and the develop a digital twin for enhancing the textile industry based on an Industry 4.0 approach. The approach presented here is novel to the defect detection field and will contribute to the advancement of textile industry. We cannot claim it generality for all types of fabrics.

3 Neighborhood-preserving PAMSE model

The architecture of this model is presented in Fig. 1. The proposed model has two main components: the feature extractor which depends on the Perceptual Fidelity Aware Mean Squared Error (PAMSE) vector with eight neighborhoods and the Extreme Learning Machine (ELM) for defect classification.

Fig. 1

Defect detection system architecture.

3.1 Neighborhood-preserving perceptual fidelity aware mean squared error methodology (NP-PAMSE)

The proposed algorithm depends on computing the PAMSE among the sliding window and its eight-neighboring windows in a 3×3 neighborhood, as presented in Fig. 1.

Algorithm: NP-PAMSE defect detection
Input: an image of size N×M
Output: defect image
1. Detecting the size [N×M pixels] of the complete
product’s image.
2. Set the size of the sliding window to 3×3 blocks, each
block size to [16×16 pixels] where b_h(block height)
=16 and b_w(block width)=16.
3. Read the complete product’s image beginning from the
top-left corner to the bottom-right corner.
4. For each window perform the following:
a. Calculate the PAMSEs among the central block and
its eight neighboring blocks P_i= PAMSE (B₀,B_i),
where B₀ is central block and B_i is the ith block,
i=1….8.
b. input the vector of PAMSE to the ELM for
classification.
5. Repeat step 4 until reaching the bottom-right corner.
6. End;

The proposed algorithm depends on a sliding window, in this scenario a 3×3 blocks (Fig. 2) consisting of 16×16 pixels for each block. New nine blocks are created along with every move of the window.

Fig. 2

A single sliding window with 8 neighboring.

PAMSEs are computed among the block in the center and the eight neighbor blocks Pi = PAMSE (B0,Bi), where B0 is central block and Bi is the ith block, i = 1 ... .8. PAMSE results in a block quality assessment index (BQAI) by using certain structure extractors, through comparing a central block with its 8 neighbors after Gaussian smooth filtering, which is much simpler to calculate but achieves rather better IQA performance than SSIM. The PAMSE can have great potentials in defect detection applications.

In [2], PAMSE was proposed as a simple yet very effective framework, for image quality assessment (IQA). The difference –and Laplacian operators are used to extract the structure error, so that the MSE between Gaussian smoothed reference and distorted images, SMSE measure the Perceptual fidelity Aware MSE (PAMSE). PAMSE is very simple to implement, highly efficient, very effective, and is valid distance metrics and is among the best ones for Image Quality Assessment tasks, works much better than the well-known Structural Similarity Index.

3.2.2 Extreme learning machine

ELM has a single hidden layer feed-forward neural network utilized in regression and classification. The ELM is able to adaptively set the hidden layer node number and arbitrarily allocate for input weights and hidden layer biases. The output layer weights are achieved via the least square process. The overall learning process is accomplished through a single pass without iteration.

ELM is an easily utilized single hidden layer feed-forward neural network, while conventional neural network learning algorithm (as Backpropagation algorithm) requires lots of training parameters, and lead to local optimum solution. Furthermore, ELM only requires setting the number of hidden layer nodes, and doesn’t require adjusting network input weights and hidden biases, and generates a unique optimum solution, with benefits of fast learning and generalization performance [24]. ELM overcomes slow training and over-fitting problems with the traditional neural network learning algorithm.

ELM is effective in numerous applications due to its superior abilities, robustness, and controllability and fast learning with more simplicity of its design [24]. Those advantages are more valuable for real-time applications, such as detecting defect.

ELM consist of three layers and the first layer nodes are utilized for feeding vector of input features into hidden layer. Both the size of the input layer and the feature set are the same. The input vector×contains the coefficients of PAMSE of the current block with its eight neighbors: X = {x₁, x₂, x₃, … … , …, x₈} which represent the eight BQAI’s. Figure 3 shows structure of Extreme Learning Machine.

Fig. 3

Structure of Extreme Learning Machine.

The output weights of ELM identify the value 1 for normal surface and the value 0 for abnormal

surface.

For C classes, the class labels are identified as y_k ∈ {1, 1 -} (1 ≤ k ≤ C). Therefore, a constructed row vector y = [y₁, , , y_k, , , , y_k] designates the class to which a sample belongs for example, if y_k = 1 and the other element in y are -1 then the sample belongs to kth class. Therefore, the training samples and matching labels are denoted as ${x_{i}, y_{i}}_{i = 1}^{n}$ , where and $y_{i} \in ℝ^{c}$ the following equation illustrates the outcome of L nodes hidden within the ELM: $f_{k} (x_{i}) = \sum_{j = 1}^{L} β_{j} h (w_{j} . x_{i} + b_{j}) = y_{i}, j = 1, . . . ., n$ (2)

Where, h(.) is a non-linear activation function, $β_{j} \in ℝ^{c}$ indicates weight vector connecting the jth hidden node to the output nodes, $w_{j} \in ℝ^{d}$ indicates weight vector connecting jth hidden node to input nodes, and bj is the bias of jth hidden node. The term w_j . x_i indicates inner product of w_j and x_i. If a value of 1 is padded to x_i to make it a (d + 1)-dimensional vector, then the bias can be considered as an element of the weight vector, which is also assigned randomly. For n formulas, (2) can be expressed as

$H β = Y$ (3) where, $Y = [y_{1}; y_{2}; . . .; y_{n}] \in ℝ^{n \times C}, β = [β_{1}; β_{2};$ $. . . .; β_{n}] \in ℝ^{L \times C}$ and H is the hidden layer output matrix of the neural network represents as follow $\begin{matrix} H = (\begin{matrix} h (x_{1}) \\ ⋮ \\ h (x_{n}) \end{matrix}) = \\ (\begin{matrix} h (w_{1} . x_{1} + b_{1}) & \dots & h (w_{L} . x_{1} + b_{L}) \\ ⋮ & ⋱ & ⋮ \\ h (w_{1} . x_{n} + b_{1}) & \dots & h (w_{L} . x_{n} + b_{L}) \end{matrix}) \end{matrix}$ (4) in (4) h (x_i) = (h (w₁ . x_i + b₁) , . . . . , h (w_L . x_i + b_L)) is hidden nodes output in response to the input x_i, which define data from d-dimensional input space to L-dimensional feature space. Hidden neurons number is much less than training samples number, i.e., L ⪡ n, least squares solution of (3) can be utilized $β^{'} = H^{†} Y$ (5) where, the matrix H^† = H^T (HH^T) ^-1 and H is the inverse of the Moore-Penrose H^† [25].

4 Performance evaluation

The percentage of correct detection (PCD) is in general utilized in evaluating the performance of the presented approaches. Though, there is a variety of measures for assessing the classifiers performance such as: $Accuracy α 4 = \frac{TA + TN}{TA + TN + FA + FN}$ (6) $Recall (Sensitivity) α 2 = \frac{TA}{TA + FN}$ (7) $Precision α 1 = \frac{TA}{TA + FA}$ (8) $Specificity α 3 = \frac{TN}{TN + FA}$ (9) $F 1 score = \frac{2 TA}{2 TA + FA + FN}$ (10)

All of the previously mentioned measures are normally stated as percentages. FN, FA. TA and TN terms mentioned in these equations are abbreviation of False Normal, False Abnormal, True Abnormal and True Normal respectively. By using the confusion matrix of detecting defects, these terms can be easily calculated [26].

Sokolova et al. [27] have reported that the measurement of accuracy doesn’t differentiate among the different classes correct labels number. The performance of classifiers for different classes are separately assessment by specificity and sensitivity. It has also been reported [27] that greater accuracy doesn’t guarantee the better performance of algorithm and the performance of algorithm gets an evaluation balance by giving the measures combinations. Discriminant Power (DP) given by Sokolova et al. [27] and Youden’sIndex, are used to assess the performance of the proposal model: ${Youden}^{'} s Index y = α 2 - (1 - α 3)$ (11) $Positive Likelihood P + = α 2 / (1 - α 3)$ (12) $Negative Likelihood P - = (1 - α 2) / α 3$ (13) $\begin{matrix} Discriminant Power DP = \sqrt{3} / π [log (α_{2} / 1 - α_{2}) \\ + log (α_{3} / 1 - α_{3})] \end{matrix}$ (14) where, α1 is precision, α2 is sensitivity and α3 is specificity.

Youden’s Index evaluates the performance of classifiers to a better degree regarding the two classes. The abnormal positive class has a superior performance when the P + positive value is greater. The normal negative class has a superior performance when the P-negative value is greater. The Discriminant Power assesses how efficient a classifier distinguishes among the surfaces of abnormal and normal. The performance of classifier is weak if DP < 1, limited if DP < 2, fair if DP < 3, and efficient in other cases [27].

5 Experiments and discussion

5.1 Database of TILDA

We made the experiments utilizing a big data set of fabrics textured images from the TILDA database [28] to assess the proposed model. The textile fabrics images were obtained in gray-scale with a resolution of 768×512 pixels (obtained from an image 1380 images with 16×16). Every sample space is scanned by the sliding window that has width (wv) and height (wh).The window size while scanning is wh×wv pixels. The image is being scanned by the window from the top-left corner until the bottom-right corner.

Table 7 gives a comparison of model performance for NP-PAMSE, neighborhood-maintaining minimum ratio (NMMR) [34], neighborhood-preserving cross-correlation coefficients (NP-CCFs) [29] and Log-Gabor (LG) [30].

Figure 4 shows the performance comparison of NP-PAMSE with different models Performed on a LENOVOG50-80 Laptop with an Intel(R) Core(TM) i3-4005UCPU@1.70 GHz(4CPUs) in a Matlab programming environment R2015a.

Fig. 4

Performance comparison of NP-PAMSE with different models Performed on a LENOVO G50-80 Laptop with an Intel(R) Core(TM) i3-4005U CPU @ 1.70 GHz (4CPUs) in a Matlab programming environment R2015a.

5.2 Effect of patch size on detection performance

Tables 8, 9 show the performance of PAMSE and NP-PAMSE for different patch size respectively. The patch size 16×16 in PAMSE gives the best performance results, but the better performance results for all patch size given within NP-PAMSE as shown in Table 10.

Figure 5 shows that the patch size 16×16 results in the best performance for the tested set of textile samples in PAMSE.

Fig. 5

Effect of Patch Size on Defect Detection Performance for PAMSE.

Figure 6 shows that all patch size results in the best performance for the tested set of textile samples in PAMSE.

Fig. 6

Effect of Patch Size on Defect Detection Performance for PAMSE.

Figures 7 and 8 shows the effect of block size on defect detection and localization for both PAMSE and NP-PAMSE.

Fig. 7

Effect of Patch Size on Defect Detection Performance in case of NP-PAMSE.

Fig. 8

Effect of Patch Size on Defect Detection Performance in case of PAMSE.

5.3 Results and discussion

Table 1 displays the results of 11 samples (images) from class 1 of TILDA database. The sign (e) in the sample space code shows the class of defect, from code 1 to code 7, the code is added as the following e1, e2...etc.,the codes are as the following (1: texture cuts and holes, 2: represents the stains, 3: represents the defects in structure, 4: represents foreign bodies, 5: represents the fabric folds, 6: represents the change in lighting conditions, 7: represents the affine distortions of the captured image) [26]. The images were obtained at gray level with a resolution of 768×512 pixels.

Table 1
Results of detecting defects sample space of textile by NP-PAMSE & PAMSE-ELM

Tables 2, 3 show the performance of PAMSE and NP-PAMSE respectively, the performance of PAMSE was not efficient at some type of defects. The better performance is shown by NP-PAMSE as displayed in Table 4.

Table 2

Performance indicators for PAMSE-ELM

Sample	TA	FA	TN	FN	Accuracy	Sensitivity	Specificity	Precision	F1 score	Y	P+	P–	DP1
1	18	7	1426	6	99.11	75.00	99.51	72.00	0.73	0.7678	158	0.23	2.49
2	28	1	1420	8	99.38	77.78	99.93	96.55	0.86	0.7771	1105	0.22	3.42
3	23	0	1426	8	99.45	74.19	100.00	100.00	0.85	0.7419	∞	0.26	∞
4	59	0	1396	2	99.86	96.72	100.00	100.00	0.98	0.9672	∞	0.03	∞
5	35	0	1413	9	99.38	79.55	100.00	100.00	0.89	0.7955	∞	0.20	∞
6	33	0	1410	14	99.04	70.21	100.00	100.00	0.83	0.7021	∞	0.30	∞
7	26	0	1430	1	99.93	96.30	100.00	100.00	0.98	0.9630	∞	0.04	∞
8	46	0	1410	1	99.93	97.87	100.00	100.00	0.99	0.9787	∞	0.02	∞
9	43	61	5863	18	98.68	70.49	98.97	41.35	0.52	0.6946	68	0.30	2.11
10	61	0	5890	34	99.43	64.21	100.00	100.00	0.78	0.6421	∞	0.36	∞
11	32	5	5909	39	99.26	45.07	99.92	86.49	0.59	0.4499	533	0.55	3.20
12	11	20	311	3	93.33	78.57	93.96	35.48	0.49	0.7253	13	0.23	1.27
13	20	14	300	11	92.75	64.52	95.54	58.82	0.62	0.6006	14	0.37	1.37
14	14	11	310	10	93.91	58.33	96.57	56.00	0.57	0.5491	17	0.43	1.48
Average					98.10	74.92	98.89	81.91	0.76	0.74	∞	0.25	∞

Table 3

Performance indicators for NP-PAMSE-ELM

Sample	TA	FA	TN	FN	Accuracy	Sensitivity	Specificity	Precision	F1 score	Y	P+	P–	DP
1	22	1	1357	0	99.93	100.00	99.93	95.65	0.9778	0.9993	1358	0.00	∞
2	52	1	1327	0	99.93	100.00	99.92	98.11	0.9905	0.9992	1328	0.00	∞
3	51	2	1327	0	99.86	100.00	99.85	96.23	0.9808	0.9985	665	0.00	∞
4	124	0	1256	0	100.00	100.00	100.00	100.00	1.0000	1.0000	∞	0.00	∞
5	92	2	1286	0	99.86	100.00	99.84	97.87	0.9892	0.9984	644	0.00	∞
6	111	1	1267	1	99.86	99.11	99.92	99.11	0.9911	0.9903	1257	0.01	∞
7	48	0	1332	0	100.00	100.00	100.00	100.00	1.0000	1.0000	∞	0.00	∞
8	89	0	1291	0	100.00	100.00	100.00	100.00	1.0000	1.0000	∞	0.00	∞
9	45	0	3581	0	100.00	100.00	100.00	100.00	1.0000	1.0000	∞	0.00	∞
10	161	1	3464	0	99.97	100.00	99.97	99.38	0.9969	0.9997	3465	0.00	∞
11	106	2	3517	1	99.92	99.07	99.94	98.15	0.9860	0.9901	1743	0.01	∞
13	14	1	619	0	99.84	100.00	99.84	93.33	0.9655	0.9984	620	0.00	∞
14	30	1	603	0	99.84	100.00	99.83	96.77	0.9836	0.9983	604	0.00	∞
15	41	2	591	0	99.68	100.00	99.66	95.35	0.9762	0.9966	296	0.00	∞
P1	4	0	181	1	99.46	80.00	100.00	100.00	0.8889	0.8333	∞	0.17	∞
P2	19	3	824	2	99.41	90.48	99.64	86.36	0.8837	0.9904	104	0.00	∞
P3	4	0	630	0	100.00	100.00	100.00	100.00	1.0000	0.7484	473	0.25	∞
Average					99.86	98.16	99.90	97.43	0.98	0.97	0.03	∞

Table 4

Comparison of Performance Indicators for PAMSE-ELM &NP-PAMSE-ELM

Performance Indicator	PAMSE	NP-PAMSE
Average Accuracy	98.10	99.86
Average Sensitivity	74.92	98.16
Average Specificity	98.89	99.90
Average Precision	81.91	97.43
Average F1 Score	0.76	0.98

Table 5 shows the comparison between system performance of NP-PAMSE using ELM and SVM. ELM shows the best and fast performance compared to the SVM classifier. The F1 score of ELM is 0.98, while SVM is 0.76.The average of training time for ELM is 0.088, while the SVM is 0.42.

Table 5

Comparison between system performance of NP-PAMSE using ELM and SVM

Performance Indicator	ELM	SVM
Average Accuracy	99.86	96.33
Average Sensitivity	98.16	99.74
Average Specificity	99.90	96.09
Average Precision	97.43	60.94
Average F1 Score	0.98	0.76
Average training time (s)	0.088	0.42

Table 6 demonstrates the comparison between the proposed approaches and the recently published approaches, which are using the same standard database “TILDA”. The proposed system results indicate better performance than other approaches.

Table 6

Comparison between our work and recently published works

Databases	Authors/Years	Approaches	Accuracy (%)	Sensitivity (%)	Specificity (%)	Precision (%)
TILDA	A. S. Tolba [29], 2011	Neighborhood-Preserving Cross-Correlation Coefficients	99.87	99.29	99.91	–
	A. S. Tolba [30], 2011	log Gabor filter	99.77	99.83	99.65	97.35
	Hazem M Raafat et al. [26], 2015	Levene’s test (LT)	99.73	99.96	99.72	93.21
	Zhang Duo et al. [31], 2016	Gabor filter and low-rank decomposition	91.2	86.3	–	70.3
	Li Liu et al. [32], 2019	Unsupervised segmentation and ELM	94.5	97.6	91.5	92
	Guang-Hua Hu et al. [33], 2014	The wavelet-domain hidden Markov tree (HMT)	92.1	–	–	–
	Ehab Essa et al. [34], 2019	Toward cognitive support	98.07	91.29	99.67	99.67
	LE TONG et al. [35], 2017	A Nonlocal Sparse Representation	94.6	97.8	91.5	92.00
	Proposed approaches	PAMSE &ADP	99.07	80.25	99.59	83.26
		NP-PAMSE &ELM	99.94	99.83	99.94	98.59

Table 7 demonstrates comparison among the proposed NP-PAMSE model and other models based on log-Gabor (LG), neighborhood-preserving cross-correlation coefficients (NP-CCF) and neighborhood-maintaining minimum ratio (NMMR) [12 , 34].This proposal model achieves higher speed than other models and similar performance. The running time of the NP-PAMSE detecting defects model is extremely fast compared to other models, because it has low computation complexity, and the training phase is done extremely fast by utilizing ELM to distinguish between the normal –and the defective surfaces.

Table 7

Performance comparison of different models

Performance Indicator	NP-PAMSE	NMMR	NP-CCF	LG
Average recall	98.1558	91.29	99.1636	100
Average specificity	99.9032	99.67	99.9091	97.77
Average accuracy	99.8560	98.07	99.8636	95.67
Average Yuden’s index	0.9730	0.9097	0.9944	0.9826
Feature extraction time (seconds)	0.00034	0.00047	0.0006	0.0087

Table 8

Effect of Patch Size on Defect Detection Performance for PAMSE

Performance Indicator	16×16	8×8	32×32
Average Accuracy	99.51	99.12	93.33
Average Sensitivity	83.45	59.92	67.14
Average Specificity	99.93	99.63	95.36
Average Precision	96.07	75.95	50.10
Average F1 Score	0.89	0.63	0.56

Table 9

Effect of Patch Size on Defect Detection Performance for NP-PAMSE

Performance Indicator	16×16	8×8	32×32
Average Accuracy	99.93	99.96	99.79
Average Sensitivity	99.89	99.69	100.00
Average Specificity	99.93	99.97	99.78
Average Precision	98.37	99.18	95.15
Average F1 Score	0.99	0.99	0.98

6 Conclusion and future work

This paper, introduced a novel NP-PAMSE for automated defect detection in textiles. Perceptual Fidelity Aware Mean Squared Error approach is used for feature extraction and the Extreme Learning Machine (ELM) is used for classification. This model depends on computing of the PAMSE vector among the located block and its eight neighboring blocks in a 3×3 neighborhood window. The scanning window in this model helps to detect and localize defects accurately. The model’s detection performance has been analyzed depending on tests performed on a large set of textile fabric samples taken from TILDA database. ELM has been found to be very appropriate for detecting defects. In the NP-PAMSE model the accuracy of detecting defects has reached 99.86%, with recall/sensitivity 98.16% and specificity 99.90%. These results demonstrate that the NP-PAMSE model outperforms many existing defect detection techniques. The discriminant power demonstrates the efficiency of the ELM in distinguishing among normal- and abnormal surfaces. A major advantage of the NP-PAMSE is fast defect detection compared to recent published approaches.

Futere work

The proposed technique is fast and accurate in defect detection and localization, so in the future we will try to use it to construct a digital twin system for textile production.

References

Madni

A.M.

, Madni

C.C.

and Lucero

S.D.

, Leveraging digital twin technology in model-based systems engineering, Systems 7(1) (2019), 7.

Xue

, Mou

, Zhang

and Feng

, Perceptual fidelity aware mean squared error. In Proceedings of the IEEE International Conference on Computer Vision, 2013, (pp. 705–712).

G.-H.

, Automated defect detection in textured surfaces using optimal elliptical Gabor filters, Optik 126(14) (2015), 1331–1340.

Mei

, Yang

and Yin

, An unsupervised-learning-based approach for automated defect inspection on textured surfaces, IEEE Transactions on Instrumentation and Measurement 67(6) (2018), 1266–1277.

Vujasinovic

, Pribic

, Kanjer

, Milosevic

N.T.

, Tomasevic

, Milovanovic...

and Radulovic

, Gray-level co-occurrence matrix texture analysis of breast tumor images in prognosis of distant metastasis risk, Microscopy and Microanalysis 21(3) (2015), 646–654.

Gao

, Zhang

, Woo

W.L.

, Tian

G.Y.

, Bai

and Yin

, Smooth nonnegative matrix factorization for defect detection using microwave nondestructive testing and evaluation, IEEE Transactions on Instrumentation and Measurement 63(4) (2013), 923–934.

, Yang

, Ling

and Xu

, Wavelet domain multifractal analysis for static and dynamic texture classification, IEEE Transactions on Image Processing 22(1) (2012), 286–299.

Zhu

, Liu

, Pan

, Gao

and Liu

, Seam detection of inhomogeneously textured fabrics based on wavelet transform, Textile Research Journal 85(13) (2015), 1381–1393.

Zuñiga

A.G.

, Florindo

J.B.

and Bruno

O.M.

, Gabor wavelets combined with volumetric fractal dimension applied to texture analysis, Pattern Recognition Letters 36 (2014), 135–143.

10.

and Zhang

, Automated vision system for fabric defect inspection using Gabor filters and PCNN, SpringerPlus 5(1) (2016), 765.

11.

Schneider

and Merhof

, Blind weave detection for woven fabrics, Pattern Analysis and Applications 18(3) (2015), 725–737.

12.

G.-H.

, Wang

Q.-H.

and Zhang

G.-H.

, Unsupervised defect detection in textiles based on Fourier analysis and wavelet shrinkage, Applied optics 54(10) (2015), 2963–2980.

13.

, Zhang

, Jing

, Li

and Zhao

, Fabric defect detection based on multi-scale wavelet transform and Gaussian mixture model method, The Journal of The Textile Institute 106(6) (2015), 587–592.

14.

Çelik

, Dülger

and Topalbekiroğlu

, Development of a machine vision system: real-time fabric defect detection and classification with neural networks, The Journal of The Textile Institute 105(6) (2014), 575–585.

15.

, Ai

and Sun

, Online fabric defect inspection using smart visual sensors, Sensors 13(4) (2013), 4659–4673.

16.

Çelik

H.I.

, Topalbekitoglu

and Dülger

, Real-time denim fabric inspection using image analysis, Fibres & Textiles in Eastern Europe, 2015.

17.

Bissi

, Baruffa

, Placidi

, Ricci

, Scorzoni

and Valigi

, Automated defect detection in uniform and structured fabrics using Gabor filters and PCA, Journal of Visual Communication and Image Representation 24(7) (2013), 838–845.

18.

Jing

, Yang

, Li

and Kang

, Supervised defect detection on textile fabrics via optimal Gabor filter, Journal of Industrial Textiles 44(1) (2014), 40–57.

19.

Ren

, Hung

and Tan

K.C.

, A generic deep-learning-based approach for automated surface inspection, IEEE transactions on cybernetics 48(3) (2018), 929–940.

20.

Weimer

, Scholz-Reiter

and Shpitalni

, Design of deep convolutional neural network architectures for automated feature extraction in industrial inspection, CIRP Annals 65(1) (2016), 417–420.

21.

Park

J.K.

, Kwon

B.K.

, Park

J.H.

and Kang

D.J.

, Machine learning-based imaging system for surface defect inspection, International Journal of Precision Engineering and Manufacturing-Green Technology 3(3) (2016), 303–310.

22.

, Zhao

and Pan

, Deformable patterned fabric defect detection with fisher criterion-based deep learning, IEEE Transactions on Automation Science and Engineering 14(2) (2017), 1256–1264.

23.

Hanbay

, Talu

M.F.

and Özgüven

Ö.F.

, Fabric defect detection systems and methods—A systematic literature review, Optik 127(24) (2016), 11960–11973.

24.

Ding

, Zhao

, Zhang

, Xu

and Nie

, Extreme learning machine: algorithm, theory and applications, Artificial Intelligence Review 44(1) (2015), 103–115.

25.

, Chen

, Su

and Du

, Local binary patterns and extreme learning machine for hyperspectral imagery classification, IEEE Transactions on Geoscience and Remote Sensing 53(7) (2015), 3681–3693.

26.

Raafat

H.M.

and Tolba

, Homoscedasticity for defect detection in homogeneous flat surface products, Textile Research Journal 85(8) (2015), 850–866.

27.

Sokolova

, Japkowicz

and Szpakowicz

, Beyond accuracy, F-score and ROC: a family of discriminant measures for performance evaluation. in Australasian joint conference on artificial intelligence. 2006. Springer.

28.

TILDA, Textile defect image database. University of Freiburg, Germany, 1996. http://lmb.informatik.unifreiburg.de/research/dfgtexture/tilda.

29.

Tolba

, Neighborhood-preserving cross correlation for automated visual inspection of fine-structured textile fabrics, Textile Research Journal 81(19) (2011), 2033–2042.

30.

Tolba

, Fast defect detection in homogeneous flat surface products, Expert Systems with Applications 38(10) (2011), 12339–12347.

31.

Zhang

, Gao

and Li.

Fabric defect detection algorithm based on Gabor filter and low-rank decomposition. in Eighth International Conference on Digital Image Processing (ICDIP 2016). 2016. International Society for Optics and Photonics.

32.

Liu

, Zhang

, Fu

, Liu

and Huang

, Unsupervised segmentation and elm for fabric defect image classification, Multimedia Tools and Applications 78(9) (2019), 12421–12449.

33.

G.-H.

, Zhang

G.-H.

and Wang

Q.-H.

, Automated defect detection in textured materials using wavelet-domain hidden Markov models, Optical Engineering 53(9) (2014), 093107.

34.

Essa

, Hossain

M.S.

, Tolba

A.S.

, Raafat

H.M.

, Elmogy

and Muahmmad

, Toward cognitive support for automated defect detection, Neural Computing and Applications, 2019, 1–9.

35.

Tong

, Wong

W.K.

and Kwong

, Fabric defect detection for apparel industry: a nonlocal sparse representation approach, IEEE access 5 (2017), 5947–5964.

Neighborhood preserving perceptual fidelity aware MSE for visual inspection of industrial flat surface products

Abstract

Keywords

1 Introduction

2 Related work

2.1 Statistical approaches

2.2 Structural approaches

2.3 Model-based approaches

2.4 Filter-based approaches

3 Neighborhood-preserving PAMSE model

5.1 Database of TILDA

Table 1 Results of detecting defects sample space of textile by NP-PAMSE & PAMSE-ELM

Futere work

References

Table 1
Results of detecting defects sample space of textile by NP-PAMSE & PAMSE-ELM