A novel intelligent fault diagnosis method based on variant sparse filtering and back-propagation

Abstract

Detecting the mechanical faults of rotating machinery in time plays a key role in avoiding accidents. With the coming of the big data era, intelligent fault diagnosis methods based on machine learning models have become promising tools. To improve the feature learning ability, an unsupervised sparse feature learning method called variant sparse filtering is developed. Then, a fault diagnosis method combining variant sparse filtering with a back-propagation algorithm is presented. The involvement of the back-propagation algorithm can further optimize the weight matrix of variant sparse filtering using label data. At last, the developed diagnosis method is validated by rolling bearing and planetary gearbox experiments. The experiment results indicate that the developed method can achieve high accuracy and good stability in rotating machinery fault diagnosis.

Keywords

fault diagnosis rotating machinery feature learning variant sparse filtering

Introduction

Gears, bearings, and shafts are widely used in transmission systems that exist in automobiles, power equipment, and construction machinery. Defects could appear in rotating machinery after a long period of operation. As a result, the life of a machine is shortened, and even casualties could occur, which can be avoided greatly using a health monitoring system. Massive signals are collected by a health monitoring system. Thus, how to process signals quickly and improve the diagnostic performance of the health monitoring system has become a great challenge.¹

Traditional fault diagnosis methods involving signal processing techniques require specialized knowledge and rich experience to extract appropriate fault information or features.^2,3 With the coming of the big data era, data-driven methods using machine learning models are attracting the attention of scholars.^4,5 The main diagnostic steps of the data-driven method are (1) obtaining features using a feature learning method and (2) recognizing health conditions using a feature classifier. A data-driven method can achieve the fault diagnosis by training the machine learning models, which can save the time required for extracting features deliberately and manually.

There are various machine learning models that can learn useful features in an unsupervised or supervised way,⁶ such as back-propagation neural networks (BPNNs),⁷ autoencoders,⁸ restricted Boltzmann machine (RBM),⁹ and convolutional neural networks (CNNs).¹⁰ As a traditional machine learning model, the core idea of BPNN has been widely applied in other models. Wang et al.¹¹ achieved rotating machinery fault diagnosis by combing stacked autoencoders (SAEs) with batch normalization. Stacked RBMs can compose a deep belief network (DBN),¹² which can further improve the feature learning capability of the network. Zhang et al.¹³ developed an enhanced CNN for bearing fault diagnosis by combing short-time Fourier transform and hierarchical regularization. An et al.¹⁴ presented a three-stage diagnosis method based on a recurrent neural network to recognize faults under time-varying working conditions. Capsule neural network has recently been a hot topic.¹⁵ Zhu et al.¹⁶ established a learning model by combing a CNN and a capsule neural network. The model can possess a strong generalization ability for bearing fault diagnosis.

Sparse feature learning methods have attracted much attention because sparse features are discriminative. Various sparse feature learning methods have been proposed, such as sparse autoencoder,¹⁷ sparse coding,¹⁸ sparse RBM,¹⁹ and sparse filtering.²⁰ One of the advantages of sparse filtering is that it does not need to reconstruct the input data. Lei et al.²¹ successfully applied sparse filtering to the bearing fault diagnosis. Compared with other data-driven methods, the diagnosis method based on sparse filtering can achieve high accuracy using fewer training samples. Wang et al.²² developed an enhanced sparse filtering method by applying an L_1/2 regularization term to recognize health conditions under variable rotational speed.

The core idea of sparse filtering is applying l₂-normalization to achieve competition between the normalized features. However, it has been demonstrated that l₂-normalization is not the optimal choice for learning discriminative features.²³ Besides, the weights of sparse filtering are not optimized when a classifier is trained using label data, which limits the feature learning capability.

One of the main contributions of this work is the development of a variant sparse filtering (VSF) method involving l₁-normalization and l₃-normalization. Compared with l₂-normalization, the application of l₁-normalization and l₃-normalization can improve the feature learning capability of sparse filtering. Another main contribution is the combination of VSF and back-propagation (BP) algorithm to achieve the VSF-BP method. The weights of the VSF model can be further optimized with label data by applying the BP algorithm.

The rest of the paper is as follows. In The proposed method, VSF is developed and further combined with the BP algorithm to achieve a VSF-BP method. In Experimental validation, the developed diagnosis method is verified by a rolling bearing dataset and a planetary gearbox dataset. Conclusions are drawn in Conclusions.

The proposed method

Variant sparse filtering

VSF can be regarded as a two-layer neural network including an input layer and a feature layer. The inputs of VSF are collected samples. It is assumed that there is a training dataset ${x^{i}}_{i = 1}^{M}$ , where $x^{i} \in ℛ^{N \times 1}$ is the ith sample, and M is the number of training samples. VSF outputs a feature set ${f^{i}}_{i = 1}^{M}$ by a weight matrix W , where $f^{i} \in ℛ^{L \times 1}$ is the ith feature vector. The relationship between the feature vector $f^{i}$ and the sample $x^{i}$ is

f^{i} = ϕ (W x^{i})

(1)

where

ϕ (\cdot)

denotes an activation function. Absolute value function is used as the activation function in VSF. The feature set

{f^{i}}_{i = 1}^{M}

is further represented as a feature matrix f for simplicity.

To achieve a sparse feature representation, each row of the feature matrix f is normalized by its l₁-norm, which yields

{\tilde{f}}_{j} = \frac{f_{j}}{{‖ f_{j} ‖}_{1}}

(2)

where

f_{j}

denotes the jth row vector of the feature matrix f . Then, each column of the normalized feature matrix

\tilde{f}

is normalized by its l₃-norm, leading to

{\hat{f}}^{i} = \frac{{\tilde{f}}^{i}}{{‖ {\tilde{f}}^{i} ‖}_{3}}

(3)

The representation of sparse features can be achieved by minimizing a cost function that involves an l₂-norm

C = \sum_{i = 1}^{M} {‖ {\hat{f}}^{i} ‖}_{2}

(4)

This cost function can be minimized with a limited-memory Broyden–Fletcher–Goldfarb–Shanno (L-BFGS) algorithm.²⁴ To ensure that the cost function is differentiable at any point, a soft-absolute value function is used to activate the features. Thus, the features are obtained by

f^{i} = \sqrt{W x^{i} + ε}

(5)

where

ε = 10^{- 8}

To explain the mechanism of VSF, a feature set containing only two feature vectors is taken as the example. Each feature vector contains only two features. The two feature vectors are represented as $f^{1} = {[f_{1}^{1}, f_{2}^{1}]}^{T}$ and $f^{2} = {[f_{1}^{2}, f_{2}^{2}]}^{T}$ . Thus, the feature matrix can be written as $f = [f^{1}, f^{2}]$ . The illustration of VSF learning sparse solutions is shown in Figure 1. The first step is the l₁-normalization of row vectors of the feature matrix f , which leads to the following constraint

| {\tilde{f}}_{j}^{1} | + | {\tilde{f}}_{j}^{2} | = 1

(6)

where

j = 1, 2

. Since the features are all positive, the normalized row vectors are restricted to a straight line in the first quadrant, as shown in Figure 1(a). The second step is the l₃-normalization of column vectors of the normalized feature matrix

\tilde{f}

, which leads to the following constraint

{| {\hat{f}}_{1}^{i} |}^{3} + {| {\hat{f}}_{2}^{i} |}^{3} = 1

(7)

where

i = 1, 2

. Similarly, the normalized column vectors are restricted to a curve in the first quadrant, as shown in Figure 1(b). It can be seen from equations (6) and (7) that there is competition between the normalized features. The increase of a feature will lead to a decrease in the other feature. The last step is minimizing the cost function involving an l₂-norm. To simplify the cost function further, only one feature vector is considered, which leads to

C = {({| {\hat{f}}_{1}^{1} |}^{2} + {| {\hat{f}}_{2}^{1} |}^{2})}^{1 / 2}

(8)

Figure 1.

Illustration of VSF learning sparse solutions.

The cost function in equation (8) is shown in Figure 1(c). It can be found that there are two sparse solutions due to the constraint in equation (7). The L-BFGS algorithm would search for the sparse solutions from an initial solution along the direction of gradient descent. As a result, VSF can learn a sparse representation of features.

VSF-BP method for fault diagnosis

The VSF-BP method mainly contains three learning stages: training a VSF model, training a softmax regression model, and fine-tuning the network. The training process of the VSF-BP method is shown in Figure 2.

(1) Training a VSF model

Figure 2.

Training process of the VSF-BP method.

Frequency-domain data are used as inputs to train the VSF model. Spectra can be easily obtained from time-domain signals using fast Fourier transformation (FFT). Therefore, the training dataset is ${x^{i}}_{i = 1}^{M}$ , where $x^{i} \in ℛ^{N \times 1}$ is the ith spectrum with N spectral lines, and M denotes the number of spectra. Minimizing the cost function in equation (4) can obtain the weight matrix of the VSF model.

(2) Training a softmax regression model

Once the VSF model is trained, a feature set ${f^{i}}_{i = 1}^{M}$ can be obtained using equation (5). Then, the feature set ${f^{i}}_{i = 1}^{M}$ and a label set ${y^{i}}_{i = 1}^{M}$ are used as inputs to train the softmax regression model, where $y^{i} \in {1,2, \dots, D}$ , and D represents the number of health conditions. The softmax regression tries to minimize the following cost function

H (θ) = - \frac{1}{M} [\sum_{i = 1}^{M} \sum_{d = 1}^{D} 1 {y^{i} = d} \log \frac{e^{θ_{d} f^{i}}}{\sum_{j = 1}^{D} e^{θ_{j} f^{i}}}] + \frac{λ}{2} \sum_{d = 1}^{D} \sum_{l = 1}^{L} θ_{d l}^{2}

(9)

where

θ

represents the weight matrix of the softmax regression model,

λ

is a regularization parameter of the decay term, and

θ_{d l}

is an element in the dth row and lth column in the matrix

θ

1 {\cdot}

is an indicator function that outputs 1 when the condition is satisfied and 0 when the condition is not satisfied.

(3) Fine-tuning the network

Once the VSF model and the softmax regression model are trained, the weight matrices W and $θ$ are used as initial parameters to optimize the whole network with the BP algorithm. The gradient descent method was used in the BP algorithm. The object of the BP algorithm is to further minimize the cost function defined in equation (9) using label data. In the minimization process, the weight matrices W and $θ$ are further fine-tuned, which can improve the feature learning capability of the VSF model. As a result, the optimal weight matrix $W^{opt}$ and bias term b are obtained for the VSF model, and the optimal weight matrix $θ^{opt}$ is obtained for the softmax regression model.

After the three-layer network is optimized, the spectra are fed into the network for testing. The health conditions of rotating machinery can be recognized by carrying out a forward propagation.

Experimental validation

Case 1: Rolling element bearing fault diagnosis

A rolling element bearing dataset provided by Case Western Reserve University²⁵ was used to verify the diagnostic performance of the VSF-BP method. The test bench mainly consisted of a motor, an acceleration sensor, and a dynamometer. The vibration acceleration signals were collected from the motor drive end where a rolling element bearing was installed. Three kinds of faults were deliberately set up, including a ball fault (BF), an inner race fault (IF), and an outer race fault (OF). Three levels of defect were set up for each fault, including 0.18, 0.36, and 0.53 mm. Therefore, the bearing dataset contained 10 different health conditions, including a normal condition (NC). For simplicity, the fault conditions were named as BF1, BF2, BF3, IF1, IF2, IF3, OF1, OF2, and OF3, respectively. The signals were collected under four loads for each health condition. Specifically, 100 signals were collected under each load with a sampling frequency of 12 kHz, and each signal had 1200 data points. FFT was applied to obtain the frequency spectra from signals. The first 600 spectral lines were chosen as a sample since the frequency spectrum was symmetric. On the whole, the rolling element bearing dataset included 4000 samples, and each sample included 600 data points. Detailed information on the rolling element bearing dataset is shown in Table 1.

Table 1.

Description of the rolling element bearing dataset.

Fault types	Severity level (mm)	Load (hp)	Sample number	Condition label
NC	—	0/1/2/3	100/100/100/100	NC
BF	0.18	0/1/2/3	100/100/100/100	BF1
	0.36	0/1/2/3	100/100/100/100	BF2
	0.53	0/1/2/3	100/100/100/100	BF3
IF	0.18	0/1/2/3	100/100/100/100	IF1
	0.36	0/1/2/3	100/100/100/100	IF2
	0.53	0/1/2/3	100/100/100/100	IF3
OF	0.18	0/1/2/3	100/100/100/100	OF1
	0.36	0/1/2/3	100/100/100/100	OF2
	0.53	0/1/2/3	100/100/100/100	OF3

In this case, the dimensions of the input layer, the feature layer, and the output layer were 600, 500, and 10, respectively. The regularization parameter $λ$ was set to $10^{- 5}$ .²¹ To achieve the average results, 20 trials were carried out for each experiment. 10% of the samples were randomly chosen for training, and the rest of the samples were used to validate the diagnostic performance of the VSF-BP method. The average accuracy and standard deviation of the VSF-BP method were 99.87% and 0.04%, respectively.

Figure 3 shows the effect of the percentage of samples for training on the average accuracy. Note that the average accuracy increased first and then decreased with the increase in the number of training samples, indicating that the developed method can achieve better performance using less training samples. The average accuracy got the highest value and a small standard deviation when 15% of the samples were applied for training. Note that the proposed method also obtained great diagnostic performance when 10% of the samples were used for training. To balance the diagnostic performance against the cost of collecting training samples in practice, 10% of the samples were selected for training.

Figure 3.

Average accuracy under different percentages of samples for training.

To demonstrate the importance of fine-tuning, the testing accuracy of 20 trials using the VSF and VSF-BP methods is shown in Figure 4. The average accuracy and standard deviation of the VSF method were 99.74% and 0.07%, respectively. Specifically, the testing accuracy varied from 99.58% to 99.89%, showing significant fluctuation. Note that the testing accuracy obtained by the VSF-BP method varied from 99.78% to 99.94%, showing better stability. Therefore, the fine-tuning process that can be achieved by the BP algorithm plays an important role in improving the diagnostic performance of the VSF-BP method. Figure 4 also shows an accuracy comparison between the proposed VSF-BP method and existing methods. It can be seen that the testing accuracy obtained by DNN²⁶ varied from 98.9% to 99.91%. The testing accuracy obtained by CNN²⁷ varied from 99.69% to 99.93%. The testing accuracy obtained by CNN²⁸ varied from 99.3% to 99.86%. The comparison shows that the existing methods are less stable in testing accuracy than the proposed methods.

Figure 4.

Testing accuracy of 20 trials using the rolling element bearing dataset.

The effect of fine-tuning on the weight vector of the VSF model is shown in Figure 5. Note that the weight vector varied little after fine-tuning, demonstrating the weights of the VSF model were fine-tuned. Thus, the fine-tuning can further optimize the network parameters of the VSF model, which can improve the diagnostic performance.

Figure 5.

Weight vectors obtained by the VSF-BP method before and after fine-tuning.

The confusion matrix of the rolling element bearing dataset using the VSF-BP method is shown in Figure 6. It can be seen that the average accuracy of the health conditions NC, BF1, IF1, IF2, and IF3 was 100%. Note that the average accuracy corresponding to the health conditions of OF1, OF2, and OF3 was almost 100%. Besides, the VSF-BP method can also recognize the health conditions of BF2 and BF3 with high accuracy. Specifically, 0.24% of the testing samples belonging to BF2 were misdiagnosed as BF1 and IF3. For the health condition of BF3, 0.43% of the testing samples were misdiagnosed as BF1, while 0.6% of the samples were misdiagnosed as BF2. It can be found that the VSF-BP method shows weaker performance in diagnosing BF. The main reason for misdiagnosis is the similarity between the signals of BF1, BF2, and BF3. On the whole, the proposed method can recognize the rolling element bearing faults effectively.

Figure 6.

Confusion matrix of the rolling element bearing dataset using the VSF-BP method.

To demonstrate the effectiveness of the proposed method, Table 2 shows the comparison of the diagnostic results of the VSF-BP method and existing data-driven methods. Specifically, sparse filtering obtained 99.7% accuracy, which was lower than that of VSF. Jia et al.²⁶ presented a deep neural network (DNN) based on SAEs, and 99.68% accuracy with a standard deviation of 0.22% was achieved. Wen et al.²⁷ developed a fault diagnosis method based on a CNN, and 99.79% accuracy with a standard deviation of 0.08% was realized. Yan et al.²⁸ presented a fault diagnosis method based on a novel DBN, and 99.7% accuracy with a standard deviation of 0.15% was achieved. Among these methods, the proposed VSF-BP method obtained the highest diagnostic accuracy using fewer training samples. Moreover, the VSF-BP method also achieved the lowest standard deviation in accuracy, which means the developed method possesses strong stability in fault diagnosis.

Table 2.

Diagnostic results of the VSF-BP method and existing methods using rolling element bearing dataset.

Methods	No. of training samples (%)	Average accuracy (%)	Standard deviation (%)
Sparse filtering	10	99.7	0.08
VSF	10	99.74	0.07
VSF-BP	10	99.87	0.04
DNN²⁶	50	99.68	0.22
CNN²⁷	83.3	99.79	0.08
DBN²⁸	50	99.7	0.15

Case 2: Planetary gearbox fault diagnosis

A planetary gearbox dataset provided by Nanjing University of Aeronautics and Astronautics²⁹ was used to validate the VSF-BP method. The test bench was mainly composed of a motor, a planetary gearbox, couplings, bearing blocks, and rotary tables, as shown in Figure 7. Seven kinds of planetary gear conditions were deliberately set up, including a normal health condition (NC), worn teeth (WT), a broken tooth (BT), a crack in the planetary gear (CG), pitting on the planetary gear (PG), a compound fault of WT and BT (CF1), and a compound fault of CG and PG (CF2), as shown in Figure 8. The time-domain vibration acceleration signals were collected from the surface of the gearbox. Two hundred signals were obtained for each health condition, and each signal had 1280 data points. The sampling frequency was 12.8 kHz. A sample with 640 spectrum lines was obtained from a signal by applying FFT. Thus, the planetary gearbox dataset contained 1400 samples.

Figure 7.

Test bench of the planetary gearbox.

Figure 8.

Planetary gears with different health conditions: (a) NC, (b) WT, (c) BT, (d) CG, and (e) PG.

In this case, the dimensions of the input layer, the feature layer, and the output layer were 640, 500, and 7, respectively. The regularization parameter $λ$ was set to $10^{- 5}$ . 10% of the samples were randomly chosen for training, and the other samples were used to validate the diagnostic performance of the VSF-BP method. The average accuracy and standard deviation of the VSF-BP method were 99.95% and 0.08%, respectively.

The testing accuracy of 20 trials using the VSF and VSF-BP methods is shown in Figure 9. The average accuracy and standard deviation of the VSF method were 97.55% and 1.37%, respectively. The testing accuracy varied from 92.22% to 98.81%, showing significant fluctuation. On the contrary, the testing accuracy of the VSF-BP method varied from 99.68% to 100%, showing better stability. Note that the VSF-BP method achieved 100% accuracy in 13 trials. Therefore, the BP algorithm plays an important role in improving the diagnostic performance of the VSF-BP method.

Figure 9.

Testing accuracy of 20 trials of the planetary gearbox dataset using the VSF-BP method.

The confusion matrix of the planetary gearbox dataset using the VSF-BP method is shown in Figure 10. Note that the average accuracy of the health conditions of NC, WT, CG, and PG was 100%. The average accuracy corresponding to the health condition of CF2 was almost 100%. The VSF-BP method obtained lower accuracy for the health conditions of BT and CF1. Specifically, 0.17% of the testing samples belonging to BT were misdiagnosed as WT, and 0.14% of the testing samples belonging to CF1 were also misdiagnosed as WT. On the whole, the VSF-BP method can recognize the planetary gearbox faults with high accuracy.

Figure 10.

Confusion matrix of the planetary gearbox dataset using the VSF-BP method.

The effect of fine-tuning on the weight vector of the VSF model using the planetary gearbox dataset is shown in Figure 11. It can be observed that there is no significant difference between the weight vectors before and after fine-tuning, indicating the weights of the VSF model were fine-tuned. The results demonstrate the necessity of fine-tuning for the VSF-BP method again.

Figure 11.

Weight vectors obtained by the VSF-BP method before and after fine-tuning.

Two-dimensional features were obtained by applying t-distributed stochastic neighbor embedding (t-SNE),³⁰ and the results are shown in Figure 12. Note that the VSF-BP method without fine-tuning can divide the learned features into seven clusters. Most of the features belonging to the same health condition were classified into the same cluster. However, some features that belong to the health condition of WT were misclassified into the cluster of BT. For the VSF-BP method with fine-tuning, all the features belonging to the same health condition were classified into the same cluster, as shown in Figure 12(b). The results indicate that fine-tuning can improve the feature learning capability of the VSF-BP method.

Figure 12.

Two-dimensional feature visualization of the planetary gearbox dataset: (a) before fine-tuning and (b) after fine-tuning.

To further demonstrate the effectiveness of the VSF-BP method, Table 3 shows the diagnostic results of the VSF-BP method and existing data-driven methods. Specifically, sparse filtering obtained 95.06% accuracy with a standard deviation of 5.62%, while VSF obtained 97.55% accuracy with a standard deviation of 1.37%. Wang et al.³¹ presented a diagnosis method by combining a Wasserstein generative adversarial network (WGAN) with SAEs, and 99.71% accuracy with a standard deviation of 0.16% was achieved. Zhang et al.³² developed a diagnosis method by combining fast intrinsic component filtering (FICF) with pseudo-normalization (PN), and 99.8% accuracy with a standard deviation of 0.12% was achieved. Note that the proposed VSF-BP method achieved the highest accuracy and the lowest standard deviation, showing high effectiveness and strong stability.

Table 3.

Diagnostic results of the VSF-BP method and existing methods using planetary gearbox dataset.

Methods	No. of health conditions	No. of training samples (%)	Average accuracy (%)	Standard deviation (%)
Sparse filtering	7	10	95.06	5.62
VSF	7	10	97.55	1.37
VSF-BP	7	10	99.95	0.08
WGAN-SAEs³¹	10	50	99.71	0.16
FICF + PN³²	4	10	99.8	0.12

Conclusions

A three-layer neural network method based on VSF and BP algorithm was presented in this paper to improve the diagnostic performance in rotating machinery. The performance of the VSF-BP method was verified by rolling bearing and planetary gearbox experiments. It is indicated that the application of l₁-normalization and l₃-normalization can help the VSF model learn useful features. The results show that the application of the BP algorithm can optimize the weight matrix of the VSF model and improve the feature learning ability, which can improve diagnostic accuracy. By Comparing with the existing diagnosis methods, the proposed VSF-BP method can obtain higher accuracy and lower standard deviation using fewer training samples, showing high effectiveness and good stability.

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported in part by the Natural Science Foundation of the Jiangsu Higher Education Institutions of China (Grant No. 18KJD460003) and the Natural Science Research Foundation of Jiangsu Normal University (Grant No. 18XLRS009).

ORCID iD

Chun Cheng

References

Flah

Nunez

Chaabene

, et al. Machine learning algorithms in civil structural health monitoring: a systematic review. Arch Comput Methods Eng 2020; 28, DOI: 10.1007/s11831-020-09471-9.

Wei