Review of rolling bearings performance degradation trend prediction

Methodology for the construction of physical degradation indicators

Physical degradation indicators construction methods contain time-domain correlation, frequency-domain correlation, entropy, and so on. Time domain analysis or entropy correlation algorithms were mainly used for indicator construction in most of the studies using traditional methods. From the monitoring signals collected by sensors, a class of indicators is extracted using statistical methods or signal processing methods, such as root mean square (RMS), spectral kurtosis, information entropy (IE), and so on. Time domain vibration signals, as the original waveforms collected, contain the most complete and comprehensive information content. Its statistical characteristics (peak value, RMS, kurtosis, etc.) are often used to reveal the degradation process of bearings performance. ⁵

Time domain analysis is the classic method for constructing performance degradation indicators. Time domain analysis is based on the vibration signal after time sorting for the corresponding description, usually calculating the signal reflecting the rolling bearings operation state of various feature values. It includes time domain features, frequency domain features, IE, time-frequency domain features, and so on. Time domain features include RMS, kurtosis, maximum value, peak-to-peak value, skewness, etc.; frequency domain features include the Centroid Frequency, mean square frequency, spectral coefficient of variation, etc.; IE includes wavelet packet energy entropy, amplitude spectral entropy, singular value spectral entropy and so on. Since time-domain signals can be relatively easily collected by sensors, the time-domain feature RMS and kurtosis are often used as rolling bearing performance degradation indicators in the study of bearing degradation prediction and life prediction. These indicators characterize the overall operating condition of the rolling bearing, which in turn predicts the life and performance degradation trend of the bearing. Among them, the RMS of the time domain signal describes the signal strength, which can reflect the overall damage to mechanical equipment. It is used to characterize the process of degradation of the performance of mechanical equipment because it is more sensitive to the degree of failure development of the mechanical components. Zhang et al. ⁶ take the difference between the RMS of the residual series obtained from the autoregressive model and the multivariate state estimation technique as an indicator of the degree of performance degradation. Zhang et al ⁷ used RMS, skewness, and Kurtosis for the construction of health indicators of degradation trends in the prediction model. Frequency domain analysis obtains frequencies reflecting the degradation characteristics of rolling bearings from time domain signals by using Fourier transform decomposition. Wu et al. ⁸ used the moment-centered value of power spectral density as the performance degradation characteristic. Meng et al. ⁹ obtained indicators for the monitoring indicator of the initial fault and the monitoring indicator of the degradation stages through the cumulative sum of coefficients of variation. Time-frequency domain analysis method combines the time domain and frequency domain information of the signal which is remarkably effective for nonlinear and nonsmoothed vibration signals. Wang et al. ¹⁰ used the stack autoencoder (SAE) method to integrate the time domain indicators and frequency domain indicators to construct the performance degradation feature indicators. Zhu et al. ¹¹ used time-frequency domain features and other features to construct a comprehensive set of bearing degradation features. Although the above methods can better reflect the degradation characteristics, all of them have poor monotonicity and the early failure characteristics are not obvious.

Entropy-based nonlinear dynamics theory is a powerful tool, which has been heavily used in the degradation feature extraction of rolling bearings in recent years. Various entropy-related algorithms are being proposed. Entropy is a method to characterize the complexity of vibration signals with explicit physical meaning. ¹² The entropy value is sensitive to distinguish between vibration signals and background noise. ¹³ The entropy value is high at the beginning of the bearing operation due to the random nature of the acquired signals; when the faults are more serious, the impact signal will cover part of the interference noise, which leads to the corresponding lower entropy value. The permutation entropy (PE) is not sensitive to the amplitude of vibration signals, and thus cannot accurately reflect the energy distribution of vibration signals in different frequency bands. Zhang et al. ¹⁴ proposed the Entropy Energy Ratio as a performance degradation assessment indicator based on the PE. Cheng et al. ¹⁵ proposed composite multiscale grey entropy (CMGE) with a weak dependence on the length of the acquired signals under the premise of effectively describing the degradation trend of rolling bearings. In addition, when constructing indicators, researchers have used entropy-related methods such as relative sample entropy, ¹⁶ fusion gray entropy, ¹⁷ relative energy entropy, ¹⁸ modal decomposition-transfer entropy, ¹⁹ multivariate multi-scale fuzzy entropy, ²⁰ probability distribution correlation, ²¹ fusion composite indicators, ²² and the entropy weight method, ²³ and so on.

Although the entropy-related method has achieved some results in rolling bearings degradation prediction, there are still several problems that need to be solved. The calculation results are inconsistent with the real nonlinear dynamical system and thus lead to errors; the calculation results are inconsistent in different scales; the length of the data required for calculation is long. Therefore, the use of virtual degradation indicators can solve these problems to a certain extent.

Methodology for the construction of virtual degradation indicators

Virtual degradation indicator refers to a class of indicators that characterize the operation state of mechanical equipment obtained by fusing multiple physical degradation indicators or sensor data. At present, there are two main types of ideas for virtual health factor construction methods: data fusion-based ideas and similarity-based ideas. ²⁴ The research based on the idea of data fusion mainly includes principal component analysis (PCA), kernel principal component analysis (KPCA), local linear embedding (LLE), restricted Boltzmann machine (RBM), etc. The principle is to obtain a new virtual degradation indicator after fusion and dimensionality reduction of multiple physical degradation indicators or sensor data.

PCA has been frequently mentioned in recent years as a powerful technique for extracting intrinsic structures from high-dimensional datasets. By extracting the main features of the original spatial data, the data is processed in a low-dimensional feature space, removing noise and redundancy from the data. The dimensionality reduction of the original data is achieved while maintaining a large amount of the original information. Thus, the PCA method can achieve a small number of irrelevant composite features in place of the original high-dimensional features and most of the information of the original variables is contained in these composite features. A few studies in recent years have used it to deal with feature variables. Currently, the use of PCA for dimensionality reduction^25–29 is one of the main uses, which helps to clearly show the changes in faults over time. Other researchers have used PCA for the fusion of feature indicators,^30–34 which helps to fully utilize the various features for predicting the degradation trend. Other researchers have used it for sensitive characterization, ³⁵ analyzing failure data, ³⁶ multi-dividing life evolution intervals, ³⁷ correcting prediction results, ³⁸ and so on.

Since the PCA is a linear technique, it cannot deal with complex nonlinear structures between the data. Therefore, nonlinear generalization has been proposed, i.e., introducing kernel methods into PCA. The resulting method is called KPCA, which is used to calculate the principal components of a nonlinear mapping to a high-dimensional feature dataset. The implementation process of KPCA is as follows. The original data are transformed into a new space by nonlinear mapping. Then PCA is performed in the new space and low dimensional features of the samples are extracted. Kang et al. ³⁹ used KPCA to remove the redundancy of information between the filtered features, to construct an approximated feature matrix for input to support vector regression (SVR). Akpudo et al. ⁴⁰ proposed the use of the KPCA feature fusion technique for degradation assessment and a deep learning model for prognostic assessment. Cao et al. ⁴¹ applied KPCA to construct indicators and state classification. Guo et al. ⁴² proposed a hybrid use of Adaptive Noise and KPCA. In practical applications, empirical errors arise because of the need to manually select kernel functions.

LLE is an unsupervised dimensionality reduction method. Compared with the PCA dimensionality reduction method, the low-dimensional data after LLE dimensionality reduction has a similar neighborhood structure as the original data. The idea of dimensionality reduction in LLE is to assume that the data is linear in the local range. Each sample point can be expressed linearly in terms of neighboring points, thus not changing the structural information of the original sample set, i.e. not changing the corresponding sample weights. For example, when the initial sample set is a spiral-shaped nonlinear structure, PCA dimensionality reduction will make the original structure greatly changed. Linear dimensionality reduction cannot express such nonlinear results as spirals, so LLE can be used to better fit such original sample sets without losing the essential data structure. In terms of dimensionality reduction algorithms for LLE, Xin et al. ⁴³ used an LLE dimensionality reduction algorithm to extract features. Wang et al. ⁴⁴ combined the LLE nonlinear feature reduction method with fuzzy C-means. Liu et al. ⁴⁵ incorporated the neighborhood parameter adaptive selection feature into the LLE algorithm for extracting the performance degradation features. In addition to these studies, LLE is also used for data fusion on high-dimensional data^46,47 and so on. Researchers mostly use the LLE method in data dimensionality reduction, which can effectively reflect the reliability and stability of the original state performance.

The locality preserving projections (LPP) algorithm is an unsupervised dimensionality reduction algorithm commonly used for high-dimensional data. Its purpose is to eliminate redundant information in high-dimensional data. By reconstructing the axes of one, the original data points are projected onto the new axes, i.e., similar samples in the high-dimensional space can be as close as possible to the low-dimensional space. The measure of the practical effectiveness of the method is how to find the optimal axes that maintain the original important features and increase efficiency. Zhang et al. ⁴⁸ used LPP to preserve the local structure of the data streaming pattern and then established negative log-likelihood probabilities based on a continuous hidden Markov model (CHMM) as a new performance degradation indicator. Peng et al. ⁴⁹ fused feature vectors to features by LPP. Li et al. ⁵⁰ used LPP to reduce the dimensionality of the general feature set while preserving the main information. The negative log-likelihood probability based on the two-dimensional hidden Markov model (HMM) is constructed as a new degradation index. The LPP typically fuses useful information and reduces the dimensionality.

RBM is a probabilistic graphical model with a network structure consisting of two layers, a visible layer and a hidden layer. The visible layer is used to describe the inputs to the neural network. The hidden layer is used to describe the dependencies (non-linear) between the input data of the visible layer. For feature fusion and dimensionality reduction. The restricted Boltzmann machine uses the training data to adjust the node connection weights and bias parameters so that the probability distribution represented by the corresponding restricted Boltzmann machine is as similar as possible to the training data. Cheng et al. ⁵¹ normalized magnitude spectra are used as training samples to build RBM, and the output is used as the basis for the construction of degradation indicators. Sun et al. ⁵² organically integrated a rolling bearing performance degradation mechanism model into a constrained RBM term in order to integrate multiple types of physical health factors and thus improve factor performance. Zhang et al. ⁵³ obtained the virtual health indicator through restricted RBM. Liu et al. ⁵⁴ Combined RBM and convolutional layers with feature processing for fault identification. As with PCA, monotonicity, trend, and robustness are neglected because it is mainly used for fusion and dimensionality reduction.

Methods based on the similarity idea include self-organizing map (SOM), Mahalanobis distance (MD), support vector data description (SVDD), etc. The historical monitoring data are input into the model for training, and then test data are input into the trained model. The virtual health factors of the test data were obtained and finally processed into virtual degradation indicators. The basic assumption of the idea is to calculate the degree of deviation of the monitoring characteristics of the performance degradation phase from the normal phase. It is positively correlated with the degree of performance degradation.

MD refers to the covariance distance of the data with good affine invariance. The MD was improved to address the insufficient matching problem of the Euclidean distance. Zheng et al. ⁵⁵ calculated the MDs in the feature vectors and used these MDs as the bearing performance degradation indicators, which were fed into the gated recurrent unit model for predicting the degradation trend. Dong et al. ⁵⁶ used the deep autoencoder (DAE) and t-distribution stochastic neighbor embedding (t-SNE) models for extracting, expressing, and approximating the performance degradation features of the bearing signals. Then the MD in low-dimensional feature space as a performance degradation indicator. In addition to constructing indicators using MD alone, Yang et al. ⁵⁷ combined the kernel-independent component analysis with the MD to calculate the degradation indicator. Zhang et al. ⁵⁸ used MD successively for fault identification and performance degradation assessment indicators construction, which effectively tracked the performance degradation. MD has also been used as indicators to construct B-spline weight health indicators (BSWHI), ⁵⁹ to evaluate model classification performance, ⁶⁰ etc.

The main idea of SVDD is to find an optimal hypersphere that contains the minimum volume of all or most of the targets. For a set of target class samples, find the optimal hypersphere that contains all or most of the target class samples. The training samples are then mapped from a low-dimensional space to a high-dimensional space by a nonlinear transformation, which provides a better description of the target samples in the new high-dimensional space. Cheng et al. ⁶¹ used the feature vectors in a normal state extracted from the unsupervised pretraining part of the deep belief networks model as training samples to build an SVDD evaluation model and obtain performance degradation indicators for rolling bearings. Pan et al. ⁶² fused features into performance indicators (PI) using SVDD that require appropriate penalty parameters C and kernel parameters. The Sparrow Search Algorithm (SSA) is introduced into the parameter optimization process of the SVDD, which is then combined with DBN for detecting the early degradation of the bearing. Wang et al. ⁶³ used SVDD to adaptively process degradation indicators. Other improved SVDD methods are the Fruit Fly Optimization Algorithm-Wavelet Support Vector Data Description (FOA-WSVDD) model, ⁶⁴ Improved Whale Particle Swarm Support Vector Machine Based (HWPSO-SVDD) model, ⁶⁵ hypersphere radius of multikernel support vector data description (MKSVDD), ⁶⁶ and combination of fuzzy support vector data description (FSVDD) and running time, ⁶⁷ multi-strategy optimized support vector data description ⁶⁸ and so on. All the above improvements are based on the way SVDD is better than the original model, and the anti-noise performance is partially optimized.

SOM is an important class of unsupervised learning methods. It updates the connection weights between neurons based on the activated neurons, thus dividing the pattern of the input space based on the similarity between the input samples. Its main function is to map the high-dimensional spatial structure of the data to the low-dimensional space. Luo et al. ⁶⁹ used the continuous wavelet transform (CWT) and the convolutional autoencoder (CAE) for the feature extraction method and then applied the SOM approach to get the Degradation Indicator (DI), and finally achieved the improvement of prediction accuracy. To avoid the high subjectivity of the screening rules for degradation features, Liu et al. ²⁶ used the SOM method to cluster the statistical features and then obtained the fused degradation feature set by dimensionality reduction through the PCA method. Cheng et al. ⁷⁰ adaptively fused the time-domain statistical features, spectral features, intrinsic mode function energy features, and wavelet packet frequency band energy features of health indices using the multiple growing self-organizing mapping (GSOM). Lv et al. ⁷¹ used leveraging multi-domain feature extraction and SOM networks to construct initial health indicators.

However, the similarity-based methods need to use a large amount of historical monitoring data for model training, and the small sample size of monitoring data does not apply to this idea.

Indicator construction methods based on deep learning

In recent years, deep learning methods have been developing rapidly. Relying on its ability to have powerful feature self-extraction and non-linear function characterization, it provides a way of thinking for constructing bearing degradation indicators that are different from the traditional construction methods. Deep learning to construct indicators is through various types of deep learning methods on the original acquisition signal self-learning to construct performance degradation indicators, which contains unsupervised learning and supervised learning to construct performance degradation indicators. Deep learning methods include convolutional neural networks (CNN), recurrent neural networks (RNN), auto-encoders (AE), deep belief networks (DBN), and so on. Unsupervised learning is mainly used for network pre-training, dimensionality reduction, etc. Supervised learning, on the other hand, utilizes data labels to deeply mine the feature information inside the data. Supervised deep learning algorithms mainly include CNN and RNN. Unsupervised deep learning algorithms mainly include AE and DBN.

CNN is one of the most widely used research hotspots in the field of deep learning, which has three major advantages: local connectivity, weight sharing, and spatial pooling. The typical structure of a CNN includes a convolutional layer, a fully connected layer, an input layer, an output layer, and a pooling layer. In this paper, the working principal diagram of one-dimensional convolutional neural network (1D CNN) is given as shown in Figure 1. The Convolutional layer is the core of the network structure of a CNN, the main role is to achieve the dimensionality reduction of the data processing and can be used to automatically extract the characteristics of the original data. The pooling layer further reduces the size of the data by spatially sampling the dimensions of the input data and makes the input data locally linearly transform invariant. After the convolutional and pooling layers are processed, the fully connected layer nonlinearly combines the feature information that has been extracted thus obtaining the output. CNN is commonly used for feature extraction, Jiang et al. ⁷² used CNN to extract the low-dimensional degradation features of rolling bearings and further extracted its high-dimensional degradation features for prediction. Yang et al. ⁷³ proposed a method to improve the construction of one-dimensional deep CNN bearings performance degradation indicators based on a combined loss function, which was experimented on both publicly available and real-world datasets to verify its effectiveness and feasibility. Improvements to CNNs include mining deep features reflecting degradation performance, ⁷⁴ automatic extraction of local abstract information from data, ⁷⁵ and hybrid dimensional convolutional neural network with bi-directional long short-term memory (DCNN-BiLSTM) input networks. ⁷⁶ OPEN IN VIEWER

Traditional machine learning models suffer from the problem of memory deficit, so RNN models are proposed for processing sequence data. RNN integrates the structure of feedback connections and possesses the function of long-term memory. A major RNN structure unfolding is given below as shown in Figure 2. Ning et al. ⁷⁷ used the Mean Impact Value (MIV) algorithm for feature selection, realized secondary feature selection by correlation coefficient identification calculation, and finally obtained bearing degradation curves by RNN. Cheng et al. ⁷⁸ proposed the use of autoencoder quasi-recurrent neural networks (AEQRNN) containing a convolutional component that can process the input data in parallel, and a pooling component with two LSTM-like gate structures to process time series data. The model automatically extracts hidden features from the monitored signals. Upon its experimental results, the method is shown to have superior prediction performance and training speed compared to various related methods. Li et al. ⁷⁹ used reinforcement learning to select the hidden number of layers and the number of hidden layer nodes fitted to the RNN of the corresponding monotone trend unit. Then the nonlinear approximation ability and generalization performance of the enhanced reinforcement learning unit matched recurrent neural network (RLUMRNN). Although the high prediction accuracy and computational efficiency of RNN-related methods are reflected in all the above methods, traditional RNNs cannot deal with the problem of long-term dependence on time series, as it leads to gradient explosion or gradient disappearance.OPEN IN VIEWER

AE is an unsupervised generative network. It can automatically extract effective features from a large amount of input data and is commonly used for dimensionality reduction processing or feature learning of data. The working principal diagram of the AE is shown in Figure 3. The traditional AE uses the full connection between hidden units. The difference is that the convolutional auto-encode (CAE) model replaces the matrix multiplication operation between hidden neurons in the AE with the convolutional operation and pooling operation. Where the convolutional layer acts as a filter and the pooling layer is used to extract invariant features. The AE acts similarly to PCA, but the AE can characterize nonlinear transformations and is suitable for situations where nonlinear data are handled. To better handle spatial information of 2D data, researchers have proposed convolutional auto-encoders, which use convolutional and pooling layers instead of the traditional fully connected layers. Chen et al. ⁸⁰ used a CAE to adaptively extract a coded feature set from rolling bearings frequency domain signals, then selected high-quality features to construct health indicators. In addition to being used to extract features, Li et al. ⁸¹ used time-domain frequency-domain features to construct high-dimensional feature vectors and constructed degraded feature indicators by CAE dimensionality reduction. To effectively improve the evaluation indicators, Qin et al. ⁸² proposed a deep convolutional auto-encoder (DCAE) network. A stacked auto-encoder can effectively extract data features, and adding sparse constraints to it can effectively reduce the training overfitting problem. Therefore, Yuan et al. ⁸³ performed deep feature learning and feature compression to construct degenerate feature vectors by using a stacked sparse auto-encoder. Xu et al. ⁸⁴ proposed to use of a deep stacked denoising auto-encoder accumulator to obtain the vibration accumulation sequence values, and finally obtained the cumulative health indicators by normalizing the vibration accumulation sequence values. Wang et al. ⁸⁵ used a single-layer Nonnegativity Constrained Autoencoder (NCAE) network with the SOM to construct health indicators. Due to the fast training speed and high generalization ability of extreme learning, Lan et al. ⁸⁶ used samples of bearings health states to train an extreme learning machine auto-encoder, using the loss between reconstructed and original data as indicators. Xu et al. ⁸⁷ used a stacked noise auto-encoder with multiple hidden layers for noise cancellation in HI. Lu et al. ⁸⁸ proposed a variance variation and SAE feature dimensionality reduction method for redundancy elimination.OPEN IN VIEWER

The DBN is a multi-hidden layer neural network consisting of multiple RBM units. Its structure and classification diagram are shown in Figure 4. It can be directly inputted into the original signal for processing, and it has a stronger nonlinear mapping ability than shallow networks such as Markov, support vector machine, logistic regression, and so on. By carrying out feature extraction more fully, it avoids uncertainty in the extraction process and achieves intelligence in the construction of recession indicators. Pan et al. ⁶² used the support vector to describe the SVDD to merge the sample features of the DBN into the performance indicators. To improve the monotonicity of the extracted parameter-free health indicators, Xu et al. ⁸⁹ used the DBN to extract sigmoid zero local minima of the health indicator curves. Then the sigmoid-based function was used to reduce the weights of the current health indicator values to achieve the elimination of oscillations. To solve the continuous deep belief network will fail to stabilize the convergence of the training process, Zhou et al. ⁹⁰ used an improved adaptive continuous deep belief network to extract the hidden fault features, and the distance between the initial operating condition state and the real-time degradation state is used to construct the indicators. The standard CDBN is composed of multiple convolutionally constrained Boltzmann machines, while Yuan et al. ⁹¹ constructed a new Gaussian matrix-variable convolutional deep belief network model by using Gaussian visual units, and performance degradation indicators are automatically constructed for the transformed time-frequency information. Wang et al. ⁹² combined DBN with SOM to process the initial signal into an effective synthetic HI.OPEN IN VIEWER

In this section of the paper, the methodology for constructing performance degradation indicators for rolling bearings has been discussed. Table 1 summarizes the various bearing indicators and construction methods based on types of bearings and datasets.

OPEN IN VIEWER

Table 1.

Summary of the Indicators and methods of construction based on different types of bearings and datasets.

Types of bearing	Datasets	Indicators and methods of construction
ZA–2115 double-row roller bearings	University of Cincinnati	RMS of the residual series, 6 moment-centered value of power spectral density , 8 cumulative sum of coefficients of variation, 9 integration of indicators using SAE, 10 Entropy Energy Ratio, 14 multivariate multi-scale fuzzy entropy, 20 probability distribution correlation, 21 correcting forecasts using PCA, 38 KPCA feature fusion technique, 40 extraction of main features using LLE, 43 converting fused feature vectors to features using LPP, 49 using the MD in the feature vector as indicators, 55 performance degradation assessment indicators construction using MD, 58 fusion of features to PI using SVDD, 62 RLUMRNN, 79 stacked noise auto-encoder 87
	NASA	CHMM, 48 BSWHI, 59 AEQRNN 78
Roller bearings	IEEE PHM 2012	Indicators constructed using RMS, skewness and kurtosis, 7 dimensionality reduction using PCA, 25 using PCA to construct indicators and state classification, 41 restricted RBM, 53 multi-strategy optimized support vector data description, 68 applying the SOM approach to get the DI, 69 leveraging multi-domain feature extraction and SOM networks, 71 stacked noise auto-encoder, 87 an improved adaptive continuous deep belief network 90
LDK UER204 rolling bearings	XJTU-SY	Indicators constructed using RMS, skewness and kurtosis, 7 integration of indicators using SAE, 10 CMGE, 15 use of PCA fusion feature indicators, 30 using PCA to construct indicators and state classification, 41 performance degradation assessment indicators construction using MD, 58 adaptive treatment of degradation indicators using SVDD, 63 applying the SOM approach to get the DI, 69 an extreme learning machine auto-encoder, 86 an improved adaptive continuous deep belief network 90

Traditional performance degradation prediction models

Traditional performance degradation models can be categorized as time series analysis, artificial neural networks (ANN), support vector machines (SVM), probability distribution models, and so on.

Time series analysis is an analytical model based on the principle of statistical analysis. Its main idea is to summarize the trend through the historical collection of data and then predict the future trend of the data. It is characterized by the randomness in the values of each data, the correlation of the data before and after the sequence, and the ups and downs of the data itself. The process of time series analysis is clear, while the neural network process is a black box. This paper summarizes the following three smooth time series models, Autoregressive Model (AR), Moving Average Model (MA), and Auto Regression Moving Average Model (ARMA). Zhou et al. ⁹⁴ used the AR to extract the coefficients and residuals of the wavelet packet nodes, which were used as combined features. The measured data are interpolated into the fuzzy C-mean model and the Hidden Markov Model, and then the combined features are input into the fuzzy C-mean model, and finally, the performance degradation curve is obtained. Zhuang et al. ⁹⁵ designed a deep autoregressive recurrent model that can consider the degradation process more comprehensively in conjunction with a statistical model. Zhuang et al. ⁹⁶ proposed an improved autoregressive integrated moving average model and used the Bayesian information criterion to determine the order of the model, which was ultimately used to predict the degradation trend of health indicators. Wu et al ⁹⁷ combined the MD with the Exponentially Weighted Moving Average (EWMA) method to reduce manual dependency and improve adaptability and generalization. The combined results are used to characterize the degradation trend of the acquisition performance, and finally, the performance prediction is performed using a fusion model of Sparse auto-encoder and gated recurrent unit (SAE-GRU). Zhao et al. ⁹⁸ used moving average coarse-grained, fast iterative filtering decomposition, PE, Wasserstein distance, and cumulative sum method to construct bearings degradation state indicators to realize the identification and evaluation of bearings degradation state. The time series analysis method is characterized by its simplicity and ease of use. However its accuracy is poor, and there are now many other prediction models to compensate for its shortcomings.

The ANN is a dynamic system built manually with a directed graph as its topology. Due to the nonlinearity and instability of vibration signals of mechanical equipment, the neural network prediction methods are very suitable for trend prediction of mechanical equipment. The ANN is a traditional neural network whose structure usually consists of an input layer, a hidden layer, and an output layer. By optimizing the implicit layer structure, it can fit any arbitrary function with any accuracy better, which is suitable for the construction of nonlinear models with high accuracy requirements. Therefore, artificial neural networks are superior to time series analysis. ANN can fully approximate any complex nonlinear relationship, and all quantitative or qualitative information isotopically distributed and stored in each neuron of the network. Therefore, ANNs are robust and fault-tolerant, which makes it possible to perform a large number of operations quickly and handle both quantitative and qualitative knowledge. ANN mainly simulates the working process of the human brain by connecting nodes in their structure and can automatically extract features from monitoring data. Wang et al. ⁹⁹ used the framework of the ARMA-ANN hybrid prediction model to predict the IMF sequences and then selected the sensitive modes containing essential information to characterize the state of the machine. However, ANN has the disadvantage of huge training parameters, which leads to a relatively long training time.

The SVM was formally proposed by Cortes & Vapnik in 1995 as an implementation of statistical learning theory, which better implements the idea of structural risk minimization. ¹⁰⁰ To cope with the difficulties of slow training speed and large storage capacity of SVM, researchers introduced the least squares support vector machine to transform solving the quadratic programming problem into solving the linear least squares problem in high-dimensional space, which simplifies the computational process and improves the training speed. SVM inherits several techniques such as relaxation variables to better solve the practical problems of high dimensionality, nonlinearity, small samples, over-learning, local minima, etc. It has come to a new research hotspot of machine learning after neural network research. Gao et al. ¹⁰¹ proposed the use of the gray model for feature value prediction and then trained a binary tree support vector machine for degradation trend prediction. Chen et al. ¹⁰² proposed degradation trend prediction based on IE, fuzzy granularity, and least squares support vector machines. Kumar et al. ¹⁰³ applied Kernel Ridge Regression (KRR) and support vector regression methods on Dynamic Time Warping (DTW) algorithm features to predict bearings degradation trend. Niu et al. ¹⁰⁴ conducted comparative experiments of four schemes, DSDAE with TDSVR, MDCUSUM with TDSVR, DSDAE with TDLSSVM, and MDCUSUM and TDLSSVM, by using open datasets. The analysis of the prediction indicators concludes that the deep stacked denoising auto-encoder (DSDAE) and the time-lagged least squares support vector machine (TDLSSVM) schemes can better predict the trend of the rolling bearing health state. Chen et al. ¹⁰⁵ characterized the variable frequency signals by IE, and these constructed feature indicators were fed into a correlation vector machine for performance degradation prediction. Gao et al. ¹⁰⁶ used the isometric mapping (ISOMAP) algorithm to reduce the dimensionality of the high-dimensional feature set, and then put the comprehensive feature indicators through the logistic regression model to establish a reliability model. Finally, the nonhomogeneous cuckoo search-least squares support vector machine (NoCuSa-LSVM) model is used to predict the degraded state characteristics of bearing performance, and the results are embedded into the constructed reliability model, so as to obtain the prediction results of the operational reliability of the bearings. Zhang et al. ¹⁰⁷ proposed a time-shifted least-squares support vector machine with integral operators for predicting performance degradation. However, the least squares support vector machine model itself does not have the data memory function and cannot uncover the special information relationship between the time series data. Zan et al. ¹⁰⁸ extracted the features of the feature values and then used the joint approximate diagonalization of eigen-matrices (JADE) method to represent the fusion features of performance degradation. Then a prediction model based on particle swarm optimization support vector machine (PSO-SVM) is constructed to predict the performance degradation trend. The SVM is one of the current research hotspots, and its gradual improvement can play a positive role in performance degradation prediction.

Some of the probability distribution models have good noise immunity. Considering the uncertainty ignored by deterministic neural networks, the probability distribution model is therefore more practically applicable. Commonly used probability distribution models are Poisson distribution, maximum entropy, Weibull distribution, Dirichlet distribution, combinatorial distribution model, and so on. Xia et al. ¹⁰⁹ proposed a method to predict the performance degradation trend of rolling bearings based on the Poisson counting process. In addition to the separate Poisson distribution model, Xia et al. ¹¹⁰ proposed new concepts such as probability of change, speed of change, and acceleration of change. Then a reliability prediction model based on the maximum entropy principle and Poisson process was constructed to predict the change process of rolling bearing vibration performance reliability. Wang et al. ¹¹¹ developed the Lévy process model to cope with the problem that machine performance degradation exhibits nonlinearity and non-uniformity by including a non-uniform composite Poisson process term for capturing transient performance variations. By calculating the moments of the Lévy model eigenfunctions, explicit expressions for predicting performance degradation and RUL probability distributions can be derived. From experiments, it is known that it outperforms multiscale crack extension models and multistage modeling based on PF estimation. Zhang et al. ¹¹² proposed a type II-GLD position scale model based on the combination of type II generalized logistic distribution (II-GLD) and the position scale model. The model performs better than the Weibull model and normal distribution assessment, and the relative errors of mean and variance between the model and the actual data are less than 2% and 8%. Zhou et al. ¹¹³ regarded the prediction of the visual geometric group (VGG) neural network as a subjective opinion by placing the Dirichlet distribution over the category probabilities capable of estimating the prediction uncertainty and avoiding the overconfidence of out-of-distribution OOD data. In addition to these models, there is the Three-parameter Weibull (3-PW) distribution model, ¹¹⁴ the generalized polynomial chaos theory model, ¹¹⁵ and so on.

Deep learning performance degradation prediction models

Deep learning models include CNN models, recurrent neural network models, etc. At present, deep learning is one of the research hotspots. There are more kinds of deep learning algorithms, which can overcome most of their problems through algorithmic improvement and reduce the burden of researchers to process data manually in the prediction field. In recent years, deep learning algorithms have achieved fruitful results in fault feature extraction, RUL prediction, and performance degradation prediction of rolling bearings.

Predictive modeling based on recurrent neural network

Degradation of rolling bearings is a process of gradual accumulation and continuous development, and the changes in its state are closely related to the real-time monitoring data, as well as the monitoring data of historical moments. The RNN is a kind of memory network that can synthesize the memory information of the current and historical stages and make predictions, which effectively solves the problem that traditional neural networks cannot make full use of historical information data. The development of RNNs is relatively fast, and their application in prediction has been very extensive. However, to be more stable and to cope with the problem of “memory loss”, researchers nowadays prefer to use the improved recurrent neural network, i.e., the long short-term memory model. Long short-term memory networks can maintain and update long-term dependency information over multiple time steps, thus overcoming the limitations of standard RNNs in dealing with excessively long temporal information.

In long-term and short-term memory neural networks are trained using the Truncated Backpropagation Through Time (BPTT) algorithm and passed backward in the temporal dimension. The long short-term memory networks (LSTM) are trained to extract features from time-series data. It reflects the bearings degradation process in the time domain, and thus can better predict the time series beyond the current moment. Huang et al. ¹²³ proposed a fault prediction model constructed based on the LSTM model. The LSTM model is trained to optimize its learning rate and hidden layer by the PSO algorithm, and the fault trend of the rolling bearings to be tested is analyzed based on the fault trend prediction results output from the fault prediction model. Wang et al. ¹²⁴ proposed a bidirectional long short-term memory network prediction model for degradation trends using historical and future information. He et al. ¹²⁵ used Multi-Resolution Singular Value Decomposition (MRSVD) for decomposition and reconstruction and then predicted bearing performance degradation using LSTM. Hu et al. ¹²⁶ optimized the model parameters of the LSTM algorithm by improved particle swarm optimization (IPSO). Wang et al. ¹²⁷ proposed an attention mechanism combined with the LSTM model (Attention-LSTM) method to improve the assessment and prediction of rolling bearings performance degradation. Lan et al. ¹²⁸ proposed a self-checking Long Short-Term Memory (Sc-LSTM) prediction model for predicting bearings performance degradation trend, which introduced the prediction effect assessment of prediction pass rate η. In addition to the above literature on similar conditions, Liu et al. ¹²⁹ proposed a novel LSS model that combines the advantages of LSTM with statistical process analysis for predicting failures in multi-stage performance degradation of aero-engine bearings. Shi et al. ¹³⁰ proposed a Bayesian optimization (BO)-based initial failure prediction model, which consists of a sliding window stacked denoising autoencoder (SWDAE) and LSTM. Chen et al. ¹³¹ constructed a new performance degradation assessment model using signal significance index (SSI), deep cumulative features (DCF), and LSTM. Table 2 lists and summarizes recent LSTM-based prediction models.

OPEN IN VIEWER

Table 2.

Summary of recent LSTM-based bearing performance degradation models.

Research methods	References	Main ideas
Combination	Sahu et al. 132	Combining the absolute cumulative modified multiscale permutation entropy (C-MMPE) feature with LSTM and introducing a bearing virtual health degree.
	Song et al. 133	Combining secondary selection algorithm with Bayesian optimization of self-attention mechanisms for bidirectional long short-term memory (BSBiLSTM).
	Yang et al. 134	Integrating convolutional neural network (CNN), variational autoencoder (VAE), and multiple bi-directional long short-term memory (MBiLSTM).
	Yao et al. 135	Results obtained by bidirectional long short-term memory network (Bi-LSTM) after noise reduction using convolutional denoising autoencoder (CDAE).
	Yu et al. 136	Using an adaptive weighting matrix to improve the stack denoising autoencoder (SDAE) before feeding it into the LSTM.
	Zhang et al. 137	Before inputting into LSTM, the signal is processed using a fully adaptive ensemble empirical modal decomposition of noise (CEEMDAN) and the CNN extracts the features.
Improvement	Li et al. 138	Optimizing the hyperparameters of the Convolutional Long-Short Term Memory Neural Network (CNN-LSTM) using the artificial Gorilla Troops Optimizer (GTO) and thus reducing the mean square error.
	Lu et al. 139	Regularization terms are constructed based on changes in indicators and remaining lifetime consistency, resulting in LSTM networks with realistic physical laws.
	Wang et al. 140	The network parameters in the LSTM were tuned using the improved sparrow search algorithm (ISSA).

Due to the relative complexity of the structure of the LSTM, it will cost too much time in training and prediction. The researchers analyzed the structure of the LSTM, adjusted the structure of the network to achieve dynamic control of the time scale and the forgetting behaviors of different units, and then proposed the model of the gated recurrent unit (GRU). The structure of the GRU is shown in Figure 5. The main difference between the GRU and the LSTM is the use of two control units, the post-merger update gate and reset gate, to control the network forgetting and updating states. The GRU not only retains the LSTM effect but also makes its internal structure simpler and more scalable. The GRU models are often used in combination with AE in bearings degradation trend prediction. Wei et al. ¹⁴¹ proposed a convolutional noise reduction auto-encoder that contains the GRU. The encoding unit consists of a noise addition layer, a first GRU network, a convolutional layer, and a pooling layer. The decoding unit consists of a connected inverse convolutional layer, an inverse pooling layer, and a second GRU network. The system can adaptively extract the timing features of a bearing’s time-sequential vibration signals to enhance the feature extraction capability and reduce the noise influence. The hidden layer features are input into a classifier for identification to achieve accurate and efficient bearing fault prediction. To enable the model to effectively screen the optimal features, Wu et al. ⁹⁷ proposed the method of combining the MD with the exponential weighted moving average (EWMA) method. Then the method of performance degradation prediction using a sparse auto encoder-gated recurrent unit (SAE-GRU) fusion model significantly improves the prediction accuracy of the model. Li et al. ¹⁴² proposed the quantum attention recurrent encoder decoder neural network (QAREDNN), in which quantum neurons are used to construct a new quantum gated recurrent unit (QGRU). The QGRU traverses the solution space more finely than the recurrent units of conventional encoders and decoders. The above studies show that the GRU model is one of the research hotspots, fewer studies are using the gated recurrent model alone, and more researchers are carrying out the improvement or combination of GRU.OPEN IN VIEWER

Figure 5.

GRU network model structure.

Traditional RNNs need to solve the problem of their prediction accuracy and have poor compatibility with multidimensional data, so they need to be combined with other methods to study new prediction models. Existing improved RNNs are multilayer RNN, bidirectional RNN, dynamic RNN, and so on.

Deep learning, as an end-to-end learning approach, is largely free from human intervention during feature extraction and empirical dependence on engineering diagnosis, but there are still a large number of parameters that require human intervention in the modeling process. At the same time, it cannot get rid of the constraints of large sample training and is incapable of uncertainty representation.

Prediction models for rolling bearing performance degradation have been discussed in this section. Table 3 summarizes the bearing performance degradation prediction methods based on types of bearings and datasets. Some of the characteristics of the models used in deep learning for bearing performance degradation trend prediction are presented in Table 4.

OPEN IN VIEWER

Table 3.

Summary of bearing performance degradation prediction methods for different bearing types and datasets.

Types of bearing	Dataset	Methods
ZA–2115 double-row roller bearings	University of Cincinnati	The inverter signal was characterized and fed into the SVM, 105 the Lévy process model, 111 a bidirectional long short-term memory network prediction model, 124 a novel LSS model, 129 SWDAE and LSTM composition models 130
	NASA	KRR and support vector regression methods applied on DTW, 103 a novel LSS model 129
Roller bearings	IEEE PHM 2012	SAE-GRU, 97 a time-shifted least-squares support vector machine with integral operators, 107 MSDRSN, 122 a novel LSS model, 129 a performance degradation evaluation model constructed by SSI, DCF and LSTM, 131 combining C-MMPE features with LSTM, 132 BSBiLSTM, 133 MBiLSTM, 134 QAREDNN 142
LDK UER204 rolling bearings	XJTU-SY	A deep autoregressive recurrent model, 95 VGG, 113 using DRSN and a deep long short-term memory network, 121 BSBiLSTM, 133 using CDAE and the Bi-LSTM, 135 using CEEMDAN, CNN and LSTM, 137 optimizing CNN-LSTM using GTO, 138 tuning the LSTM using ISSA, 140 QAREDNN, 142 AdaDSC strategy with the DDE-GRU module1 143

OPEN IN VIEWER

Table 4.

Comparison of deep learning-based models for bearing performance degradation prediction.

Research methods	Advantages	Disadvantages
CNN	It has strong image processing capability and is widely used in various models.	The computation requires a large amount of data and a large amount of data to be processed, and the solution is poorly interpreted.
TCN	Convolutional operations are used to promote efficient computation. The length of input data can be flexibly adjusted. Gradient propagation is stable during training.	Inability to accurately represent short-term dependencies in response to some types of data relative to LSTM etc.
DRSN	It makes up for the inability to achieve constant mapping on nonlinear transformations during the training of traditional models while suppressing the interference of noisy data samples and redundant data samples. It makes the network more inclined to utilize the information extracted from the samples and ensures better robustness.	As with other deep learning methods can ignore the general pattern of samples and suffer from overfitting problems.
RNN	Processing context-dependent information has the advantage of being able to share weights at each time step during processing.	Dependencies of long sequential information cannot be found when the gradient vanishes. The problem of gradient explosion may occur when processing the data.
LSTM	It can effectively obtain the dependencies of long sequence information. Compensates the problem of gradient explosion relative to RNN.	The complexity of the memory unit structure leads to difficulties in interpreting some of the network processing. The high number of parameters leads to an increase in the amount of data required.
GRU	Solves the problem of gradient explosion for long sequential data as well as LSTM. The simplicity of the model and the small number of parameters lead to fast training.	Like the RNN structure cannot solve the problem of non-parallelizable computation and may touch the bottleneck of the data volume getting larger.
Generative Adversarial Networks (GAN)	Clear distribution of modeling data. Ability to train on one of multiple generator networks. No need for iterative sampling using Markov chains.	Model training requires a high rate of synchronization between the generator and the discriminator. The learning process may suffer from missing patterns.
Deep Forest (DF)	Because the model has fewer hyperparameters, fewer adjustments need to be made. Training is efficient and can handle large datasets.	The arithmetic is taking up more space. Cascading over multiple numbers of layers gives a small boost to the decision tree.