Remaining useful life prediction for aircraft engines based on phase space reconstruction and hybrid VNS-SVR model

Abstract

Based on phase space reconstruction (PSR) and hybrid VNS-SVR model, a remaining useful life (RUL) prediction method for aircraft engines is proposed. The proposed hybrid model combines support vector regression (SVR), which has been successfully adopted for regression problems, with the variable neighborhood search (VNS). First, the phase space reconstruction is used to transform the selected one-dimensional performance sequences of aircraft engines into matrix forms, which increases the data information and improve the learning efficiency of the model effectively. Then, SVR is used to construct the prediction model. Meanwhile, a VNS algorithm is proposed to optimize the kernel parameters. Finally, the hybrid model is used to RUL prediction of the aircraft engines. The experimental results show that the method has a good prediction performance.

Keywords

Remaining useful life support vector regression phase space reconstruction variable neighborhood search aircraft engine

1 Introduction

The remaining useful life (RUL) refers to the operating time of the machine before maintenance or replacement, which is a key component in predictive and health management (PHM) system [1, 2]. As the core component of the aircraft, the realization of aircraft engines safe operation is very important [3 –5]. At present, there is a relatively complete PHM system in aerospace field, which has a significant advantage in fault prospecting, automated management and decision making for maintenance. RUL prediction of aircraft engines is the core technology of the aero-engine PHM system. Predicting the RUL of aircraft engines effectively and accurately based on monitoring parameters is a prerequisite for rational maintenance planning and scientific management.

The aircraft engine is a complex system, the performance of which relates to the health status of each component, as well as the environment, load, and operation. With the rapid development of sensors and monitoring technology, airlines have accumulated more and more engines performance monitoring data during the use of aircraft engines and established a data warehouse to store these data, which facilitates the extraction of useful information for decision support. The major engine manufacturers in the world have successively developed corresponding engine health monitoring software, such as the SAGE system of General Motors, the EHM system of Pratt & Whitney, and the COMPASS system of Rolls-Royce. Figure 1 shows the diagram of main components for engine model [6]. Engine performance monitoring mainly includes the following three aspects:

Fig. 1

Diagram of main components for engine model.

Gas circuit performance monitoring, including engine exhaust temperature, rotor speed, pressure, fuel flow, etc.

Oil monitoring, including oil abrasive monitoring, oil temperature, and pressure monitoring, average oil consumption rate monitoring, and oil quality monitoring.

Vibration monitoring, including vibration offset of high and low-pressure rotors, etc.

The commonly used RUL prediction methods include reliability-based methods, failure mechanism-based methods, and data driven-based methods. The aircraft engine is a highly reliable device, its operating conditions are complex and variable, and different individuals vary widely. In practice, it is hard to collect plenty of failure data. Therefore, traditional reliability-based RUL prediction methods proves to be ineffective. Furthermore, the aircraft engine is a type of complex and high-precision equipment, which makes it challenging to find a mechanism-based model to characterize the working state. Thus, failure mechanism-based RUL prediction methods are difficult to be implemented. The data-driven approach attempts to predict the device RUL directly from the online monitoring data. The establishment of the data-driven RUL prediction model only needs to collect enough performance data, which makes it become the most widely used modeling method currently. Gertsbakh and Kordonskiy [7] first used the performance degradation data to assess the reliability of equipment and gave a simple linear degradation model with random parameters of intercept and slope. From then on, a series of research on RUL prediction with data-driven methods began. Dong and He [8] built a prediction model for diagnostics and prognostics based on semi-Markov method. Peng et al. [9] proposed a RUL estimation approach by modifying the echo state network. Chen and Tsui [10] applied the degradation signals to predict the RUL. Moghaddass and Zuo [11] diagnosed the fault based on the multistate deterioration process. Other studies on data-driven RUL prediction include support vector machine (SVM) [12 –16], Gaussian process regression [17], hidden Markov models (HMM) [18], and various neural networks (NN) [19 –23], etc.

The degradation process data sequence of aircraft engines exhibit strong non-linear and non-stationary characteristics, hence, the RUL prediction for aircraft engine is a typical non-linear time series prediction problem. Some recent references on RUL prediction of aircraft engines are listed in Table 1. Machine learning methods have performed well in solving prediction problems in recent years. Zhang and Xu [24, 25] predicted the solubility and solidification cracking through LSBoost. Ignacio et al. [26] used machine learning & uncertainty modelling to predict the ransformer lifetime. For other studies on prediction using machine learning, see [27 –29]. Table 2 shows the comparison of common machine learning methods. As shown in Table 2, compared with other models, support vector regression (SVR) is capable of handling non-linear and high-dimensional data and has high generalization ability. Consider the non-linearity and non-stationarity of the degradation process data sequence of aircraft engines, SVR is selected to solve the RUL prediction problem of aircraft engines. Based on the advantages of SVR, it has been widely used in many fields to get better accuracy. Pai and Hong [36] established a recurrent SVR model to predict the rainfall. Li et al. [37] built a manufacturing knowledge model based on SVR and the manufacturing experience. SVR model also was used to solve the traffic flow prediction problem [38] and the electric load prediction problem [39].

Table 1

RUL prediction problems of aircraft engines in recent years

Publications	Problems	Methods	Data Driven
Ellefsen et al. [2]	RUL prediction for turbofan engine degradation	Deep learning + GA	√
Nieto et al. [6]	RUL prediction of aircraft engines	PSO-SVM-based model	√
Wang et al. [30]	Prognostic of Aero-system lifetime	Similarity-based model
Le et al. [31]	RUL prediction of aircraft engines	Stochastic deterioration model
Malinowski et al. [32]	RUL prediction of critical machinery	RUL-shapelets extraction	√
Babu et al. [33]	RUL prediction of aircraft engines	Convolutional neural network	√
Li et al. [34]	RUL estimation of aircraft engine	Deep convolution neural networks	√
Wu et al. [35]	RUL estimation of aircraft engine	Long Short-Term Memory neural network	√

Table 2

Comparison of common machine learning methods

Model	Advantages	Disadvantages
Linear Regression	1. Intuitive to understand and explain	1. Unable to handle non-linear relationships
	2. very simple to implement	2. Not flexible enough to recognize complex patterns
Naive Bayes	Perform well on small-scale data	Very sensitive to the expression of input data
Logistic Regression	1. Simple calculation and fast speed	The performance is poor when faced with multivariate or non-linear decision boundary
	2. Avoid overfitting through regularization
Decision Tree	1. Able to apply to samples with missing attribute values	Easy to overfit
	2. Strong robustness to outliers
Multi-layer Perceptron	Perform well on non-linear data	1. Long training time
	2. The computational complexity is proportional to the network complexity
Support Vector Regression	1. Strong generalization ability	Sensitive to the selection of parameters and kernel function
	2. Can apply to high-dimensional non-linear data
	3. Low computational complexity

The prediction accuracy of a SVR model is not only concerned with the input features, but also closely relates to the selection of model parameters. The traditional parameter selection methods are grid search method and empirical method. The former method is time-consuming, and the latter method is too subjective. In addition, the above two parameter selection methods only adjust the parameters one by one, and can not achieve the overall unified adjustment. Recently, more and more intelligent optimization algorithms have been applied to parameter selection of SVR model, such as GA [40, 41], PSO [42, 43].

Chaos is the most commonly used model for characterizing non-linear dynamic systems. In a chaos system, enough information can be extracted from current state variables to forecast the future state [44]. The system state is not only caused by external factors, but also relates to the internal motion process determined by certain rules of the system itself. The values of the state variables in a time series result from the complex interaction among the system elements [45]. In order to extract the characteristics of a chaos system, phase space reconstruction (PSR) is widely used to process the time series data. Baydaroğlu and Koçak [46] proposed a SVR-based evaporation prediction method combined with chaotic approach. Yang et al. [47] predicted the electric load based on PSR and an learning SVR. In this paper, PSR is employed to preprocess the aircraft engine time series data.

Predicting remaining useful life through the health condition can help airlines solve engine management problems and improve their management level. Due to the importance of the RUL prediction in aircraft engines, extensive research has been undertaken to obtain higher prediction accuracy, as shown in Table 1. Although the prediction accuracy has improved in recent years, it is not enough for policymakers. Based on previous studies and the analysis on RUL prediction, this paper aims to obtain a higher prediction accuracy.

The main work of this paper concerns the performance of the SVR coupling with an intelligent algorithm for parameter optimization of RUL prediction in aircraft engines. The innovations of this paper are as follows: 1) First, PSR is used to select the features adaptively. The aircraft engine is a power system, and the phase space reconstruction of the engine performance data can help to restore the real operation process of aircraft engines. It can increase the useful information and improve the learning efficiency of the model. 2) Second, the VNS algorithm is used to optimize the parameters of the SVR model. Different from other algorithm, the VNS designed in this paper can use a variety of combinational optimization modes to find the optimal scheme of parameters, which improves the speed of finding the optimal solution. 3) Finally, the model is performed on the degradation processes data sequence of aircraft engines. The results show that the proposed method can be successfully applied to the RUL prediction of aircraft engines.

The rest of the paper is organized as follows. In section 2, phase space reconstruction algorithm is introduced and the hybrid VNS-SVR model is designed for the RUL prediction of aircraft engines. In section 3, a real example of RUL prediction for aircraft engines is used to illustrate the proposed model. The conclusions are shown in section 4.

2 Methodology

This section will introduce the necessary background knowledge and our proposed model. First, PSR and the selection methods of delay time and embedding dimension are elaborated. Then the proposed VNS-SVR model is described in detail. The flowchart of the proposed scheme is illustrated in Fig. 2. The detailed steps are as follows:

Fig. 2

The flow chart of proposed method in this paper.

Step 1. Divide the sample data into training data and predicted data.

Step 2. Use C-C method to calculate the delay time τ and embedding dimension m for both training data and testing data.

Step 3. Reconstruct the phase space with the delay time τ and embedding dimension m.

Step 4. Apply Wolf method to calculate the maximum Lyapunov index λ.

Step 5. Train and generate a SVR model based on the reconstructed training data.

Step 6. Use the designed VNS algorithm to optimize the parameters C, ɛ and γ.

Step 7. Use the trained model to predict the RUL of the testing data and output the results.

2.1 Phase space reconstruction algorithm

Affected by the coupling of varieties of factors, the performance degradation time series of aircraft engines have typical chaotic characteristic. To improve the RUL prediction accuracy of aircraft engines, based on the chaos characteristics, this paper establishes chaotic systems of performance degradation time series.

For univariate chaotic time series x (t) , t = 1, 2, ⋯ , T, Packard and Takens [48] proposed the PSR based on delay-coordinate method, $X (t) = {x (t), x (t + τ), \dots, x [t + (m - 1) τ]}^{T}$ (1) where t = 1, 2, ⋯ , T - (m - 1) τ, τ is the delay time, m is the embedding dimension.

Then the reconstructed phase space X is obtained by using the delay sequence X_t, with delay time τ and embedding dimension m, as shown in Equation (2). $X = [\begin{matrix} X_{1} \\ X_{2} \\ \begin{matrix} ⋮ \\ X_{M} \end{matrix} \end{matrix}] = [\begin{matrix} x_{1} & x_{1 + τ} & x_{1 + 2 τ} & \dots & x_{1 + (m - 1) τ} \\ x_{2} & x_{2 + τ} & x_{2 + 2 τ} & \dots & x_{2 + (m - 1) τ} \\ ⋮ & ⋮ & ⋮ & ⋮ & ⋮ \\ x_{M} & x_{M + τ} & x_{M + 2 τ} & \dots & x_{M + (m - 1) τ} \end{matrix}]$ (2) where M = T - (m - 1) τ and each row represents a vector and can be viewed as a phase point X_t (t = 1, 2, ⋯ , M) in the phase space.

Next, we will discuss the phase space reconstruction of multivariate time series. Set the N-dimensional multivariate time series X₁, X₂, ⋯ , X_N, where $X_{i} = (x_{1}^{i}, x_{2}^{i}, \dots, x_{T}^{i}), i = 1, 2, \dots, N$ . Then the phase space can be reconstructed into $V_{i} = (x_{i}^{1}, x_{i - τ_{1}}^{1}, \dots, x_{i - (m_{1} - 1) τ_{1}}^{1},, \dots, x_{i - (m_{N} - 1) τ_{N}}^{N})$ (3) where τ_i and m_i are the delay time and embedding dimension of X_i.

A key step of the phase space reconstruction is to select appropriate values for m and τ. Theoretically, there is no limit to the choice of delay time τ for the time series without noise. However, in actual situation, due to the limitation of the length of time series and noise signal, the determination methods of τ and m are of great significance to the quality of the reconstruction.

2.1.1 Delay time and embedding dimension

The selection of delay time τ is crucial. If τ is too small, the trajectory of phase space will be in the same location, and then redundant errors will be generated. If τ is too large, the shape dynamics of the adjacent moments will change dramatically and the system signal will be distorted. For the selection of embedding dimension m, in general, as long as m is large enough, the singular attractors of chaotic systems can be appropriately described, and the memory motion law of chaotic systems can be revealed. However, if m is too large, it will increase the computation. According to Takens theorem [48], when m ⩾ 2D + 1, where D is the correlation dimension, the reconstructed phase space is equivalent to the original dynamic system. In this paper, C-C method [49] is applied to calculate the τ and m.

2.1.2 Chaos characteristic judgment

Only time series with chaotic characteristics can be predicted accurately by using relevant models of chaotic theory. Therefore, to predict time series using various models of chaos theory based on phase space reconstruction, we need to prove the chaotic characteristic of time series first. The maximum Lyapunov index λ shows the divergence of two adjacent orbital spaces and measures the sensitivity of a chaotic system to the initial value, which is an important basis for judging chaotic characteristics. Based on the definition of Lyapunov index, Wolf et al. [50] proposed Wolf method to extract Lyapunov index directly according to the evolution process of the phase space. In this paper, Wolf method is used to calculate the maximum Lyapunov index λ. If λ is positive, the system has initial sensitivity and presents a chaotic state.

2.2 Support vector regression

SVR is a regression prediction method. Suppose (x₁, y₁) , ⋯ , (x_n, y_n), x_i ∈ R^m, y_i ∈ R, are the sample data. A linear function, namely SVR function, is given by $f (x) = w^{T} φ (x) + b$ (4) where w is a vector and b is the deviation. φ is a non-linear mapping and κ (x_i, x_j) = φ (x_i) , φ (x_j) denotes the kernel function that can map points from low-dimensional space to high-dimensional space.

The problem of solving the optimal hyperplane can be treated as the following optimization problem. $min_{w, b, ɛ} \frac{1}{2} | | w | |^{2} + C \sum_{i = 1}^{n} (ξ_{i}^{+} + ξ_{i}^{-})$ (5) $\begin{matrix} s . t . {\begin{matrix} w^{T} φ (x) + b - y_{i} ⩽ ɛ + ξ_{i}^{+} \\ y_{i} - w^{T} φ (x) - b ⩽ ɛ + ξ_{i}^{-} \\ ξ_{i}^{+}, ξ_{i}^{-} ⩾ 0 \end{matrix}} \\ i = 1, 2, \dots, n \end{matrix}$ (6) where C is the regularization factor. In Equation (6), ɛ is a positive constant. Losses are calculated only if the deviation between f (x) and y_i is greater than ɛ. $ξ_{i}^{-}$ and $ξ_{i}^{+}$ are the training errors bellow -ɛ and above +ɛ respectively.

To solve a quadratic problem with linear constraints, the Kalush-Kun-Tuck (KKT) optimal conditions are necessary and sufficient. Then the solution can be obtained by a combination of support vectors (s.v.),

$\begin{matrix} w = \sum_{S . V .} β_{i} φ (x_{i}) \Rightarrow f_{w, b} (x) \\ = \sum_{S . V .} β_{i} 〈 φ (x_{i}), φ (x) 〉 + b = \sum_{S . V .} β_{i} {kx}_{i}, x + b \end{matrix}$ (7)

Kernel selection is one of the key technologies to improve the ability of SVR. This paper uses the radial basis function as shown in the following equation: $κ (x_{i}, x_{j}) = e^{- γ | | x_{i} - x_{j} | |^{2}}$ (8)

The prediction accuracy and the generalization performance of the SVR model depends on the settings of the hyperparameters C, ɛ, and the kernel parameter γ. Next, we will introduce the VNS algorithm to optimize SVR parameters. Figure 3 shows the process of the combining SVR with VNS.

Fig. 3

Flowchart of VNS-SVR model.

2.3 VNS algorithm

The performance of SVR depends greatly on the parameters C, ɛ and γ in the model. Therefore, how to select proper parameters according to the training set to ensure a good performance is a key step in designing SVR model. In this paper, a VNS algorithm is proposed to optimize the above three parameters. The pseudocode of VNS is displayed in Algorithm 1.

Algorithm 1. The VNS algorithm
1. Input: A set of neighborhood structures N_k
2. Output: The best solution found
3. s = Initial()
4. Iteration
5. while stopping rule is not satisfied do
6. k = 1
7. while k ⩽ k_max. do
8. s′ = Shake(s)
9. s″ = LocalSearch(s′)
10. if f (s″) > f (s) then
11. s = s″, k = 1
12. else
13. k = k + 1
14. end if
15. end while
16. end while

2.3.1 Initial solution

First, an initial solution needs to be generated. Obviously, the initial solution has a positive effect on the performance of the VNS algorithm. A good initial solution can not only improve the accuracy of the algorithm but also reduce the running time. In this paper, we determine the initial solution of the parameters C, ɛ and γ according to Cherkassky and Ma [51].

Selection of ɛ. For a training data set { (x₁, y₁) , ⋯ , (x_n, y_n) }, the parameter ɛ can be calculated by the following prescription. $ɛ = 3 σ \sqrt{\frac{lnn}{n}}$ (9) where σ is the standard deviation of input data noise. Because σ is unknown, we use the k-Nearest Neighbor (KNN) to estimate the noise of data, $σ^{2} = \frac{n^{1 / 5} k}{n^{1 / 5} k - 1} \frac{1}{n} \sum_{i = 1}^{n} {(y_{i} - \hat{y_{i}})}^{2}$ (10) where k is the number of nearest neighbors, and $\hat{y_{i}}$ is the estimation of y_i by KNN method.

Selection of C. The initial value of parameter C can be calculated as follows: $C = \max (| \bar{y} + 3 σ_{y} |, | \bar{y} - 3 σ_{y} |)$ (11) where $\bar{y}$ and σ_y are the mean and the standard deviation of the values of y in training data.

Selection of γ. RBF kernel function is used in this paper, and the parameter γ should be selected to reflect the input range of the training data. For multivariate d-dimensional problems, γ is set to make γ^d ∈ (0.1, 0.5), which proves to yield a good SVR performance for various regression data sets.

2.3.2 Shaking

The main purpose of the shaking procedure is to find a better neighborhood of the current solution. In the shaking phase, a new solution is randomly selected in the N_k (s) structure of the current best solution s. The pseudo-code is given in Algorithm 2.

Algorithm 2. Shaking procedure.
1. Input: The current solution s
2. Output: A new solution s′
3. k = random(1,k_max)
4. s′ = N_k (s)

2.3.3 Neighborhood structures

Considering the characteristics of VNS, we design a six-digit integer coding (an integer between 0 and 10⁶) in this paper. Given the range of parameters, the values of parameters can be obtained by multiplying the coding with the corresponding coefficients. In addition, VNS explores the solution space exhaustively by sampling increasingly larger neighborhoods. Considering the above two points, we set the neighborhood structure of VNS as follows:

Algorithm 3. Neighborhood structures.
1. Input: The number of neighborhoods k_max and the current
solution s
2. Output: The perturbed vector pop
3. for j = 1 to pop_size
4. for k = 1 to k_max
5. random_num = rand (1, 3)
6. if random_num = 1 then
7. i = rand (1, 3)
8. pop_ji = P_{10^a(i),10^b(i)} (s_i ± rand (10^-m_k, 10^{m _k}))
9. else if random_num = 2 then
10. i₁ = rand (1, 3)
11. i₂ = rand (1, 3)
12. while i₁ = i₂
13. i₂ = rand (1, 3)
14. pop_{ji ₁} = P_{10^a(i₁),10^b(i₁)} (s_{i ₁} ± rand (10^-m_k, 10^{m _k}))
15. pop_{ji ₂} = P_{10^a(i₂),10^b(i₂)} (s_{i ₂} ± rand (10^-m_k, 10^{m _k}))
16. else
17. for i = 1 to 3
18. pop_ji = P_{10^a(i),10^b(i)} (s_i ± rand (10^-m_k, 10^{m _k}))
19. end for
20. end if
21. end for
22. end for
23. return pop

Where {10^a(i), 10^b(i)} is the range of the kth element in the code. m (k) reflects the size of the kth neighborhood. A larger k indicates a bigger size. P_a,b (c) is a function to guarantee that the solution after neighborhood transformation can be mapped to the solution space [52]. $P_{a, b} (x) = {\begin{matrix} a & if & x < a \\ x & if & a ⩽ x ⩽ b \\ b & if & x > b \end{matrix}$ (12)

As a matter of fact, the neighborhood structures N_k, k = 1, ⋯ , k_max, contain as many possible cases in the solution space as possible. The local search stage finds the optimal solution according to the fitness function. Finally, we obtain the optimal combination of parameters (C, ɛ, γ) by VNS algorithm, which contributes to improving the performance of SVR.

2.3.4 Fitness function

In this paper, the coefficient of determination R² is used as the main criteria to measure the accuracy of algorithm. Let y_i be the observed values and $\hat{y_{i}}$ be the predicted values, ${SS}_{err} = \sum_{i = 1}^{n} {(y_{i} - \hat{y_{i}})}^{2}$ (13) ${SS}_{tot} = \sum_{i = 1}^{n} {(y - \bar{y})}^{2}$ (14) where $\bar{y} = \frac{1}{n} \sum_{i = 1}^{n} y_{i}$ is the mean of the observed data. SS_err is the residual sum of squares. SS_tot is the total sum of squares, which is directly proportional to the sample variance. Then, R² can be calculated by: $R^{2} = 1 - \frac{{SS}_{err}}{{SS}_{tot}}$ (15)

The larger the R², the better the prediction rusults.

In addition, the scoring function (S) [53] is considered as the criterion to measure the algorithm performance in this research. The score S can be calculated by: $S = {\begin{matrix} \sum_{i}^{n} e^{(\frac{y_{i} - \hat{y_{i}}}{13})} - 1, for y_{i} - \hat{y_{i}} < 0 \\ \sum_{i}^{n} e^{(\frac{y_{i} - \hat{y_{i}}}{10})} - 1, for y_{i} - \hat{y_{i}} ⩾ 0 \end{matrix}$ (16)

The lower the score S is, the higher performance the prediction presents.

3 Experiment and results

The dataset used in the following experimental study is from NASA’s C-MAPSS turbofan engine, as seen in Table 3. Every subset consists of 3 operating parameters and 21 sensor measurements at different cycling time of the engines. Each subset of multivariate time series reflects the life cycle of each engine from inception to failure.

Table 3
The C-MAPSS dataset

Dataset FD001 FD002 FD003 FD004

Time series training set 100 260 100 249

Time series test set 100 256 100 248

Operating conditions 1 6 1 6

Fault conditions 1 1 2 2

Maximum life 362 378 525 543

span (Cycles)}

Dataset	FD001	FD002	FD003	FD004
Time series training set	100	260	100	249
Time series test set	100	256	100	248
Operating conditions	1	6	1	6
Fault conditions	1	1	2	2
Maximum life	362	378	525	543
span (Cycles)}

As shown in Table 3, time series training set and time series test set represent the number of engines in the training sample and the test sample in each subset, respectively. There is only one operation condition for FD001 and FD003. FD002 and FD004 have 6 operation conditions, which makes it more difficult to predict than FD001 and FD003. FD001 and FD002 have one fault condition, and FD003 and FD 004 have two fault conditions. Maximum life span (Cycles) shows the maximum running time of engine in each subset. See [53, 54] for a deeper understanding of the dataset.

3.1 Experiment on FD001

The time each engine begins to degrade is unknown and the life cycle of each engine varies greatly. In addition, the major degradation occurs in the second half of the life cycle. To show the engine performance degradation more accurately, inverse life is defined as follows: $Inverse Life = i - life cycle$ (17) where i is the current cycle of the engines. Inverse life = 0 indicates the failure of the engine.

Before carrying out the experiment, we find that the values of variables Temp_fan, Pressure_fan, Pressure_bypass, Engine_pressure_ratio, Burner_fuel-air_ratio, Demanded_corrected_fan_speed, and Demanded_fan_speed keep constant in the life cycle. Therefore, we choose another 14 variables as the input features for dataset.

To eliminate the influence of different dimensions on the numerical values, further normalization of data is required. The normalization formula is given by $a_{ij}^{'} = \frac{a_{ij} - a_{imin}}{a_{imax} - a_{imin}}$ (18) where a_ij is the initial sample data to be normalized, a_imin and a_imax are the minimum and maximum values in the column sample values.

Then, the phase space is reconstructed for FD001 dataset. The time series consisting of the mean value of selected sensor measurements in a certain cycle (which is represented as red lines in Fig. 4) is used to calculate the parameters τ and m of the reconstructed phase space. Table 4 shows the results of the C-C method and Wolf algorithm.

Fig. 4

The mean value of selected sensor measurements for 100 engines.

From Table 4, the maximum Lyapunov index of each feature is greater than 0, which indicates that each time series is chaotic, allowing us to reconstruct the phase space.

Table 4

The phase space reconstruction parameters and the maximum Lyapunov index of each feature

Feature	The delay time τ	The embedding dimension m	The maximum Lyapunov index
1	3	2	0.0512
2	4	5	0.0766
3	2	3	0.0490
4	2	2	0.0667
5	3	2	0.0224
6	4	5	0.1141
7	4	3	0.0302
8	2	2	0.0864
9	4	2	0.0619
10	4	4	0.1224
11	3	2	0.0369
12	2	2	0.0699
13	3	3	0.0741
14	5	2	0.0349

Before the experiment, the dataset is divided into two parts. The first 80 engines are training set and the last 20 engines are testing set. We conduct an experiment to evaluate the proposed algorithm in predicting RUL for aircraft engines.

The experiment is run on Intel Core i7-7700 8 GB, the Microsoft Windows 10 operating system, and the development environment of Python 3.6.6, PyCharm 2018.3, and 500 iterations are executed. The parameter settings are shown in Table 5.

Table 5

List of preset parameters in proposed model

Parameters	Value
Population size	10
Iteration times	500
Neighborhood structures m	(5,4,3,2,1)
Allowable error ɛ	[10^–3,10^–1]
Search range of penalty parameter C	[10^–3,10³]
Search range of kernel width γ	[10^–6,10¹]

Allowable error ɛ, penalty parameter C and kernel width γ are the hyper-parameters that need to be optimized by VNS algorithm. The ranges of ɛ, C and γ are [10^-3, 10^-1], [10^-3, 10³] and [10^-6, 10¹], respectively. According to above specified parameters, 500 iterations have been performed based on the training set and testing set, and the results are reported in Table 6.

Table 6

Optimal hyper-parameters of the PSR-VNS-SVR model

Algorithm	Values of optimal hyper-parameters	Optimum coefficient of determination (R²)	Score
PSR-VNS-SVR	C = 0.7234, ɛ = 0.0442, γ = 1.1631	0.9117	425

From the Table 6, we can know that the PSR-VNS-SVR has a good effect on predicting the RUL of aircraft engines. Figure 5 visually shows the prediction performance of the method. From Fig. 6, we find that almost all R² for these predictions and for each engine are above 80% (the 99th is 78.53%). Figure 7 describes the degree of fluctuation of errors between the observed and predicted values As shown in Fig. 8, through Pearson’s correlation coefficient, we can see a strong correlation is exist between the observed and predicted values. From the comprehensive analysis of Figs. 5, 6, 7, and 8, we can conclude that the proposed method has a good performance on predicting RUL of aircraft engines.

Fig. 5

Comparison between the actual RUL and the predicted RUL.

Fig. 6

Coefficient of determination (R2) for the RUL prediction for each turbine by the proposed method.

Fig. 7

Boxplot representing the difference between the actual RUL and the predicted RUL.

Fig. 8

The correlation between the actual RUL and the predicted RUL.

As shown in Fig. 7, almost half of the boxplots have significant outliers. This is because the prediction results on some engines are not very good, such as engines 81, 84, 86, 88, 92, 95, 96, 98. At the high point of engine RUL, the predicted results are very poor. Moreover, the closer the RUL gets to 0, the better the predicted results become. From Fig. 6, the predicted effect of engine 90 is better than that of engine 99. However, in Fig. 7, the predicted effect of engine 90 is worse than that of engine 99. This situation also occurs in previous research, refer to engine 11 and engine 24 of group1 in Nieto et al. [6]. The reason may be that the predicted value of engine 90 is better than that of engine 99 in some stages, whereas the predicted value of other stages is not accurate and the difference between the observed and predicted values is large. The predicted value of engine 99 is not accurate overall, but the difference between the observed and predicted values is very small.

3.2 Comparative experiments and results

Many scholars have reported studies on all sub-datasets C-MAPSS datasets using different methods. To demonstrate the superiority of the proposed method, four neural network models Convolutional Neural Networks (CNN), Multilayer Perceptron (MLP), Convolutional Neural Networks Gated Recurrent Units (CNN-GRU) and Convolutional and Recurrent Neural Networks (CNN-RNN) and five SVR related models are selected to compare with the proposed method. Table 7 recapitulates the results of the proposed method and the comparative methods on the RUL estimation problem. We find that the proposed hybrid model outperforms the selected comparative methods.

Table 7
Score comparison with other methods on C-MAPSS dataset

Method FD001 FD002 FD003 FD004

CNN [33] 1287 1.357e4 1596 7886

MLP [56] 1.797e4 7.803e6 1.741e4 5.617e6

CNN-GRU [57] 761 8110 1400 7610

CNN-RNN [58] 820 1.592e4 951 7212

SVR [59] 1381 5.9e5 1598 3.711e5

RVR [59] 1503 1.742e4 1431 2.651e4

SVR-W [60] 1832 / 2034 /

SVR-P-W [60] 797 / 1087 /

SVR-FCM [61] 647 / / /

Proposed method 425 5210 593 6794

Method	FD001	FD002	FD003	FD004
CNN [33]	1287	1.357e4	1596	7886
MLP [56]	1.797e4	7.803e6	1.741e4	5.617e6
CNN-GRU [57]	761	8110	1400	7610
CNN-RNN [58]	820	1.592e4	951	7212
SVR [59]	1381	5.9e5	1598	3.711e5
RVR [59]	1503	1.742e4	1431	2.651e4
SVR-W [60]	1832	/	2034	/
SVR-P-W [60]	797	/	1087	/
SVR-FCM [61]	647	/	/	/
Proposed method	425	5210	593	6794

Table 7 shows the score (S) comparison with the other methods on C-MAPSS dataset. Score function represents that the penalty grows exponentially with increasing error [53]. For engine degradation, late predictions are penalized more severely than early ones. Therefore, the lower the score S is, the higher performance the prediction presents. As shown in Table 7, the proposed method in this paper indicates substantially improved S on all subsets compared with other methods. The result is mainly due to the following two reasons.

First, the aircraft engine is a power system, and the phase space reconstruction of the engine performance data can help to restore the real operation process of aircraft engines. Second, a VNS algorithm is designed to optimize the SVR parameters. The superiority of combination of SVR and VNS is reflected in the following three aspects: 1) The initial solution generated method in VNS can not only increase the speed of convergence to the optimal parameters, but also get better parameters under the same number of iterations. 2) The shaking procedure of VNS can find a better neighborhood of the current solution. 3) In local search stage, VNS explores the solution space exhaustively by sampling increasingly larger neighborhoods. 4) We can obtain the optimal combination of parameters (C, ɛ, γ) by VNS algorithm, which contributes to improving the performance of SVR.

Although the proposed model performs well among many models, it still has some limitations. VNS optimizes the parameters of SVR in each subset by training individuals on the training set and evaluating the scores on the testing set. The average running time for each subset is 87 hours. The proposed model trades time for to accuracy a large extent. More computational-efficient SVR and faster parameter tuning mechanism should be explored to reduce the running time. With the development of computer technology, the number of layers of neural networks that can handle is increasing, and the performance of deep learning methods has surpassed machine learning in many fields. To improve the performance of SVR, it is necessary to improve the objective function, constraint conditions and kernel function of the SVR model based on the problem itself. This is what we will consider in future study.

4 Conclusions

Considering the non-linear and non-stationary characteristics of degradation processes data sequence of aircraft engines, a RUL prediction method for aircraft engines is proposed based on PSR and hybrid VNS-SVR in this paper. VNS is used to optimize the parameters of the SVR model. Experiments show that the proposed model has a better prediction accuracy of RUL prediction of aircraft engines. The main innovations of this research work can be summarized as follows:

To avoid insufficient information of the input for SVR, the features are adaptively selected by the phase space reconstruction, and the SVR predictor is trained by using the phase points as input features. It can increase the useful information and improve the learning efficiency of the model.

VNS algorithm is used to select the parameters of SVR model. The automatic optimization of model parameters selection can improve the generalization ability and training speed of prediction model.

Finally, the model is performed on the degradation processes data sequence of aircraft engines. The results show that the proposed method can be successfully applied to the RUL prediction of aircraft engines.

There are also some problems in this study. Although some features were removed before the experiment, the running time is still very long because of the phase space reconstruction of the data set. In the next study, parallel technology and methods will be utilized to improve the learning performance and reduce the computational cost.

Footnotes

Acknowledgments

This work is supported by The National Key Research and Development Program of China (2019YFB1705300), the Fundamental Research Funds for the Central Universities (Nos. JZ2020HGTB0035), the National Natural Science Foundation of China (Nos. 71801071, 71922009, 72071056, 71871080, 71601065, 71690235, 71501058, 71601060), and the Decision-making in the Manufacturing Process of Complex Product (111 project).

References

Kalgren

P.W.

, Byington

C.S.

and Roemer

M.J.

, Defining PHM, a lexical evolution of maintenance and logistics, 2006 IEEE autotestcon (2006), 353–358.

Ellefsen

A.L.

, Jørlykhaug

E.B.

, Æsøy

, Ushakov

and Zhang

, Remaining useful life predictions for turbofan engine degradation using semi-supervised deep architecture, Reliability Engineering & System Safety 183 (2019), 240–251.

Bazovsky

, Reliability theory and practice, Courier Corporation (2004).

O’Connor

and Kleyner

, Practical reliability engineering, John Wiley & Sons (2012).

Wei

, Tao

, Zhuo

S.D.

and Zio

, A dynamic particle filter-support vector regression method for reliability prediction, Reliability Engineering & System Safety 119 (2013), 109–116.

Nieto

P.J.G.

, Garcia

G.E.

, Lasheras

F.S.

and De

C.J.F.J.

, Hybrid PSO–SVM-based method for forecasting of the remaining useful life for aircraft engines and evaluation of its reliability, Reliability Engineering & System Safety 138 (2015), 219–231.

Gertsbakh

I.B.

and Kordonskiy

K.B.

, Models of failure, Springer-Verlag (1969).

Dong

and He

, A segmental hidden semi-Markov model (HSMM)-based diagnostics and prognostics framework and methodology, Mechanical Systems and Signal Processing 21(5) (2007), 2248–2266.

Peng

, Wang

, Liu

and Peng

, A modified echo state network based remaining useful life estimation approach, 2012 IEEE Conference on Prognostics and Health Management (2012), 1–7.

10.

Chen

and Tsui

K.L.

, Condition monitoring and remaining useful life prediction using degradation signals: revisited, Iie Transactions 45(9) (2013), 939–952.

11.

Moghaddass

and Zuo

M.J.

, An integrated framework for online diagnostic and prognostic health monitoring using a multistate deterioration process, Reliability Engineering & System Safety 124 (2014), 92–104.

12.

Lei

, Fault prognostic algorithm based on multivariate relevance vector machine and time series iterative prediction, Procedia engineering 29 (2012), 678–686.

13.

Benkedjouh

, Medjaher

, Zerhouni

and Rechak

, Remaining useful life estimation based on nonlinear feature reduction and support vector regression, Engineering Applications of Artificial Intelligence 26(7) (2013), 1751–1760.

14.

Geng

, Huang

M.L.

, Li

M.W.

and Hong

W.C.

, Hybridization of seasonal chaotic cloud simulated annealing algorithm in a SVR-based load forecasting model, Neurocomputing 151 (2015), 1362–1373.

15.

Fan

G.F.

, Peng

L.L.

, Hong

W.C.

and Sun

, Electric load forecasting by the SVR model with differential empirical mode decomposition and auto regression, Neurocomputing 173 (2016), 958–970.

16.

Chen

, Xu

, Chu

, Li

, Wu

, Ni

, Bao

and Wang

, Short-term electrical load forecasting using the Support Vector Regression (SVR) model to calculate the demand response baseline for office buildings, Applied Energy 195 (2017), 659–670.

17.

Pan

, Liu

J.B.

and Cao

, Remaining useful life estimation using an inverse Gaussian degradation model, Neurocomputing 185 (2016), 64–72.

18.

Azadeh

, Asadzadeh

S.M.

, Salehi

, Firoozi

and Azadeh

, Condition-based maintenance effectiveness for series–parallel power generation system—A combined Markovian simulation model, Reliability Engineering & System Safety 142 (2015), 357–368.

19.

Kelo

and Dudul

, A wavelet Elman neural network for short-term electrical load prediction under the influence of temperature, International Journal of Electrical Power & Energy Systems 43(1) (2012), 1063–1071.

20.

Ghofrani

, hayekhloo

G.M.

, Arabali

and Ghayekhloo

, A hybrid short-term load forecasting with a new input selection framework, Energy 81 (2015), 777–786.

21.

Qiao

, Shi

and An

, An application of systemic prediction evaluation parameters for neural network remaining useful life predictions models, 2015 IEEE Conference on Prognostics and Health Management (PHM), (2015), 1–4.

22.

Khwaja

A.S.

, Zhang

, Anpalagan

and Venkatesh

, Boosted neural networks for improved short-term electric load forecasting, Electric Power Systems Research 143 (2017), 431–437.

23.

Singh

and Dwivedi

, Integration of new evolutionary approach with artificial neural network for solving short term load forecast problem, Applied energy 217 (2018), 537–549.

24.

Zhang

and Xu

, Solubility predictions through LSBoost for supercritical carbon dioxide in ionic liquids, New Journal of Chemistry 44(1) (2020), 20544–20567.

25.

[ Zhang

and Xu

, Predictions of the Total Crack Length in Solidification Cracking Through LSBoost, Metallurgical and Materials Transactions A 52 (2021), 985–1005.

26.

Ignacio

A.U.J.

, Stephen

M.A.

, Brian

, Brandon

, James

and Victoria

, Adaptive Power Transformer Life-time Predictions through Machine Learning & Uncertainty Modelling in Nuclear Power Plants, IEEE Transactions on Industrial Electronics 66(6) (2019), 4726–4737.

27.

Zhang

and Xu

, Predicting the material removal rate during electrical discharge diamond grinding using the Gaussian process regression: a comparison with the artificial neural network and response surface methodology, The International Journal of Advanced Manufacturing Technology 113(5) (2021), 1527–1533.

28.

Zhang

and Xu

, Predicting Multiple Properties of Pervious Concrete through the Gaussian Process Regression, Advances in Civil Engineering Materials 10(1) (2021), 56–73.

29.

Zhang

and Xu

, Machine Learning Properties of Electrolyte Additives: A Focus on Redox Potentials, Industrial & Engineering Chemistry Research 60(1) (2020), 343–354.

30.

Lee

, A similarity-based prognostics approach for remaining useful life estimation of engineered systems, 2008 International Conference on Prognostics and Health Management (2008), 1–6.

31.

S.K.

, Fouladirad

, Barros

, Levrat

and Iung

, Remaining useful life estimation based on stochastic deterioration models: A comparative study, Reliability Engineering & System Safety 112 (2013), 165–175.

32.

Malinowski

, Chebel-Morello

and Zerhouni

, Remaining useful life estimation based on discriminating shapelet extraction, Reliability Engineering & System Safety 142 (2015), 279–288.

33.

Babu

G.S.

, Zhao

and Li

X.L.

, Deep convolutional neural network based regression approach for estimation of remaining useful life, International Conference On Database Systems For Advanced Applications, Springer (2016), 214–228.

34.

, Ding

and Sun

J.Q.

, Remaining useful life estimation in prognostics using deep convolution neural networks, Reliability Engineering & System Safety 172 (2018), 1–11.

35.

, Yuan

, Dong

, Lin

and Liu

, Remaining useful life estimation of engineered systems using vanilla LSTM neural networks, Neurocomputing 275 (2018), 167–179.

36.

Pai

P.F.

and Hong

W.C.

, A recurrent support vector regression model in rainfall forecasting, Hydrological Processes: An International Journal 21(6) (2007), 819–827.

37.

D.C.

, Liu

C.W.

, Fang

Y.H.

and Chen

C.C.

, A yield forecast model for pilot products using support vector regression and manufacturing experience–The case of large-size polarizer, International Journal of Production Research 48(18) (2010), 5481–5496.

38.

Hong

W.C.

, Dong

, Zheng

and Lai

C.Y.

, Forecasting urban traffic flow by SVR with continuous ACO, Applied Mathematical Modelling 35(3) (2001), 1282–1291.

39.

Hong

W.C.

, Dong

, Zhang

W.Y.

, Chen

L.Y.

and Panigrahi

B.K.

, Cyclic electric load forecasting by seasonal SVR with chaotic genetic algorithm, International Journal of Electrical Power & Energy Systems 44(1) (2013), 604–614.

40.

Niu

and Zhang

, Model of turbine optimal initial pressure under off-design operation based on SVR and GA, Neurocomputing 78(1) (2012), 64–71.

41.

Kalteh

A.M.

, Wavelet genetic algorithm-support vector regression (wavelet GA-SVR) for monthly flow forecasting, Water Resources Management 29(4) (2015), 1283–1293.

42.

Qin

, Zeng

and Guo

, Robust prognostics for state of health estimation of lithium-ion batteries based on an improved PSO–SVR model,y, Microelectronics Reliabilit 55(9-10) (2015), 1280–1284.

43.

Zhang

, Peng

, Liang

, Fan

and Li

, A soft self-repairing for FBG sensor network in SHM system based on PSO–SVR model reconstruction, Optics Communications 343 (2015), 38–46.

44.

Packard

N.H.

, Crutchfield

J.P.

and Shaw

R.S.

, Geometry from a Time Series, Physical Review Letters 45(9) (1980), 712–716.

45.

Takens

, Detecting strange attractors in turbulence, Springer Berlin Heidelberg 898 (1981), 361–381.

46.

Baydaroğlu

Ö.

and Koçak

, SVR-based prediction of evaporation combined with chaotic approach, Journal of Hydrology 508 (2014), 356–363.

47.

Yang

Y.L.

, Che

J.X.

, Li

Y.Y.

, Zhao

Y.J.

and Zhu

S.L.

, An incremental electric load forecasting model based on support vector regression, Energy 113 (2016), 796–808.

48.

Krstic

, Kanellakopoulos

and Kokotovic

, Nonlinear and Adaptive Control Design, Wiley (1995).

49.

Sim

K.H.

, Eykholt

and Salas

J.D.

, Nonlinear dynamics, delay times, and embedding windows, Physica D: Nonlinear Phenomena 127(1-2) (1999), 48–60.

50.

Wolf

, Swift

J.B.

, Swinney

H.L.

and Vastano

J.A.

, Determining Lyapunov exponents from a time series, Physica D Nonlinear Phenomena 16(3) (1985), 285–317.

51.

Cherkassky

and Ma

, Practical selection of SVM parameters and noise estimation for SVM regression, Neural Networks 17(1) (2004), 113–126.

52.

Alba

, Anguita

, Ghio

and Ridella

, Using Variable Neighborhood Search to Improve the Support Vector Machine Performance in Embedded Automotive Applications, IEEE International Joint Conference on Neural Networks (2008)

53.

Saxena

, Kai

, Simon

and Eklund

, Damage propagation modeling for aircraft engine run-to-failure simulation, 2008 International Conference on Prognostics and Health Management (2008), 1–9.

54.

Nieto

P.J.G.

, Combarro

E.F.

, Del

C.D.J.J.

and MontaEs

, A SVM-based regression model to study the air quality at local scale in Oviedo urban area (Northern Spain): A case study, Applied Mathematics and Computation 219(17) (2013), 8923–8937.

55.

Chapelle

, Vapnik

, Bousquet

and Mukherjee

, Choosing Multiple Parameters for Support Vector Machines, Machine Learning 46(1–3) (2002), 131–159.

56.

Liao

T.W.

and Chen

L.J.

, Manufacturing Process Modeling and Optimization Based on Multi-Layer Perceptron Network, Journal of Manufacturing Science & Engineering 120(1) (1998), 109–119.

57.

Wang

, Heng

and Fan

, Predict Remaining Useful Life of Aerospace Engine Based on CNN and GRU, the 37th Chinese Control Conference, Wuhan, China, (2018), 9118–9123.

58.

Zhang

, Dong

, Wen

, Lu

and Li

, Remaining Useful Life Estimation Based on a New Convolutional and Recurrent Neural Network, 2019 IEEE 15th International Conference on Automation Science and Engineering (CASE), (2019), 317–322.

59.

Hsu

C.S.

and Jiang

J.R.

, Remaining useful life estimation using long short-term memory deep learning, 2018 IEEE International Conference on Applied System Invention (ICASI) (2018), 58–61.

60.

Mathew

, Vadakkepat

, Luo

and Pang

C.K.

, An Ensemble of Modified Support Vector Regression Models for Data-Driven Prognostics, 2019 24th IEEE International Conference on Emerging Technologies and Factory Automation (ETFA), (2019), 885–890.

61.

, An Improved SVR-FCM Method for Remaining Useful Life Prediction of Aircraft Engines, 2020 International Conference on Computing and Data Science (CDS) 2020, 62–68.

Remaining useful life prediction for aircraft engines based on phase space reconstruction and hybrid VNS-SVR model

Abstract

Keywords

1 Introduction

2.1.2 Chaos characteristic judgment

2.2 Support vector regression

2.3.1 Initial solution

2.3.3 Neighborhood structures

Table 3 The C-MAPSS dataset Dataset FD001 FD002 FD003 FD004 Time series training set 100 260 100 249 Time series test set 100 256 100 248 Operating conditions 1 6 1 6 Fault conditions 1 1 2 2 Maximum life 362 378 525 543 span (Cycles)}

Footnotes

Acknowledgments

References

Table 3
The C-MAPSS dataset

Dataset FD001 FD002 FD003 FD004

Time series training set 100 260 100 249

Time series test set 100 256 100 248

Operating conditions 1 6 1 6

Fault conditions 1 1 2 2

Maximum life 362 378 525 543

span (Cycles)}