A self-adapting multi-LSTM ensemble regression mode for failure prediction of transmission line network from wireless mesh nodes’ data

Abstract

Transmission lines’ condition monitoring is an important part of smart grid construction. To ensure fast and efficient transmission of data, many mash-based wireless networks devices are adopted to collect status information. Since these nodes are exposed to the natural environment, vulnerable to damage, so it is very necessary to be predicting nodes’ fault. However, these mesh nodes are affected by a variety of complex and time-series factors, and traditional models are difficult to achieve effective failure prediction. To solve this problem, this paper proposes a self-adapting multi-LSTM ensemble regression model for transmission line network’s wireless mesh node failure prediction (MLSTM-FP), through establishes the corresponding relationship between similar time factors and LSTMs, the proposed model can realize multi time series data self-adapting and accurate failure prediction of transmission line network’s wireless mesh nodes, The experimental results show that the proposed method has a good prediction ability than traditional methods.

Keywords

LSTM deep learning transmission line network mesh failure prediction

1. Introduction

Transmission lines’ condition monitoring is an important part of smart grid construction. To ensure fast and efficient transmission of data, many mash-based wireless networks are deployed [1]. In a real operating environment, the nodes of the mash network may be continuously eroded and destroyed by various natural/human factors, resulting in the failure of the network nodes; therefore, the prediction of transmission line network’s wireless mesh nodes is very important for the grid scientific management [2]. It is obviously impossible to adapt to the inspection of a large number of nodes by means of manual one by one, and it is necessary to introduce an artificial intelligence automated method [3]; using this technique to prediction can grasp the mesh nodes’ life and damage trends and has good research value.

There are a lot of research achievements in the prediction of power equipment status: Through wavelet transform, algorithms can extract some key feature from the existing load data, thereby improving the accuracy of the forecasting [4, 5]. To cope with more parameters generated by wavelet transform, data selection method and multi-model integrated method can be to enhance the stability of forecasting [6]. Based on wave analysis, various models such as neural networks, support vector machine and hybrid-model are introduced into the field to improve the quality of prediction [7, 8]. Due to the complexity characteristics of the short-term load forecasting, it is necessary to explore deeper features from historical data, so in recent years, deep learning technology has received extensive attention in this field. By constructing the input data as feature maps, convolutional neural networks (CNNs) can perform forecasting and obtain better accuracy [9]. Based on the introduction of fuzzy time series, feature maps can contain more descriptive information and further improve the ability of CNN to cope with more complex load date [10, 11]. For the data contains obvious time series characteristics, the memory mechanism of the item contained in the neural network models are very important [13]. Recurrent neural networks (RNN) and long short-term memory networks (LSTM) can be used in load forecasting, and deep character of items and relationship between nods can be obtained from complicated data [16].

Although above researches have achieved good results in prediction of power equipment status, but mesh nodes are affected by a variety of complex and time-series factors, and traditional algorithms are difficult to achieve effective prediction [12]; When factors hidden in data are complicated but corresponding training data’s amount is limited, single times series model will tend to over-fitting with training data, thereby reducing the forecast ability of the model [14, 15]; Therefore, it is necessary to construct an adaptive mechanism of input data, and create sub-models according to different time series to prevent over-fitting and improve the generalization ability of the model. In this paper a self-adapting multi-LSTM ensemble regression model for failure prediction of transmission line network from wireless mesh nodes’ data (MLSTM-FP), through establishes the ensemble relationship between similar time factors and LSTMs, the proposed model can realize multi time series data self-adapting and accurate failure prediction of transmission line network’s wireless mesh nodes, The experimental results show that the proposed method has a good prediction ability than traditional methods.

2. Methodology

2.1 Long Short Term Memory network

Long Short Term Memory networks (LSTMs) are an effective nonlinear recent neural network. Owning to the excellent ability in processing series and nonlinear relationship of data, these network models are widely adopted in time series data analysis application, The LSTM can solve the long-term dependency problem of the traditional time series processing algorithm and achieve higher prediction accuracy. LSTM has the structure shown below.

Figure 1.

Structure of the LSTM.

As shown in Fig. 1, the LSTM consists of a number of repeating modules, each module corresponding to the input $x_{t}$ , the output $h_{t}$ and the state $C_{t}$ transmitted to the next module, where $t$ is the sequence number. For a module whose internal part contains 4 neural network parts, the formula for the cell forgetting part of the input data is as follows:

$\displaystyle f_{t}=\textit{sigmoid}(W_{f}[C_{t-1},h_{t-1},x_{t}]+b_{f}),$ (1)

where $W_{f}$ and $b_{f}$ are the weights and bias of the part, through this formula the degree of memory of the output of the previous module can be determined. The formula corresponding to the repeated of cell memory part is as follows:

$\displaystyle i_{t}=\textit{sigmoid}(W_{i}\cdot[C_{t-1},h_{t-1},x_{t}]+b_{i}),$ (2)

where $W_{i}$ and $b_{i}$ are the weights and bias corresponding to an input gate layer, and an alternative state is constructed based on:

$\displaystyle\tilde{C}_{t}=\tanh(W_{C}\cdot[h_{t-1},x_{t}]+b_{C}),$ (3)

where $\tilde{C}_{t}$ is the alternative state, $W_{C}$ and $b_{C}$ are the corresponding weights and bias. The state of the output of this module is combined with Eqs (1)–(3):

$\displaystyle C_{t}=f_{t}\times C_{t-1}+i_{t}\times\tilde{C}_{t},$ (4)

Based on this output state, the output of the module is:

$\displaystyle h_{t}=\textit{sigmoid}(W_{O}[C_{t-1},h_{t-1},x_{t}]+b_{O})\times% \tanh(C_{t}),$ (5)

Through this process, the process of transferring the memory of the previous node to the next node is realized, and the weight of each part is optimized by the training data to achieve the purpose of sequence data analysis. It can be seen from the formula that each module of LSTM contains two parts, forgetting and memory, which can realize the memory process of input values within a certain sequence range. This feature is very important in the process of sequence data processing. However, because the operation environments of mesh nodes are affected by many factors; so the frequency and amplitude of data corresponding to factors are also vary greatly. The use of a single LSTM is likely to cause confusion in different sequences. When using a larger complex neural network, this problem will cause over-fitting in training data, Therefore, new methods need to be proposed to solve the current problem.

2.2 A self-adapting multi-LSTM ensemble regression model for failure prediction of transmission line network’s from wireless mesh nodes data

This paper proposes a self-adapting multi-LSTM ensemble regression model for failure prediction of transmission line networks from wireless mesh nodes’ data (MLSTM-FP). The main goal of this method is to use multiple LSTMs to independently process the influencing factors of condition, and then integrate the outputs of these LSTMs to predict the mesh nodes’ condition. The MLSTM-FP model’s structured are shown as below.

Figure 2.

Structure of MLSTM-FP.

As shown in above figure, the input of MLSTM-FP model is multi-factor time series data of mesh nodes’ historical data $D_{\textit{historical}}$ ; and the output is time series data of regression result $O_{\textit{regression}}$ . MLSTM-FP includes four components: self-adapting separator, LSTM group, time series ensemble and regression.

(1) Self-adapting separator component: It duty is to divide $D_{\textit{historical}}$ into $M$ groups $G={\{}g_{1},\linebreak g_{2},\ldots,g_{M}{\}}$ , and adapting different characters of input series data. This component analyze each factor of the input data, and each factor constructs a time series absolute value correlation matrix $C M$ , and use a decision weight $W_{\textit{seperator}}$ to decide groupnum of a factor belongs

$\displaystyle\textit{groupnum}=\textit{round}(\textit{soft}\max(CM\times W_{% \textit{sperator}})),$ (6)

The weight $W_{\textit{seperator}}$ can be obtained by following algorithm:

Groupnum decision algorithm (GD-alg) Input: historical data Dhistorical, number of groups $M$ Output: Wseperator, Grouping result $G$ Begin while (The number of dimensions in each item of $G$ is not close to each other) Initialize grouping result $G$ in to empty set; foreach (factor in $H$ ) { $M$ $=$ calculate its correlation with all other factors groupnum $=$ calculate through Eq. (6). $g$ [groupnum] $\leftarrow$ factor Adjust the $W_{\textit{seperator}}$ using the gradient descent method return Wseperator, $G$ End

The above algorithm can adaptively process the input historical data, and separate whole data set into groups according to the characteristics of each factor.

(2) LSTM group component: After previous component, the MLSTM-FP model create $M$ number of LSTMs, each LSTM corresponding to an item in $G$ ; through training, each LSTM can output a result with $K$ time serious data. The whole output of this component is $LG={\{}{lg}_{1},{lg}_{2},\ldots,lg_{M}{\}}$ and $lg_{i}$ has $K$ time serious item, through this way, the uniformity of all factors in the sequence of changes is achieved.

(3) Time series ensemble component: This component ensemble the output of previous LSTM group component, the detail of algorithm is:

Data ensemble algorithm (DE-alg) Input: the output of previous component $L G$ Output: ensemble times serous data $D E$ Begin $DE\leftarrow\Phi$ ; for $i$ in 1: $M$ $de_{i}\leftarrow\Phi$ ; for $j$ in 1: $K$ $de_{i}\leftarrow lg_{j}$ ; $DE\leftarrow de_{i}$ ; return $D E$ ; End

Through DE-alg, the integration of time series data can be realized.

(4) Regression component: In this component a regression LSTM model is constructed, this model corresponding to $K$ time series elements, and each element has $M$ dimensions. Through input DE from previous component, this LSTM can obtain regression result.

Through the above four components, MLSTM-FP can obtain a group of time series factors with different trends and frequencies. These factors are mapped into a unified multi-dimensional sequence data after the three components of model. This sequence data includes the mesh nodes’ key characteristics, and based on these characters, the regression model achieve more accurate and stable predictions.

2.3 The overall process of the method

The overall process of the method is shown as follows.

Figure 3.

Overall process of the method.

As can be seen from Fig. 3, the overall process of method has four steps:

Input data: Select typical transmission line network’s wireless mesh nodes, analyze factors that may affect their characters changes and store data.

Data preprocessing: First, normalize all the data to map them into [0, 1], create a data with times series grid of 96 sampling points per day, and interpolate all factor data so that they can input and output together; Integrate process of preprocessing obtains a data preprocessing model.

Training: Input data for training and obtain the MLSTM-FP model.

Forecasting: Input mesh nodes’ factors use preprocessing model and MLSTM-FP model process factors sequentially to obtain prediction result.

3. Experiments

3.1 Program realization and dataset

This paper realize all the algorithms in Python 3.6, where the deep neural networks is implemented by TensorFlow, the grouping and data preprocessing use Numpy and Scikit-learn extension packages. All algorithms were executed on an Intel Xeon/16G/GTX 1070 computer, and the all experimental results were obtained on this computer. This paper introduces the data of 100 mesh nodes of transmission line network in the of year 2016, 2017 and 2018 in a region of northern China, in which 2016 and 2017 are used as training data and 2018 as test data. There are many factors that influence the status of a mesh node. The input data includes the following contents:

1.
Temperature change data: Daily, hourly, weekly temperature change range, minimum and maximum temperature.
2.
Rainfall data: The average rainfall in the area.
3.
Light intensity data: The average daily light intensity, the light intensity at two o’clock in the afternoon.
4.
Wind speed data: The average wind speed of the area.
5.
Equipment usage frequency and date: Equipment data transmission amount, holiday and working day parameters.

The typical characteristics of the above factors are that the features and frequencies are different greatly. Integrate these factors into a single multi-dimension data for time series analysis may cause analysis confusion and thus reduce the quality of prediction, so the method proposed in this paper is needed for effective processing these factors.
3.2 Results and comparisons

In order to verify the effectiveness of the algorithm, the algorithm is compared with LSTM, LSTM $+$ CNN, neural network (NN) and Support Vector Machine (SVM); and the Mash Node Life Index (MNLI) and Node Failure Average Forecasting Accuracy (NFAFA) are introduced as evaluation indicators. The evaluation formula corresponding to these two indicators is:

$\displaystyle\textit{MNLI}=\frac{1}{n}\sum\limits_{i=1}^{n}{\left|{\frac{x_{% \textit{act}}(i)-x_{\textit{preb}}(i)}{x_{\textit{act}}(i)}}\right|},$ (7) $\displaystyle\textit{NFAFA}=\frac{1}{n}\sum\limits_{i=1}^{n}\left(1+\frac{|x_{% \textit{act}}(i)-x_{\textit{preb}}(i)|}{x_{\textit{act}}(i)}\right)\times 100,$ (8)

where $x_{\textit{act}}(i)$ is actual load value, and $x_{\textit{act}}(i)$ is forecasting result of models. For all 2018 data, perform 100 nodes’ prediction, and 2 evaluation indicators for 5 algorithms are compared as shown in the following table.

Table 1

Evaluation indicators comparison of 5 algorithms

Methods	Evaluation indicators
	MNLI	NFAFA
MLSTM-FP	0.0621	93.79
LSTM	0.1048	89.52
LSTM $+$ CNN	0.0942	90.58
NN	0.1323	86.77
SVM	0.1712	82.88

As can be seen from the Table 1, due to the lack of directly processing sequence data ability, the NN (NFAFA $=$ 86.77) and SVM (NFAFA $=$ 82.88) have lower precision result, and the accuracy of LSTM is obviously higher than that of the NN and SVM algorithms which NFAFA $=$ 89.52, this result prove that using LSTM to process time series data is a feasible way to predict load data. LSTM $+$ CNN can solve different factors to some extent in convolutional process stage, and its accuracy is further improved which NFAFA $=$ 90.58. The MLSTM-FP reaches the highest accuracy among all the methods which NFAFA $=$ 93.79. To test the failure prediction ability of methods, we intercept 20000 sample data from real operating environment, these sample data are separated into training data set and test data set, all these set contain 50% are failure and 50% are normal operation data. In order to further verify the accuracy and stability of the algorithm, the training data is selected from 1000, 2000, 3000 to 10000 as the training data group, each group of fault data and normal operation data account for 50%. The overall predict accuracy of 5 methods are listed in Table 2.

Table 2

Overall accuracy comparison of five methods

Training sample number	Overall accuracy (%)
	NN	SVM	LSTM $+$ CNN	LSTM	MLSTM-FP
1000	60.42	62.09	69.54	65.35	69.97
2000	62.15	65.62	64.34	68.8	73.51
3000	62.80	71.23	73.28	72.85	78.97
4000	64.91	63.9	76.59	74.03	79.81
5000	71.36	70.34	78.22	78.96	83.17
6000	67.27	77.52	81.25	78.14	84.46
7000	73.84	65.17	84.65	78.94	86.92
8000	74.84	76.03	78.18	80.31	88.15
9000	76.74	75.63	76.57	80.89	89.55
10000	78.70	76.52	86.52	83.08	92.08

As can be seen from the Table 2, the accuracy of NN and SVM processing is often low, and the accuracy is significantly improved after the introduction of LSTM, but the accuracy fluctuation is more obvious when there are fewer samples; It has the ability to process sequence data, and the accuracy is improved compared with SVM. The MLSTM-FP proposed in this paper has achieved good results under each sample size. The accuracy trend of the five methods is shown in the Fig. 4.

Figure 4.

Accuracy trend of the five methods.

It can be seen from the Fig. 4 that NN and SVM always archive lower accuracy than deep models, this shows that the deep models in the field of failure prediction are superior to traditional shallow models. MLSTM-FP not only achieves the highest detection accuracy among the five algorithms, but also its accuracy changes are relatively stable; especially in the case of small sample size, the accuracy has always been improved; It is proved that the MLSTM-FP algorithm is also superior to other four traditional methods in stability.

3.3 Analysis

Transmission line network’s data is affected by a variety of noise factors. These factors will change with changes in the real application environment. Adapting to these influencing factors has a greater impact on the accuracy of failure prediction. The noise information simulated by the laboratory cannot exhaust all possible factors, so this situation leads to the lower accuracy of the traditional model in practical applications. The ensemble model proposed in this paper perfectly solves this problem: Each sub-model has predictive capabilities, and the solution space received by each sub-model is different, so the response of each model to noise is also different; multiple models making decisions together these models can greatly enhance the anti-noise capability of the overall model, thereby obtaining stable and effective prediction capabilities. It can be seen from Table 2 and Fig. 4 that MLSTM-FP has great advantages in accuracy and stability, especially when samples number is small. At the same time, deep learning models usually require more samples to obtain good results. Using ensemble is a very good strategy adapt with small sample size. The entire model does not over-fit with a small sample size; these results proved the real application value of MLSTM-FP.

4. Summary

The lifetime of the mash network nodes on the transmission line is affected by many factors. How to deeply extract the features behind the data and make effective forecasting is very important. This paper proposes an MLST-FP model that uses multiple LSTMs to map complex factors to a uniform series data, and then integrate data and perform regression analysis. The experimental results show that the proposed method is superior to the traditional method in prediction accuracy and stability. It has good practical value in the field of energy management and grid fault detections.

Footnotes

Acknowledgments

This work was supported in part by the Scientific and Technological Planning Project of Jilin Province (20180101057JC). A Project Supported by Scientific and Technological Planning Project of Jilin Province (2018C036-1).

References

Peng

Smith

W.A.

Borghesani

Randall

R.B.

and Peng

, Comprehensive planet gear diagnostics: Use of transmission error and mesh phasing to distinguish localised fault types and identify faulty gears, Mechanical Systems and Signal Processing 127(1) (2019), 531–550.

Tronchoni

A.B.

Bretas

A.S.

and Gazzana

D.S.

, Two-dimensional transmission line modeling method: An algorithm considering non-homogeneous media and ionization, Electric Power Systems Research 173(1) (2019), 220–229.

Baliyan

Gaurav

and Mishra

S.K.

, A review of short term load forecasting using artificial neural network models, Procedia Computer Science 48(1) (2015), 121–125.

Wang

and Goel

, Short-term load forecasting by wavelet transform and evolutionary extreme learning machine, Electric Power Systems Research 122(1) (2015), 96–103.

Bessec

and Fouquau

, Short-run electricity load forecasting with combinations of stationary wavelet transforms, European Journal of Operational Research 264(1) (2018), 149–164.

Ghofrani

Ghayekhloo

and Ghayekhloo

, A hybrid short-term load forecasting with a new input selection framework, Energy 81(1) (2015), 777–786.

Khwaja

A.S.

Naeem

Anpalagan

Venetsanopoulos

and Venkatesh

, Improved short-term load forecasting using bagged neural networks, Electric Power Systems Research 125(1) (2015), 109–115.

Kavousi-Fard

, A new fuzzy-based feature selection and hybrid TLA-ANN modelling for short-term load forecasting, Journal of Experimental & Theoretical Artificial Intelligence 25(4) (2012), 543–557.

Guo

Zhou

Zhang

and Yang

, A deep learning model for short-term power load and probability density forecasting, Energy 160(1) (2018), 1186–1200.

10.

Wang

Yang

Hong

and Lin

, Online reliability time series prediction via convolutional neural network and long short term memory for service-oriented systems, Knowledge-Based Systems 159(1) (2018), 132–147.

11.

Sadaei

H.J.

Silva

P.C.

Guimarães

G.G.

and Lee

M.H.

, Short-term load forecasting by using a combined method of convolutional neural networks and fuzzy time series, Energy 175(1) (2019), 365–377.

12.

hou

Feng

Liu

and Yang

, A hybrid short-term load forecasting model based on variational mode decomposition and long short-term memory networks considering relevant factors with Bayesian optimization algorithm, Applied Energy 237(1) (2019), 103–116.

13.

, Load forecasting via deep neural networks, Procedia Computer Science 122(1) (2017), 308–314.

14.

Ahmad

and Chen

, Deep learning for multi-scale smart energy forecasting, Energy 175(1) (2019), 98–112.

15.

Wang

and Wang

, Bi-directional long short-term memory method based on attention mechanism and rolling update for short-term load forecasting, International Journal of Electrical Power & Energy Systems 109(1) (2019), 470–479.

16.

Muzaffar

and Afshari

, Short-term load forecasts using LSTM networks, Energy Procedia 158(1) (2019), 2922–2927.