Stacked autoencoders and extreme learning machine based hybrid model for electrical load prediction

Abstract

Electrical load prediction plays an important role in power system management and economic development. However, because electrical load has non-linear relationships with several factors such as the political environment, the economic policy, the human activities, the irregular behaviors and the other factors, it is quite difficult to predict power load accurately. In order to further improve the electrical load forecasting performance, a hybrid model is proposed in this paper. The proposed hybrid model combines the Stacked AutoEncoders (SAE) and extreme learning machines (ELMs) to learn the characteristics of the time series data of electrical load. In this proposed method, in order to utilize the characteristics of the electrical load in different depths, the outputs of each layer of the SAE are taken as the inputs of one specific ELM. Then, the obtained results from the constructed different ELMs are integrated by the linear regression to obtain the final output. The linear regression part is trained by the least square estimation method. In addition, the hybrid model is applied to predict two real-world electrical load time series. And, detailed comparisons with the SAE, ELM, the back propagation neural network (BPNN), the multiple linear regression (MLR) and the support vector regression (SVR) are done to show the advantages of the proposed forecasting model. Experimental and comparison results demonstrate that the proposed hybrid model can achieve much better performance than the comparative methods in electrical load forecasting application.

Keywords

electrical load prediction hybrid model stacked autoencoder extreme learning machine

1 Introduction

With the rapid development of the economy, the power grid is facing an increasingly complex operating environment. In order to adapt to economic growth and make best of electric energy, improving the quality and safety of electricity has become a crucial issue. There are many ways to improve the electrical performance, and electrical load forecasting is one of the most effective methods. Power load forecasting is to construct the models to systematically deal with the past and future loads which can reflects some important factors, such as the capacity increase decision, the system operation characteristics, the natural conditions and the social impact, and to predict the load value at a specific time in the future on the premise of meeting certain accuracy [1]. Reliable electrical load forecasting can decrease energy consumption and maintain the safe and stable operation of the power grid.

As the electrical load prediction has the properties of nonlinearity and high levels of uncertainties, many scholars have conducted comprehensive researches on this topic and have provided different kinds of methods [2 –4]. One typical method is the artificial intelligence [5 –8]. In [9], the support vector regression (SVR) was used for annual load forecasting. In [10], the random forest model was successfully applied to predict the short-term electrical load. In [11], the kernel-based support vector quantile regression and Copula theory were presented for estimating the short-term power load probability density prediction. In [12, 13], the least square support vector machine (LSSVM) to mid-long term electrical load forecasting was provided. Artificial neural networks (ANNs) has advantages in time series modeling and nonlinear fitting, and different kinds of ANNs have been applied to the electrical load prediction application. For example, in [14], a similar day-based wavelet neural network was used to forecast the tomorrow’s electrical load, and the proposed method has been extended successfully for holiday load forecasting. In [15], the back propagation based wavelet neural network was designed for the very short-term load prediction. ANNs were also utilized to improve the accuracy of electrical load forecasting in [16 –18]. On the other hand, the hybrid models combing different methods can achieve better performance than the single models in time series prediction application. In [19], the hybrid model which combines the fuzzy neural networks with wavelet fuzzy neural network was presented to estimate the electrical load. In this hybrid model, the fuzzified wavelet features were used as the inputs and the Choquet integral was employed as the outputs. In [20], the SVR and Bayes methods were integrated to improve the performance of the model for electrical load prediction. In [21 –23], the feasibility and applicability of the hybrid models were also confirmed in this application. However, because electrical load has non-linear relationships with several factors such as the political environment, the economic policy, the human activities, the irregular behaviors and the other factors, it is still a quite difficult task to predict the power load accurately.

Deep learning is developed on the foundation of artificial neural network. In the deep learning models, deeper representative features can be extracted from the bottom to the top of the deep networks. Deep learning is more feasible and effective than the conventional ANNs in dealing with high-dimensional data and complex functions [24, 25]. Therefore, deep learning method has been universally used in various fields. In [26 –28], the generative adversarial net was applied to the parallel prediction of the building energy consumption, and conputational experiments methods were also given. In [29], the feed-forward denoising convolutional neural networks (DnCNNs) were utilized for image denoising to speed up the training process as well as to boost the denoising performance. In [30 –32], deep learning technologies were designed for image classification. In [33, 34], deep learning algorithms were applied to speech recognition. And, the problem of sentiment analysis was discussed in [35, 36]. In [37], a novel prediction method which composed DBN, multi-layer perceptron (MLP) and autoregressive integrated moving average (ARIMA) was given to analyze time series data. As a specific deep learning method, stacked autoencoders (SAE) utilizes the outputs of the previous layer as the inputs of the current layer, thus forming a deep network, to better represent the characteristics of input data. The SAE has been applied in many fields, especially in the building energy consumption prediction [38]. Existing results have verified that the SAE has better performance than traditional methods.

In this paper, in order to further improve the electrical load forecasting performance, a SAE based hybrid model is proposed. The proposed hybrid model combines the SAE with the extreme learning machine (ELM) to extract the characteristics of time series data of the electrical load. This proposed method takes the outputs of each layer of the SAE as the inputs of different ELMs, then, integrates the outputs of the ELMs to generate the final output by the linear regression method. In addition, to verify the effectiveness of the proposed hybrid model, it is applied to predict two real-world electrical load time series. Further, detailed comparisons with the SAE, ELM, BPNN, MLR and SVR are done to show the advantages of the proposed forecasting model.

The remainder of this paper is as follows. In Section 2, the SAE and the ELM will be introduced respectively. In Section 3, the proposed hybrid model will be presented in detail. In Section 4, two electrical load prediction experiments will be done. In addition, the experimental results and comparative experiments will be given in this section. Finally, in Section 5, the conclusions will be made.

2 Methodologies

In this section, the SAE and ELM will be introduced briefly. These two methods will be used as the main components of the proposed hybrid model.

2.1 AutoEncoder

AutoEncoder (AE) network is a shallow neural network which consists of the input layer, the hidden layer and the output layer [39, 40]. The structure of one typical AE is shown in Fig. 1. It is an unsupervised neural network and uses backpropagation algorithm to train weights. For each input vector, the expected output is set to be the same input vector. The output of this network approximates the input data, through minimizing the error between the two. The aim of an AE is to learn a compressed or sparse representation for a set of data.

The AE includes two processes: the encoding and the decoding. The encoding process from input layer to hidden layer for primary data x can be represented as [41]

Fig.1

The structure of one typical AE

$g_{1} (x) = f (W_{1} x + b_{1})$ (1) where W ₁ is the encoding matrix, b ₁ is the encoding bias vector, and f (·) represents the activation function. The decoding process from the hidden layer to the output layer can be computed as [42]

$g_{2} (x) = f (W_{2} g_{1} (x) + b_{2})$ (2) where W ₂ is the decoding matrix, b ₂ is the decoding bias vector, and f ( ·) represents the activation function. Then, the reconstruction error loss function of the data can be expressed as

$J (W, b) = \frac{1}{N} \sum_{l = 1}^{N} \frac{1}{2} | | g_{2} (x^{(l)}) - x^{(l)} | |^{2}$ (3) where N is the number of the training samples, x ^(l) is the l-th input training data. In the AE model, the hidden layer can be regarded as another characteristic expression of original data. Because of its special capability, the AEs are widely used for dimensionality reduction or feature learning [43].

2.2 Stacked autoEncoder

SAE is a neural network with two or more layers of automatic encoder, which is used in an unsupervised way. The structure of the SAE is shown in Fig. 2. Its main idea is to capture high-order features from data. The SAE uses greedy layer-wise training method to train each layer of the network in turn, and then pre training the whole depth neural network to better learn the data characteristics [44]. It trains each hidden layer separately, and the output of each hidden layer is used as the input of the next layer. Let’s take the training of a stacked autoencoder with two hidden layers as an example. Firstly, an AE with a hidden layer is created and trained, and the initial feature g⁽¹⁾ is activated after training. Then, the data is provided to the second AE, and the second feature g⁽²⁾ is obtained as a new representation of the data.

Fig.2

The structure of the SAE

2.3 Extreme learning machine

Extreme learning machine (ELM) is a new learning method proposed by Huang et al. [45] for the single hidden layer feedforward neural network. It has fast learning speed and satisfactory approximation performance. In this method, the number of hidden layer nodes is set before training. The input weights and the parameters of the hidden layers are randomly assigned in the course of training. And, the weights between the hidden layer and the output layer are then determined by the least square method. The whole process needs no iteration and can produce one unique optimal solution [46, 47].

Suppose that the training data set is ${(x^{(l)}, y^{(l)})}_{l = 1}^{N}$ where N is the number of samples, x ^(l) ∈ Rⁿ and y^(l) ∈ R.

Then, the prediction result for the input x can be expressed as

$\hat{y} (x) = \sum_{i = 1}^{d} β_{i} h (w_{i} x + b_{i})$ (4) where h ( x ) represents the activation function, w _i and b_i are the parameters in the activation function, and d is the number of the hidden nodes in the ELM.

ELM tries to approximate the training samples with no error, which can be mathematically expressed as

$\sum_{l = 1}^{N} ∥ {\hat{y}}^{(l)} - y^{(l)} ∥ = 0$ (5) where y^(l) and ${\hat{y}}^{(l)}$ are respectively the lth real value and its corresponding predicted value.

Then, Equation (5) can be rewritten as

$H β = y$ (6) where H is the training matrix, β is the vector of all the weights between the hidden layer and output layer, and they can computed as $H = [\begin{matrix} h (w_{1} x^{(1)} + b_{1}), \dots, h (w_{d} x^{(1)} + b_{d}) \\ h (w_{1} x^{(2)} + b_{1}), \dots, h (w_{d} x^{(2)} + b_{d}) \\ ⋮ \\ h (w_{1} x^{(N)} + b_{1}), \dots, h (w_{d} x^{(N)} + b_{d}) \end{matrix}]$ (7) $y = [y^{(1)}, y^{(2)}, \dots, y^{(N)}]^{T}$ (8) $β = [β_{1}, β_{2}, \dots, β_{d}]^{T}$ (9)

Finally, the weight vector β can be determined as

$β = H^{†} y$ (10) in which H ^† is the Moore-Penrose pseudo inverse of the training matrix H .

3 The proposed hybrid model

In this section, the proposed hybrid prediction model will be given. Firstly, we will present the framework of the proposed hybrid prediction model. Then, we will discuss how to determine the input variables for this hybrid model, and generate the original training data set based on the observed electrical load time series. Further, the design processes of this model will be discussed step by step.

3.1 Framework of the proposed hybrid model

The structure of the proposed hybrid model is demonstrated in Fig. 3. The outputs of each layer of SAE are calculated firstly and used as the inputs of the ELMs. Then, the outputs of the constructed different ELMs will be ensembled by the linear regression method to generate the final output of the hybrid model. It can be summarized into the following four steps.

Step 1. Firstly, determine the input variables of the prediction model. In order to realize this task, we adopt the partial autocorrelation function (PACF) to analyze the correlation between time series. Once the input variables are determined, the original training data set will be generated from the observed electrical load time series data.

Step 2. Train the SAE layer by layer, and generate one corresponding mediate training data set from each of the SAE’s hidden layers.

Step 3. Construct one corresponding ELM according to each newly generated mediate training data set.

Step 4. Integrate the SAE and the constructed different ELMs to generate the final output. In this step, the linear regression method is adopted to ensemble the prediction results from such models.

Fig.3

The structure of the proposed hybrid model

3.2 Determination of input variables and generation of the original training data set

Suppose that we have the electrical load time series data as p (1) , p (2) , p (3) , ⋯ , p (M). The constructed prediction model uses the previous values at time t to predict the value at time t + 1. This can be mathematically expressed as

$\hat{p} (t + 1) = \hat{y} (p (t), p (t - 1), \dots, p (t - n - 1))$ (11) where $\hat{y} (\cdot)$ is the predicted value from the prediction models, e.g. the proposed hybrid model. In this prediction model, to set up the input variables of the prediction models, the important task is to determine the delay n according to the electrical load time series data.

In this study, the partial autocorrelation function (PACF) is adopted to realize this task. The PACF is one of the main tools to analyze the correlation between time series. It reflects the internal relations and interdependencies of time series variables and plays an important role in analyzing time series.

We adopt the steps in [48, 49] to calculate the delay n. Such steps are summarized as follows:

Calculate the autocovariance of the electrical load time series as

$r (j) = \frac{1}{M} \sum_{i = j + 1}^{M - j} [p (i) - μ] [p (i - j) - μ]$ (12) where j = 0, 1, ⋯ , M - 1, $μ = \frac{1}{M} \sum_{i = 1}^{M} p (i)$ is the mean of the observed electrical load time series data.

Calculate the sample autocorrelation function (ACF) as

$ρ_{j} = \frac{r (j)}{r (0)}$ (13)

Solve the following Yule-Walker equation to get the PACF value ψ (j) = ψ_jj as

$[\begin{matrix} 1 & ρ_{1} & \dots & ρ_{n - 1} \\ ρ_{1} & 1 & \dots & ρ_{n - 2} \\ ⋮ & ⋮ & ⋱ & ⋮ \\ ρ_{n - 1} & ρ_{n - 2} & \dots & 1 \end{matrix}] [\begin{matrix} ψ_{n 1} \\ ψ_{n 2} \\ ⋮ \\ ψ_{nn} \end{matrix}] = [\begin{matrix} ρ_{1} \\ ρ_{2} \\ ⋮ \\ ρ_{n} \end{matrix}]$ (14)

When n satisfies the following conditions $\begin{matrix} ψ (n - 1) > \frac{2}{\sqrt{M + n}}, and \\ \frac{- 2}{\sqrt{M + n}} < ψ (n) < \frac{2}{\sqrt{M + n}} \end{matrix}$ (15) it can be chosen as the delay for predicting the electrical load.

Once the time delay is determined, the original training data set can be obtained as

${(x^{(l, 0)}; y^{(l)}) | x^{(l, 0)} \in R^{n}, y^{(l)} \in R}_{l = 1}^{N}$ (16) where x ^(l,0) = (p (l) , p (l + 1) , ⋯ , p (l + n)), y^(l) = p (l + n + 1) and N = M - n - 1.

3.3 New data set generation

Based on this original data set, we will train the SAE to generate the mediate new data sets from the hidden layers of the SAE.

Firstly, train the first layer of the SAE, then we can obtain the first new data set ${(x^{(l, 1)}; y^{(l)})}_{l = 1}^{N}$ , where x ^(l,1) = f ( x ^(l,0)) which can be derived from the first layer of the SAE.

Then, based on this newly generated data set ${(x^{(l, 1)}; y^{(l)})}_{l = 1}^{N}$ , we train the second layer of the SAE, and then the second new data set can be generated as ${(x^{(l, 2)}; y^{(l)})}_{l = 1}^{N}$ , where x ^(l,2) = f ( x ^(l,1)) is the output of the second layer of the SAE.

Continue the above process, through training the ith layer of the SAE, we can obtain the ith new data set ${(x^{(l, i)}; y^{(l)})}_{l = 1}^{N}$ , where x ^(l,i) = f ( x ^(l,i-1)).

Assume that we have k hidden layers in the SAE. Then, using the above new data set generation process, we can obtain k + 1 data sets for training the prediction model. The data sets include the original training data set and the newly generated ones which are listed below

${(x^{(l, 0)}; y^{(l)})}_{l = 1}^{N}, \dots, {(x^{(l, k)}; y^{(l)})}_{l = 1}^{N}$ (17)

3.4 Training different ELMs

For each one of the k + 1 data sets, we construct one ELM. In other words, using the data set ${(x^{(l, j)}; y^{(l)})}_{l = 1}^{N}$ , we construct the jth ELM where j = 0, 1, 2, ⋯ , k.

Following the ELM method in Subsection 2.3, the constructed jth ELM model has the following input-output mapping

${\hat{y}}_{j, e} (x) = \sum_{i = 1}^{d_{j}} β_{i}^{j} h (w_{i}^{j} x + b_{i}^{j})$ (18) where d_j is the number of hidden neurons in the jth ELM, $w_{i}^{j}$ and $b_{i}^{j}$ are the parameters in the activation functions which are determined randomly, and the weights $β_{i}^{j}$ s (i = 1, ⋯ , d_j) can be determined as follows

$\begin{matrix} β^{j} & = [β_{1}^{j}, β_{2}^{j}, \dots, β_{d_{j}}^{j}]^{T} \\ = {[\begin{matrix} h (w_{1}^{j} x^{(1, j)} + b_{1}^{j}), \dots, h (w_{d_{j}}^{j} x^{(1, j)} + b_{d_{j}}^{j}) \\ h (w_{1}^{j} x^{(2, j)} + b_{1}^{j}), \dots, h (w_{d_{j}}^{j} x^{(2, j)} + b_{d_{j}}^{j}) \\ ⋮ \\ h (w_{1}^{j} x^{(N, j)} + b_{1}^{j}), \dots, h (w_{d_{j}}^{j} x^{(N, j)} + b_{d_{j}}^{j}) \end{matrix}]}^{†} \\ [\begin{matrix} y^{(1)} \\ y^{(2)} \\ ⋮ \\ y^{(N)} \end{matrix}] \end{matrix}$ (19) in which † means the Moore-Penrose pseudo inverse.

3.5 Models integration

From above discussion, we can obtain the original prediction result ${\hat{y}}_{0, e} (x)$ from the ELM constructed by the original data set, and the mediate prediction outputs ${\hat{y}}_{j, e} (x), j = 1, 2, \dots, k$ from the ELMs constructed by the mediate new data sets. In addition, the SAE can also output the prediction result ${\hat{y}}_{sae} (x)$ . In order to ensemble the prediction results from such models, the linear regression method is adopted.

In other words, the final prediction output of the hybrid model can be obtained as

$\begin{matrix} \hat{y} (x) = & c_{0} + c_{1} {\hat{y}}_{sae} (x) + c_{2} {\hat{y}}_{0, e} (x) + c_{3} {\hat{y}}_{1, e} (x (1)) \\ + \dots + c_{k + 2} {\hat{y}}_{k, e} (x (k)) \end{matrix}$ (20) where x (1) , ⋯ , x (k) are respectively the outputs of the corresponding layer of the SAE for the input x , c₀, ⋯ , c_k+2 are the weights of the different SAE and ELMs.

For the original training data set, we expect that

$\begin{matrix} \hat{y} (x^{(l, 0)}) = & c_{0} + c_{1} {\hat{y}}_{sae} (x^{(l, 0)}) + c_{2} {\hat{y}}_{0, e} (x^{(l, 0)}) \\ + c_{3} {\hat{y}}_{1, e} (x^{(l, 1)}) + \dots + c_{k + 2} {\hat{y}}_{k, e} (x^{(l, k)}) \\ = & y^{(l)} \end{matrix}$ (21)

This formula can be rewritten in the matrix and vector form as

$A c = y$ (22) where $c = [c_{0}, c_{1}, c_{2}, \dots, c_{k + 2}]^{T}$ (23) $A = [\begin{matrix} 1 & {\hat{y}}_{sae} (x^{(1, 0)}) & \dots & {\hat{y}}_{k, e} (x^{(1, k)} \\ 1 & {\hat{y}}_{sae} (x^{(2, 0)}) & \dots & {\hat{y}}_{k, e} (x^{(2, k)}) \\ ⋮ \\ 1 & {\hat{y}}_{sae} (x^{(N, 0)}) & \dots & {\hat{y}}_{k, e} (x^{(N, k)}) \end{matrix}]$ (24) $y = [y^{(1)}, y^{(2)}, \dots, y^{(N)}]^{T}$ (25)

Then, the parameters c₀, c₁, ⋯ , c_k+2 of the linear regression part of the hybrid model can be obtained as $c = A^{†} y,$ (26) where A ^† is the Moore-Penrose pseudo-inverse of the matrix A .

4 Experiments

In order to show the effectiveness of the proposed hybrid prediction model, this section will apply it to two real-world electrical load prediction applications. Furthermore, to demonstrate the advantages of the proposed method, it will be compared with several popular artificial intelligence methods. In this section, we will firstly introduce the comparative methods and the evaluation indices, and then give the experiments and comparisons in detail.

4.1 Comparative methods

In this study, the proposed hybrid method will be compared with the pure SAE and ELM which are also the ensembled components of the hybrid prediction model. In addition, the proposed method will compared with some popular methods, such as the Back Propagation Neural Network (BPNN) [50, 51], the Support Vector Regression (SVR) [52] and the Multiple Linear Regression (MLR) [53, 54].

BPNN is a multilayer feedforward neural network based on the error back propagation algorithm. It has at least three layers of neurons and can realize arbitrary continuous mapping [50]. It has been widely used in various fields, such as the building energy consumption prediction [15], the network traffic forecasting [51], and so on.

SVR is a variant of Support Vector Machine (SVM). It adopts the kernel functions to solve some complex high-dimensional modeling or prediction problems. It has the superiorities of simplicity and strong generalization ability [52].

MLR is a correlation prediction method. It can determine the mathematical relationship between model variables by reasonably analyzing the observed data of the samples, and finally achieve prediction. It has been widely used in the river discharge forecasting [53], solar energy prediction [54] and other industrial fields.

4.2 Evaluation indices

In this paper, we adopt the following four indices which are the mean absolute error (MAE), the root mean square error (RMSE), the mean relative error (MRE) and the mean absolute error percentage (MSPE) to evaluate the performance of the proposed hybrid model and the comparative methods. Such indices can be computed as $MAE = \frac{1}{N} \sum_{l = 1}^{N} | {\hat{y}}^{(l)} - y^{(l)} |$ (27) $RMSE = \sqrt{\frac{\sum_{l = 1}^{N} ({\hat{y}}^{(l)} - y^{(l)})^{2}}{N}}$ (28) $MRE = \frac{1}{N} \sum_{l = 1}^{N} \frac{| {\hat{y}}^{(l)} - y^{(l)} |}{y^{(l)}} \times 100 %$ (29) $MSPE = \frac{1}{N} \sqrt{\sum_{y = 1}^{N} (\frac{{\hat{y}}^{(l)} - y^{(l)}}{y^{(l)}})^{2}}$ (30) where N is the number of training or test samples, ${\hat{y}}^{(l)}$ and y^(l) are respectively the predicted values and real values of the electrical load.

MAE is the average of absolute values of deviations between all observations and arithmetic average, and it can accurately reflect the actual prediction error. RMSE is the square root of the ratio of the sum of the observed and true deviations to the number of observations. RMSE is very sensitive to outliers. MRE and MSPE are also often used to measure the quality of a model’s predictions. Greater values of these four indicators mean greater differences between the predicted values and the original values and worse prediction performance. In other words, for these comparison indices, smaller values imply better performance.

4.3 Experiment 1

4.3.1 Applied data set

The electrical load data set in the first experiment was downloaded from the public website: https://openei.org/. In this study, we use the electrical load data of the Oak Ridge National Laboratory for our first experiment. The Oak Ridge National Laboratory is an Integration Center of the Campbell Creek Research House, and it is a research center for building technologies. In this experiment, the data sampling period was fifteen minutes, including 35040 samples from October 1, 2013 to October 1, 2014. This experiment divides the data into two parts: the training set and the test set. The data samples from October 1, 2013 to September 1, 2014 are used for training while the left data samples are for testing.

4.3.2 Experimental setting

Following the PACF method presented in Subsection 3.2, the delay n is determined to be 22, i.e. there are 22 input variables in the prediction models.

On the other aspect, the number of hidden layers and the number of hidden units in each hidden layer are significant to the performance of SAE model. In order to determine the optimal structure of the SAE model in this experiment, the number of hidden layers and the number of hidden units in each hidden layer were taken as the key research factors. The number of hidden layers was tested from 2 to 5, while the number of hidden units in each hidden layer was chosen from 150, 200, 250, 300, 350, 400, 450, 500, 550, 600, 650, 700, 750, 800. Thus, 56 trials are given.

The values of the MAE, MRE, RMSE and MSPE of the SAE model in the 56 trials in this experiment are shown in Table 1. From Table 1, we can see that when the number of hidden layers is 4 and the number of hidden units in each hidden layer is 800, the four indices have the smallest values among all the 56 trials. In other words, the SAE model can achieve the best performance when it has 4 hidden layers and each its hidden layer has 800 neurons. For the ELM in the hybrid model, the number of neuron nodes is set to be 110.

Table 1
The values of the four indices of the SAE model in the 56 trials in the first experiment.

h k MAE RMSE MRE(%) MSPE(%) h k MAE RMSE MRE(%) MSPE(%)

2 150 1.6184 2.3765 7.4137 97.24 4 150 1.6240 2.4387 7.1481 92.64

2 200 1.5150 2.3189 6.5727 85.34 4 200 1.4865 2.3073 6.5504 90.59

2 250 1.4330 2.2297 6.3570 81.92 4 250 1.4585 2.2295 6.6264 82.27

2 300 1.3974 2.1605 6.5246 88.51 4 300 1.3678 2.1497 6.3206 86.30

2 350 1.3256 2.0739 6.0883 76.24 4 350 1.2786 2.0497 5.9336 78.64

2 400 1.2577 2.0647 5.7614 76.42 4 400 1.2353 2.0490 5.8087 73.73

2 450 1.2139 1.9983 5.4068 66.11 4 450 1.1976 1.9985 5.4269 63.96

2 500 1.1944 1.9567 5.3667 62.17 4 500 1.1556 1.9545 5.2686 63.18

2 550 1.1685 1.9118 5.3297 60.72 4 550 1.1419 1.8849 5.2929 63.85

2 600 1.0923 1.8300 5.0757 59.88 4 600 1.1127 1.8740 4.9901 54.84

2 650 1.1022 1.8447 4.9825 55.45 4 650 1.1165 1.8468 5.1105 62.41

2 700 1.0934 1.8296 5.0041 56.21 4 700 1.0399 1.7599 4.7286 51.08

2 750 1.0582 1.7658 4.8348 53.21 4 750 1.0542 1.8004 4.7252 52.14

2 800 1.0533 1.7951 4.7774 55.59 4 800 1.0353 1.7337 4.6500 50.45

3 150 1.6672 2.4534 7.4055 94.93 5 150 1.6445 2.4353 7.3808 98.36

3 200 1.5641 2.3762 6.9989 95.41 5 200 1.5039 2.2966 6.7659 93.49

3 250 1.4654 2.2347 6.6605 87.52 5 250 1.4908 2.2403 6.7157 86.66

3 300 1.4136 2.1866 6.3052 84.10 5 300 1.4358 2.1987 6.5015 83.73

3 350 1.3307 2.0987 6.2767 85.65 5 350 1.2835 2.0497 5.8990 75.24

3 400 1.2401 2.0163 5.5461 64.05 5 400 1.2291 1.9970 5.9931 68.72

3 450 1.2939 2.0454 5.9376 76.03 5 450 1.2074 1.9785 5.4551 66.37

3 500 1.1711 1.9567 5.3406 64.59 5 500 1.1633 1.9135 5.2719 62.88

3 550 1.1542 1.8992 5.1944 58.93 5 550 1.1403 1.8861 5.2715 66.34

3 600 1.1092 1.8843 5.0239 57.70 5 600 1.1337 1.8807 5.1645 61.78

3 650 1.1246 1.8390 5.2648 62.93 5 650 1.1039 1.8384 5.0818 59.47

3 700 1.0737 1.7724 4.9503 52.17 5 700 1.0736 1.7972 4.8745 54.89

3 750 1.0828 1.7967 5.0012 56.72 5 750 1.0977 1.8264 4.9747 54.08

3 800 1.0827 1.7747 5.0671 56.87 5 800 1.0547 1.7569 4.8656 52.25

h	k	MAE	RMSE	MRE(%)	MSPE(%)	h	k	MAE	RMSE	MRE(%)	MSPE(%)
2	150	1.6184	2.3765	7.4137	97.24	4	150	1.6240	2.4387	7.1481	92.64
2	200	1.5150	2.3189	6.5727	85.34	4	200	1.4865	2.3073	6.5504	90.59
2	250	1.4330	2.2297	6.3570	81.92	4	250	1.4585	2.2295	6.6264	82.27
2	300	1.3974	2.1605	6.5246	88.51	4	300	1.3678	2.1497	6.3206	86.30
2	350	1.3256	2.0739	6.0883	76.24	4	350	1.2786	2.0497	5.9336	78.64
2	400	1.2577	2.0647	5.7614	76.42	4	400	1.2353	2.0490	5.8087	73.73
2	450	1.2139	1.9983	5.4068	66.11	4	450	1.1976	1.9985	5.4269	63.96
2	500	1.1944	1.9567	5.3667	62.17	4	500	1.1556	1.9545	5.2686	63.18
2	550	1.1685	1.9118	5.3297	60.72	4	550	1.1419	1.8849	5.2929	63.85
2	600	1.0923	1.8300	5.0757	59.88	4	600	1.1127	1.8740	4.9901	54.84
2	650	1.1022	1.8447	4.9825	55.45	4	650	1.1165	1.8468	5.1105	62.41
2	700	1.0934	1.8296	5.0041	56.21	4	700	1.0399	1.7599	4.7286	51.08
2	750	1.0582	1.7658	4.8348	53.21	4	750	1.0542	1.8004	4.7252	52.14
2	800	1.0533	1.7951	4.7774	55.59	4	800	1.0353	1.7337	4.6500	50.45
3	150	1.6672	2.4534	7.4055	94.93	5	150	1.6445	2.4353	7.3808	98.36
3	200	1.5641	2.3762	6.9989	95.41	5	200	1.5039	2.2966	6.7659	93.49
3	250	1.4654	2.2347	6.6605	87.52	5	250	1.4908	2.2403	6.7157	86.66
3	300	1.4136	2.1866	6.3052	84.10	5	300	1.4358	2.1987	6.5015	83.73
3	350	1.3307	2.0987	6.2767	85.65	5	350	1.2835	2.0497	5.8990	75.24
3	400	1.2401	2.0163	5.5461	64.05	5	400	1.2291	1.9970	5.9931	68.72
3	450	1.2939	2.0454	5.9376	76.03	5	450	1.2074	1.9785	5.4551	66.37
3	500	1.1711	1.9567	5.3406	64.59	5	500	1.1633	1.9135	5.2719	62.88
3	550	1.1542	1.8992	5.1944	58.93	5	550	1.1403	1.8861	5.2715	66.34
3	600	1.1092	1.8843	5.0239	57.70	5	600	1.1337	1.8807	5.1645	61.78
3	650	1.1246	1.8390	5.2648	62.93	5	650	1.1039	1.8384	5.0818	59.47
3	700	1.0737	1.7724	4.9503	52.17	5	700	1.0736	1.7972	4.8745	54.89
3	750	1.0828	1.7967	5.0012	56.72	5	750	1.0977	1.8264	4.9747	54.08
3	800	1.0827	1.7747	5.0671	56.87	5	800	1.0547	1.7569	4.8656	52.25

For the BPNN, the number of hidden layer nodes and the iteration number were respectively set to be 200 and 20000. Additionally, the sigmoid function and the gradient descent-based algorithm were used in the training process. For the SVR, the radial basis function was chosen as the kernel function and the penalty coefficient was set to 100. In addition, the parameter γ in the SVR was set to be 0.001.

4.3.3 Experimental results

The prediction results of the hybrid model in the first experimental are demonstrated in Fig. 4. In order to show the details more clearly, we plotted the results of five days’ electrical load forecasting. It can be seen from Fig. 4 that the proposed hybrid model has strong prediction ability and the predicted results are satisfactory.

Fig.4

The prediction results of five days from the hybrid model in the first experiment.

In order to examine the superiority of the hybrid model, the values of the four indices in the six prediction models in this experiment are listed in Table 2. Further, the box-plots of the absolute prediction errors in this experiment are demonstrated in Fig. 5.

Table 2

The comparison values of the six models in the first experiment.

	Hybrid Model	SAE	ELM	BPNN	MLR	SVR
MAE	1.0006	1.0353	1.3354	1.9645	2.1873	1.5022
RMSE	1.6577	1.7337	2.1059	3.1442	3.4874	2.8599
MRE (%)	4.6415	4.6500	5.8606	8.1667	9.1902	6.2103
MSPE(%)	45.18	50.45	77.41	132.66	142.92	112.68

Fig.5

Box-plots of the absolute prediction errors in the first experiment, (a) Hybrid model, (b) SAE, (c) ELM, (d) BPNN, (e) MLR, (f) SVR.

4.4 Experiment 2

4.4.1 Applied data set

The electrical load data set in the second experiment was downloaded from the website: https://www.aeso.ca. This historical electrical load dataset was collected by the Albert Electric System Operator (AESO) and provided for market participants. The data sampling period was one hour, including 8760 samples from January 1, 2016 to December 31, 2016. This experiment divides the data into two parts: the training set and the testing set. The data samples from January 1, 2016 to November 31, 2016 are used for training while the data samples in December, 2016 are for testing.

4.4.2 Experimental setting

In this experiment, the PACF was also adopted to determine the input variables for electrical load prediction and 23 input variables were selected.

In this case, to determine the optimal or suboptimal structure of the SAE, the number of its hidden layers was also tested from 2 to 5, while the number of hidden units in each hidden layer was chosen from 50, 100, 150, 200, 250, 300, 350, 400, 450, 500. Consequently, 40 trials are given.

Similarly, the values of the MAE, MRE, RMSE and MSPE of the SAE model in the 40 trials in this experiment are computed and listed in Table 3. From Table 3, we can observe that when the number of hidden layers is 3 and the number of hidden units in each hidden layer is 300, the SAE model can achieve the best performance. For the ELM in the hybrid model, the number of neuron nodes was set to be 100.

Table 3
The values of the four indices of the SAE model in the 40 trials in the second experiment.

h k MAE RMSE MRE(%) MSPE(%) h k MAE RMSE MRE(%) MSPE(%)

2 50 112.4229 147.5348 1.0081 1.94 4 50 138.8388 188.2630 1.3316 3.24

2 100 55.2143 71.7175 0.5421 0.50 4 100 62.9499 80.4270 0.6148 0.61

2 150 47.5050 62.5705 0.4650 0.37 4 150 46.8131 60.4893 0.4607 0.36

2 200 48.7494 63.1448 0.4770 0.38 4 200 46.1867 60.1698 0.4525 0.35

2 250 44.3529 57.9975 0.4339 0.32 4 250 44.1617 56.9074 0.4338 0.31

2 300 48.8183 62.8551 0.4747 0.37 4 300 48.0509 60.1485 0.4699 0.34

2 350 48.6453 63.2069 0.4735 0.37 4 350 47.8686 62.6626 0.4653 0.36

2 400 44.8670 59.1052 0.4387 0.33 4 400 44.8611 58.5717 0.4394 0.33

2 450 45.0519 59.1931 0.4400 0.33 4 450 45.0751 59.4122 0.4395 0.33

2 500 45.4234 58.7219 0.4330 0.32 4 500 46.7091 61.2578 0.4557 0.35

3 50 133.8474 173.5295 1.2945 2.73 5 50 142.7713 186.4379 1.3910 3.24

3 100 52.3474 68.3183 0.5137 0.45 5 100 55.8387 73.2779 0.5462 0.51

3 150 48.4743 62.2657 0.4761 0.37 5 150 50.3351 65.0720 0.4923 0.40

3 200 51.2143 66.6275 0.5001 0.42 5 200 48.7521 63.3593 0.4778 0.38

3 250 45.1038 59.4786 0.4432 0.34 5 250 48.0814 62.3880 0.4690 0.36

3 300 42.6413 55.6614 0.4186 0.30 5 300 45.0798 58.1522 0.4408 0.32

3 350 50.2742 65.8387 0.4894 0.40 5 350 44.3952 58.2227 0.4362 0.33

3 400 44.2158 57.4908 0.4322 0.31 5 400 44.3795 57.4545 0.4340 0.31

3 450 45.5010 58.4403 0.4448 0.32 5 450 51.2085 67.6475 0.4975 0.42

3 500 41.1120 61.5276 0.4585 0.31 5 500 46.8722 60.0746 0.4577 0.34

h	k	MAE	RMSE	MRE(%)	MSPE(%)	h	k	MAE	RMSE	MRE(%)	MSPE(%)
2	50	112.4229	147.5348	1.0081	1.94	4	50	138.8388	188.2630	1.3316	3.24
2	100	55.2143	71.7175	0.5421	0.50	4	100	62.9499	80.4270	0.6148	0.61
2	150	47.5050	62.5705	0.4650	0.37	4	150	46.8131	60.4893	0.4607	0.36
2	200	48.7494	63.1448	0.4770	0.38	4	200	46.1867	60.1698	0.4525	0.35
2	250	44.3529	57.9975	0.4339	0.32	4	250	44.1617	56.9074	0.4338	0.31
2	300	48.8183	62.8551	0.4747	0.37	4	300	48.0509	60.1485	0.4699	0.34
2	350	48.6453	63.2069	0.4735	0.37	4	350	47.8686	62.6626	0.4653	0.36
2	400	44.8670	59.1052	0.4387	0.33	4	400	44.8611	58.5717	0.4394	0.33
2	450	45.0519	59.1931	0.4400	0.33	4	450	45.0751	59.4122	0.4395	0.33
2	500	45.4234	58.7219	0.4330	0.32	4	500	46.7091	61.2578	0.4557	0.35
3	50	133.8474	173.5295	1.2945	2.73	5	50	142.7713	186.4379	1.3910	3.24
3	100	52.3474	68.3183	0.5137	0.45	5	100	55.8387	73.2779	0.5462	0.51
3	150	48.4743	62.2657	0.4761	0.37	5	150	50.3351	65.0720	0.4923	0.40
3	200	51.2143	66.6275	0.5001	0.42	5	200	48.7521	63.3593	0.4778	0.38
3	250	45.1038	59.4786	0.4432	0.34	5	250	48.0814	62.3880	0.4690	0.36
3	300	42.6413	55.6614	0.4186	0.30	5	300	45.0798	58.1522	0.4408	0.32
3	350	50.2742	65.8387	0.4894	0.40	5	350	44.3952	58.2227	0.4362	0.33
3	400	44.2158	57.4908	0.4322	0.31	5	400	44.3795	57.4545	0.4340	0.31
3	450	45.5010	58.4403	0.4448	0.32	5	450	51.2085	67.6475	0.4975	0.42
3	500	41.1120	61.5276	0.4585	0.31	5	500	46.8722	60.0746	0.4577	0.34

For the BPNN, the number of hidden layer nodes and the iteration number were respectively set to be 200 and 15000. The same as in the first experiment, the sigmoid function and the gradient descent-based algorithm were used in its training process. For the SVR, the radial basis function was chosen as the kernel function and the penalty coefficient was set to be 120. Again, the parameter γ was set to be 0.001.

4.4.3 Experimental results

The prediction results of the hybrid model is demonstrated in Fig. 6. The comparison values of the six prediction models are shown in Table 4. From this table, we can see that the hybrid model has the smallest values of the four indices which imply that the accuracy and fitting ability of the hybrid model are better than the other five models. The box-plots of the absolute prediction errors in this experiment are also plotted and demonstrated in Fig. 7. From this figure, we can also observe that the proposed hybrid model has the better forecasting performance than the other predictors.

Fig.6

The prediction results of the hybrid model in the second experiment

Table 4

The comparison values of the six models in the second experiment.

	Hybrid Model	SAE	ELM	BPNN	MLR	SVR
MAE	40.0051	42.6413	46.0772	55.6481	59.9760	49.5572
RMSE	52.2701	55.6614	61.2546	74.5512	79.8772	65.7898
MRE (%)	0.3900	0.4186	0.4521	0.5422	0.5831	0.5138
MSPE (%)	0.26	0.30	0.36	0.52	0.60	0.42

4.5 Comparison analysis and discussion

From the above figures and tables, we can observe the following facts or make the following conclusions.

Fig. 4 and Fig. 6 demonstrated parts of the prediction results of the proposed hybrid model. From both figures, we can observe that the proposed hybrid model can give satisfactory prediction results. This verified the effectiveness of the proposed hybrid model.

In this study, we compared the proposed hybrid model with the SAE, ELM, BPNN,MLR, and SVR. The comparison values of four comparison indices of these six models in the first and second experiments are respectively listed in Table 2 and Table 4. From both tables, we can observe that the proposed hybrid model still performed best. Taking the comparison index MSPE for example, the prediction performance of the proposed hybrid model can improve at least 10% compared with the other comparative methods.

Fig. 5 and Fig. 7 depicts the box-plots of the absolute prediction errors of the six prediction models. In descriptive statistics, box-plot is a method for graphically depicting groups of numerical data through their quartiles. On each box, the central mark indicates the median, and the bottom and top edges of the box indicate the 25th and 75th percentiles, respectively. The whiskers extend to the most extreme data points not considered outliers, and the outliers are plotted individually using the ’+’ symbol. Thus, the box-plots in Fig. 5 and Fig. 7 can reflect the details of the distributions of the absolute prediction errors. Again, from such box-plots, we can see that the proposed hybrid model has the smallest prediction error median and the smallest heights between the 25th and 75th percentiles. This once again verified the advantages of the proposed method.

Fig.7

Box-plots of the absolute prediction errors in the second experiment: (a) Hybrid model, (b) SAE, (c) ELM, (d) BPNN, (e) MLR, (f) SVR.

5 Conclusions

Electrical load prediction plays a guiding role in power grid. In order to further improve the precision of electrical load prediction, a hybrid model was proposed in this paper. The proposed hybrid model combines the SAE and the ELMs to learn different layers of characteristics of the electrical load time series data. In addition, the proposed hybrid model was applied to predict two real-world electrical load time series. Further, in order to prove its superiority, the proposed model was compared with five popular artificial intelligence methods which are the SAE, ELM, BPNN, MLR and SVR. Experimental and comparison results demonstrated that the proposed hybrid model can achieve much better performance than the comparative methods in this electrical load forecasting application.

In this study, we only used the electrical load time series to realize the prediction. However, as well known, the electrical load may be affected by some important factors, such as the weather changing, the human activities, etc. Hence, it will be useful for further improving the prediction performance through incorporating such factors into our proposed hybrid model. This will be done in our future work. On the other hand, the proposed hybrid model can be applied to similar time series prediction problems, e.g. the traffic flow prediction, the wind speed estimation, and the stock market forecasting. This will also be one of our research directions.

Footnotes

Acknowledgment

This work was financially supported by the National Natural Science Foundation of China (61473176, 61573225), the Taishan Scholar Project of Shandong Province (TSQN201812092), and the Innovation Team of the Co-Innovation Center for Green Building of Shandong Province in Shandong Jianzhu University.

References

Sauer

P.W.

and Pai

M.A.

, Power system steady-state stability and the load-flow jacobian, IEEE Transactions on Power Systems 5(4) (1990)1374–1383.

Wang

and Zhang

, Adaptive finite time control of nonlinear systems under time-varying actuator failures, IEEE Transactions on Systems, Man, and Cybernetics: Systems, online first, Doi: 10.1109/TSMC.2018.2868329

, Yi

, Wang

, Zhang

and Li

, Interval data driven construction of shadowed sets with application to linguistic word modelling, Information Sciences, online first, Doi: 10.1016/j.ins.2018.11.018

Wang

, Chen

, Sun

and Lin

, Finite time control of switched stochastic nonlinear systems, Fuzzy Sets and Systems 365 (2019)140–152.

, Zhao

, Yu

, Lin

and Wu

, Fuzzy Finite-Time Command Filtered Control o Nonlinear Systems With Input Saturation, IEEE Transactions on Cybernetics 48(8) (2018)2378–2387.

, Shi

and Zhao

, Finite-time command filtered backstepping control for a class of nonlinear systems, Automatica 92 (2018)173–180.

, Shi

, Lin

and Yu

, Adaptive neural command filtering control for nonlinear MIMO systems with saturation input and unknown control direction, IEEE Transactions on Cybernetics, 2019, DOI: 10.1109/TCYB.2019.2901250

, Zhao

, Yu

and Lin

, Barrier Lyapunov functions-based command filtered output feedback control for full-state constrained nonlinear systems, Automatica 105 (2019)71–79.

Wang

, Li

, Niu

and Tan

, An annual load forecasting model based on support vector regression with differential evolution algorithm, Applied Energy 94(6) (2012)65–70.

10.

Dudek

, Short-term load forecasting using random forests, in: Intelligent Systems 2014, Springer, 2014, pp. 821–828.

11.

, Liu

, Li

, Wang

and Lu

, Short-term power load probability density forecasting method using kernel-based support vector quantile regression and copula theory, Applied Energy 185 (2017), 254–266.

12.

Niu

, Ma

and Liu

, Power load forecasting by wavelet least squares support vector machine with improved fruit fly optimization algorithm, Journal of Combinatorial Optimization 33(3) (2016), 1–22.

13.

, Li

and Liu

, A least squares support vector machine model optimized by moth-flame optimization algorithm for annual power load forecasting, Applied Intelligence 45(4) (2016), 1–13.

14.

Chen

, Luh

P.B.

, Guan

, Zhao

, Michel

L.D.

, Coolbeth

M.A.

, Friedland

P.B.

and Rourke

S.J.

, Shortterm load forecasting: Similar day-based wavelet neural networks, IEEE Transactions on Power Systems 25(1) (2010), 322–330.

15.

Guan

, Luh

P.B.

, Michel

L.D.

and Chi

, Hybrid kalman filters for very short-term load forecasting and prediction interval estimation, IEEE Transactions on Power Systems 28(4) (2013), 3806–3817.

16.

Zjavka

and Snášel

, Short-term power load forecasting with ordinary differential equation substitutions of polynomial networks, Electric Power Systems Research 137 (2016), 113–123.

17.

Amjady

and Keynia

, A new neural network approach to short term load forecasting of electrical power systems, Energies 4(3) (2011), 488–503.

18.

Sun

, Ouyang

and Yue

, Short-term load forecasting model based on multi-label and bpnn, in: Advanced Computational Methods in Life System Modeling and Simulation, Springer, 2017, pp. 263–272.

19.

Hanmandlu

and Chauhan

B.K.

, Load forecasting using hybrid models, IEEE Transactions on Power Systems 26(1) (2011), 20–29.

20.

, Ren

and Sun

, Short-term load forecasting based on bayes and svr, in: International Conference on Intelligent Robotics and Applications, Springer, 2008, pp. 643–650.

21.

Huang

C.-M.

, Yang

H.-T.

, Huang

Y.-C.

and Huang

K.-Y.

, One-day-ahead hourly load forecasting of smart building using a hybrid approach, in: Proceedings of the 2nd International Conference on Intelligent Technologies and Engineering Systems (ICITES2013), Springer, 2014, pp. 453–460.

22.

Bao

, Lin

, Gong

and Shao

, Hybrid short-term load forecasting using principal component analysis and mea-elman network, in: International Conference on Intelligent Computing, Springer, 2016, pp. 671–683.

23.

Liu

, Tang

, Zhang

, Fan

and Liu

, A hybrid forecasting model with parameter optimization for short-term load forecasting of micro-grids, Applied Energy 129 (2014), 336–345.

24.

LeCun

, Bengio

and Hinton

, Deep learning, Nature 521(7553) (2015), 436.

25.

Schmidhuber

, Deep learning in neural networks: An overview, Neural Networks 61 (2015), 85–117.

26.

Xue

, Kou

, Wang

and Liu

, Computational experiment research on the equalization-oriented service strategy in collaborative manufacturing, IEEE Transactions on Services Computing 11(2) (2018), 369–383.

27.

Xue

, Wang

, Zhang

, Feng

and Guo

, Social Learning Evolution (SLE): Computational Experiment-based Modeling Framework of Social Manufacturing, IEEE Transactions on Industrial Informatics 2018, Doi: 10.1109/TII.2018.2871167.

28.

Tian

, Li

, Zhang

and Lv

, Data driven parallel prediction of building energy consumption using generative adversarial nets, Energy and Buildings 186 (2019), 230–243.

29.

Zhang

, Zuo

, Chen

, Meng

and Zhang

, Beyond a gaussian denoiser: Residual learning of deep cnn for image denoising, IEEE Transactions on Image Processing 26(7) (2016), 3142–3155.

30.

, Chen

and Wu

, A recognition method of hand gesture with cnn-svm model, in: Bio-Inspired Computing-Theories and Applications, Springer, 2016, pp. 399–404.

31.

Wang

, Yang

, Mao

, Huang

and Xu

, Cnn-rnn: A unified framework for multi-label image classification, in: Computer Vision and Pattern Recognition, 2016, pp. 2285–2294.

32.

Zhao

and Du

, Spectral–spatial feature extraction for hyperspectral image classification: A dimension reduction and deep learning approach, IEEE Transactions on Geoscience and Remote Sensing 54(8) (2016), 4544–4554.

33.

Soutner

, Zelinka

and Müller

, On a hybrid nn/hmm speech recognition system with a rnn-based language model, in: International Conference on Speech and Computer, Springer, 2014, pp. 315–321.

34.

Hinton

, Deng

, Yu

, Dahl

G.E.

, Mohamed

A.-r.

, Jaitly

, Senior

, Vanhoucke

, Nguyen

, Sainath

T.N.

, et al., Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups, IEEE Signal Processing Magazine 29(6) (2012), 82–97.

35.

Kuta

, Morawiec

and Kitowski

, Sentiment analysis with tree-structured gated recurrent units, in: International Conference on Text, Speech, and Dialogue, Springer, 2017, pp. 74–82.

36.

Shamim Biswas

E.C.

and Ahmad

, Sentiment analysis with gated recurrent units, Computer Science and Information Technology 2(11) (2015), 59–62.

37.

Hirata

, Kuremoto

, Obayashi

, Mabu

and Kobayashi

, Time series prediction using dbn and arima, in: Computer Application Technologies (CCATS), 2015 International Conference on, IEEE, 2015, pp. 24–29.

38.

, Ding

, Zhao

, Yi

and Zhang

, Building energy consumption prediction: An extreme deep learning approach, Energies 10(10) (2017), 1525.

39.

Gao

, Zhu

H.-D.

, Gan

and Shang

, Extract features using stacked denoised autoencoder, in: International Conference on Intelligent Computing, Springer, 2014, pp. 10–14.

40.

Gehring

, Miao

, Metze

and Waibel

, Extracting deep bottleneck features using stacked autoencoders, in: IEEE International Conference on Acoustics, Speech and Signal Processing, 2013, pp. 3377–3381.

41.

Zhou

, Guo

and Wang

, Motion recognition by using a stacked autoencoder-based deep learning algorithm with smart phones, in: International Conference on Wireless Algorithms, Systems, and Applications, 2015, pp. 778–787.

42.

Shin

H.-C.

, Orton

M.R.

, Collins

D.J.

, Doran

S.J.

and Leach

M.O.

, Stacked autoencoders for unsupervised feature learning and multiple organ detection in a pilot study using 4d patient data, IEEE Transactions on Pattern Analysis and Machine Intelligence 35(8) (2013), 1930–1943.

43.

Vincent

, Larochelle

, Lajoie

, Bengio

and Manzagol

P.A.

, Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion, Journal of Machine Learning Research 11(12) (2010), 3371–3408.

44.

Zabalza

, Ren

, Zheng

, Zhao

, Qing

, Yang

, Du

and Marshall

, Novel segmented stacked autoencoder for effective dimensionality reduction and feature extraction in hyperspectral imaging, Neurocomputing 185(C) (2016), 1–10.

45.

Huang

G.B.

, Zhu

Q.Y.

and Siew

C.K.

, Extreme learning machine: Theory and applications, Neurocomputing 70(1) (2006), 489–501.

46.

Zong

and Huang

G.B.

, Learning to rank with extreme learning machine, Neural Processing Letters 39(2) (2014), 155–166.

47.

Huang

G.B.

, Zhou

, Ding

and Zhang

, Extreme learning machine for regression and multiclass classification, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) 42(2) (2012), 513–529.

48.

Quan

, Srinivasan

and Khosravi

, Short-term load and wind power forecasting using neural networkbased prediction intervals, IEEE transactions on Neural Networks and Learning Systems 25(2) (2014), 303–315.

49.

Yang

and Chai

, Data-driven demand forecasting method for fused magnesium furnaces, in: 12th World Congress on Intelligent Control and Automation (WCICA 2016), IEEE, 2016, pp. 2015–2022.

50.

Wang

, Zeng

and Chen

, Back propagation neural network with adaptive differential evolution algorithm for time series forecasting, Expert Systems with Applications 42(2) (2015), 855–863.

51.

Haviluddin and N. Dengen, Comparison of sarima, narx and bpnn models in forecasting time series data of network traffic, in: International Conference on Science in Information Technology, 2017, pp. 264–269.

52.

Ceperic

, Ceperic

and Baric

, A strategy for shortterm load forecasting by support vector regression machines, IEEE Transactions on Power Systems 28(4) (2013), 4356–4364.

53.

Sehgal

, Tiwari

M.K.

and Chatterjee

, Wavelet bootstrap multiple linear regression based hybrid modeling for daily river discharge forecasting, Water Resources Management 28(10) (2014), 2793–2811.

54.

Abuella

and Chowdhury

, Solar power probabilistic forecasting by using multiple linear regression analysis, in: SoutheastCon 2015, IEEE, 2015, pp. 1–5.