Optimal weight support vector regression ensemble with cluster-based subsampling for electricity price forecasting

Abstract

Accurate prediction of short-term electricity price is the key to obtain economic benefit and also an important index of power system planning and management. Support vector regression (SVR) based ensemble works have gained remarkable achievements in terms of high accuracy and steady performance, but they are highly dependent on data representativeness and have a high computational complexity O (k * N ³) of data samples and parameter selection. To further improve the data representativeness and reduce its computational complexity, this paper develops a new approach to forecast electricity price via optimal weighted ensemble. In the model, the cluster-based subsampling algorithm is proposed to categorize the inputs being seasonally decomposed into several groups, and representative data are drawn from each group in a certain proportion to ensure that each subset trained with SVR has the same representativeness and features. Moreover, the optimal weighted combination method is presented to assign weights to the sub-SVRs to obtain the optimal support vector regression ensemble model (OWSSVRE). The experimental results show that the improved support vector regression ensemble model with the same features and representativeness of the subset has better performance in electricity price forecasting. As a result, it is suitable to support decision making in the energy and other sectors.

Keywords

Electricity price forecasting support vector regression K-means clustering optimal weight subsampling

1 Introduction

Accurate electricity price forecasting is the common concern to all the power participants, and it can provide guidance for power suppliers, consumers, and market regulators.

1.1 Background

Affected by the global economic recovery, monetary easing and carbon-neutral commitment, global commodity prices and energy prices had continued to rise to large extent in 2021, and electricity prices in major EU economies had risen sharply. At present, the whole of Europe is in the dilemma of “electricity shortage”. Driven by this situation, the cost of electricity consumption for most market participants has increased significantly. For a long time, electricity price is not only a tool to maintain social production balance, but also the key to each market participant’s economic benefits. The rise of electricity price has caused certain fluctuations in the market balance. In order to minimize the effects of the fluctuations, market participants use the electricity price forecasting model to forecast the electricity price to expect different benefits.

Europe and the United States have always been a deeply market-oriented electricity market. The essence of electricity is a commodity, and its price is constrained by the relationship between supply and demand, which conforms to the law of market as well as the most basic requirements of forecasting. For power generation companies, electricity price forecasts help to formulate optimal bidding strategies to maximize benefits and minimize costs.

Therefore, it is very promising to steadily forecast short-term electricity price data to help market participants make cost strategies, electricity consumption plans and other decisions. However, extreme volatility makes electricity price forecasting more complex than other commodity prices, and a study of annualized volatility of commodities shows that electricity spot prices have volatility of up to 200%, far exceeding that of oil (30%), natural gas (50%) and electricity (60%) ([1]). In addition, electricity cannot be stored for a long time. These two factors lead to the high volatility and non-stationary, multi-seasonal and nonlinear characteristics of electricity prices. To cope with the multi-feature superimposed nature of electricity prices, a stable and accurate method needs to be developed to forecast electricity prices.

1.2 Discussion on electricity price forecasting

Many forecasting methods have been put into use in electricity price forecasting over the years, and there is no accurate answer as to which method is more appropriate. The forecasting methods can be broadly classified into two categories. One is the causal prediction method, such as the differential equation model, which looks for the causal relationship between the independent variable and the dependent variable, and is suitable for short, medium and long-term prediction. It can not only reflect internal laws and the internal relationship of things, but also analyze the correlation between two factors. The other is the time series forecasting method that predicts the future development trend of the market through statistical analysis using historical data according to the continuity law of objective things. As mentioned above, electricity prices are affected by many complex factors, and it is difficult to determine in real life. It is a very challenging task to establish an accurate forecasting model. Therefore, the research on electricity price forecasting is more inclined to electricity price forecasting based on time series forecasting method. There are two most commonly used time series forecasting methods: statistical models and artificial intelligence (AI) models.

In the traditional statistical model, the widely used models mainly deal with electricity price data characteristics modeling, including seasonality, strong mean regression, time variation moments and price fluctuations. Swider and Weber [2], and Bowden and Payne [3] studied the forecasting ability of the extended ARMA electricity price model and three time series models based on ARIMA electricity price forecasting separately. Their experimental results show that these models significantly improve the representation of random price processes and are more suitable for estimating seasonal and cyclical price trends in electricity markets. Inglesi-Lotz [4] used Kalman filter to estimate the price elasticity of electricity in South Africa during 1980-2005 and concluded that the higher the price of electricity, the higher the sensitivity of consumers to price fluctuations. For volatility modeling, Ioannidis et al. [5] proposed the GARCH periodic electricity price model with conditional skewness and kurtosis components, by using the empirical results of hourly wholesale prices in the German electricity market in 10 years.

In addition to modeling the structural characteristics of electricity prices, scholars believe that the higher accuracy prediction of electricity price could reduce the adverse impact of electricity crisis on the economy by fully utilizing AI modeling in recent years. Anbazhagan and Kumarappan [6] utilized a one-dimensional discrete cosine transform input feature feedforward neural network (DCT-FFNN) for electricity price forecasting and validated the effectiveness of the method by forecasting electricity prices in the New York electricity market in 2010. Yang et al. [7] combined wavelet transform, kernel-based ELM optimized by self-adaptive particle swarm optimization (SAPSO) to forecast electricity prices. Evaluated by using electricity price data for the Pennsylvania-New Jersey-Maryland, Australia and Spain markets, the hybrid model is shown to be of better versatility and practicability. By leveraging the advantages in capturing bidirectional data features, Shao et al. [8] established the BiLSTM forecasting model to predict the fluctuations of electricity price and demonstrated the superiority of the proposed model including high stability, accuracy and adaptability.

As mentioned earlier, electricity prices are affected by a series of complex factors, and it can no longer meet the requirements to improve the accuracy of electricity price prediction by relying solely on traditional model or artificial intelligence model simply. Therefore, more and more optimization strategy based forecasting models are now coming up with new ideas to reduce model forecast errors. Combining the data mining method to enhance feature selection, Ghayekhloo et al. [9] applied a recurrent neural networks model trained using variation Bayes to predict the daily price of electricity in the power market. Experimental results show that the proposed model has a better performance than other forecasting models. Gollou et al. [10] proposed a prediction model combining a two-step approach feature selection, wavelet transform and hybrid prediction based on neural networks and optimisation algorithms. An empirical study on the PJM electricity market illustrates the effectiveness of the proposed strategy. To improve the forecasting performance, Yang et al. [11] developed a framework for forecasting electricity price, which integrated the adaptive parameter-based variational mode decomposition technology, optimal kernel-based extreme learning machine models optimized by leave-one-out optimization strategy and the optimal model selection strategy, and achieved the best performance compared with the popular traditional statistical models and computational intelligence models.

Each single prediction model has certain unique advantages in characterizing local data features and hence the improvement degree of an individual model is limited. How to combine the advantages of multiple different individual models has become a widely concerned issue in time series prediction. Combining the extreme learning machine optimized by the differential evolution algorithm, Zhang et al. [12] introduced the concept of ensemble learning to reconstruct the weights of the predicted components. The approach addressed the limitation of equal-weight reconstruction prediction and demonstrated strong performance. Inspired by the ability of each individual model to characterize some local data features, Meng and Xu [13] proposed a support vector regression ensemble model based on an ant colony optimization algorithm to predict pricing mechanisms in a cloud manufacturing environment. To obtain an accurate and timely forecast of skin lesions, Kalpana et al. [14] proposed a support vector regression ensemble model with hybrid intelligent optimization. The experimental results of these studies show that the ensemble method based on support vector regression has better generalization performance, better mean square error and more reliable integrated learning stochastic results. In general, ensemble models based on support vector regression (SVR) have remarkable capabilities in terms of high accuracy and stable performance. However, the ensemble models have a high computational complexity O (k * N³) with the sample N and ensemble size k.

Subsampling [15, 16] is an efficient scheme to reduce the computational complexity of ensemble model. However, we find that the representativeness of the subsamples and the difference of prediction ability among sub-models are not taken into account in current subsampled ensemble models. Our proposed model well copes with these two issues. In this paper, we propose an optimal weight support vector regression ensemble model with cluster-based subsampling (KMC-OWSSVRE). Specifically, the feature data is first categorized into several clusters separately using K-means clustering. Secondly, to make subsampling be representative, cluster-based subsampling is adopted to produce a series of subsamples. Thirdly, based on the subsamples drawn from clusters, a series support vector regression models are derived with particle swarm optimization algorithm. Lastly, all the sub-models are combined to generate an ensemble model. Owing to the fact that different sub-models may have different prediction performances, different weights should be assigned to different sub-models in the ensemble model. We adopt an optimal weight strategy in the least squares sense to form a final ensemble model. To verify the validity of the model, we test it using the electricity price data of New South Wales and Queensland, respectively. To fully utilize the features of the data set, seasonal decomposition is performed to generate trend-cycle component, seasonal component and residual component owing to the periodicity of the electricity price. Then KMC-OWSSVRE model are applied to model the trend-cycle component and residual component. The results demonstrate that KMC-OWSSVRE model is of high accuracy and robust performance. The framework of the electricity price forecasting model is roughly shown in Fig. 1.

Fig. 1

Framework flow chart.

1.3 Our contribution

The main contributions of the research are featured in the following aspects:

1.
This paper constructs a new approach to forecast electricity price via an optimal weight ensemble model.
2.
A novel cluster-based subsampling algorithm is proposed to categorize the inputs being seasonally decomposed into several groups, and representative data are drawn from each group in a certain proportion to ensure that each subset trained with SVR has the similar representativeness and features.
3.
The seasonal decomposition method can reduce the noise of the inputs, and the K-means clustering method categorize the feature data into clusters with elements of similar features and representativeness, which can make the experimental samples enter the comfortable working range of SVR.
4.
The weights among sub-SVR models are optimized in the least squares sense to obtain the optimal ensemble model to reduce the prediction error.

The remainder sections are structured as follows. Section 2 describes the proposed methodology, highlighting its main differences in relation to recently developed ensemble methods for forecasting. Section 3 describes the data, performs data preprocessing and summarizes the results and assesses their implications. Finally, section 4 concludes this paper.
2 A framework for forecasting methods

This paper presents a framework for deep learning ensemble forecasting models with cluster-based subsampling based on seasonal decomposition and cluster sampling. In the ensemble model, K-means clustering is utilized to cluster the multi-dimensional input data. Secondly, cluster-based subsampling method is implemented to generate a series of subsamples. Thirdly, a series of support vector regression sub-models are constructed based on the subsamples extracted with K-means clustering. Finally, an optimal weight ensemble strategy is employed to form an ensemble model to improve the prediction efficiency.

2.1 Support vector regression

In the field of machine learning, there exist a large collection of forecasting algorithms. Among them, support vector regression (SVR) has received a large number of researchers’ attention due to its remarkable performance. In this paper, we only consider the ∈-insensitive support vector regression (∈-SVR). Assume a given data set ${x_{i}, y_{i}}_{i = 1}^{N}$ , where x_i is the input sample and y_i is the real-value output. In effect, ∈-SVR takes the following function form:

f (x) = \sum_{j = 0}^{n} ({\hat{α}}_{j} - α_{j}) K (x_{j}, x) + b,

(1)

where x is the input feature to be predicted, K (· , x) is the kernel function. The coefficients α = (α₁, ⋯ , α_n) and

\hat{α} = ({\hat{α}}_{1}, \dots, {\hat{α}}_{n})

can be determined by the following objective function with constraints:

\begin{matrix} min_{α, \hat{α}} \frac{1}{2} (α - \hat{α})^{T} K (α - \hat{α}) \\ + (α - \hat{α})^{T} y + \in (α + \hat{α})^{T} 1, \end{matrix}

(2)

s . t . {\begin{matrix} (α - \hat{α})^{T} 1 = 0, \\ 0 \leq α_{j} \leq C, j = 1, \dots, n, \\ 0 \leq \hat{α_{j}} \leq C, j = 1, \dots, n, \end{matrix}

(3)

where 1 is a vector of all ones,

K = (K_{ij})_{i, j = 1}^{n}

is a matrix whose elements K_ij = K (x_i, x_j). In this paper, Gaussian radial basis kernel is used.

K (x_{i}, x) = exp (\frac{- ‖ x - x_{i} ‖^{2}}{2 σ^{2}}),

(4)

where σ > 0 is the bandwidth of the Gaussian kernel.

From the objective function (2) and (3) conditions, we can see that the choice of hyperparameters ∈, C and σ immediately determines the performance of the SVR model. Therefore, it is essential to optimize the hyperparameters. A main approach to seek for sufficiently good parameters for SVR is to make use of optimization techniques, such as Genetic Algorithm (GA) [17], Grey Wolf Optimizer (GWO) [18], Particle Swarm Optimization (PSO) [19], and so on. In this study, PSO algoirhtm is explored to optimize the hyperparameters of SVR.

2.2 Optimal weight subsampled support vector regression ensemble

SVR is an effective tool in real case regression tasks. However, the space and time complexity of SVR are O (N²) and O (N³), respectively, where N is the number of training samples. Therefore, the traditional SVR is not suitable for large data set since it takes too much time and space to train. Thus, subsampling techniques can be utilized for accelerating the learning processes for large-scale tasks. In this work, we consider a subsampling technique based support vector regression ensemble (SSVRE) model. Specifically, we construct the SSVRE model from the convex combination of a series of subsampled support vector regression models:

f (x) = \sum_{i = 1}^{m} w_{i} f_{i} (x),

(5)

where w_i ≥ 0, (i = 1, 2, ⋯ , m) are the combination weights satisfying

\sum_{i = 1}^{m} w_{i} = 1

{f_{i} (x)}_{i = 1}^{m}

are the support vector regression models based on subsamples chosen from the original dataset:

f_{i} (x) = \sum_{j = 1}^{N_{0}} ({\hat{α}}_{i_{j}} - α_{i_{j}}) K (x_{i_{j}}, x) + b_{i},

(6)

where N₀ is the subsampled number and i_j (j = 1, 2, ⋯ , N₀) are the subscripts of the drawn data.

From Equation (5), we know the forecasting performance of SSVRE relies heavily on the weight distribution. If we simply employ the average weights ( $w_{i} = \frac{1}{m}$ , i = 1, ⋯ , m), then we get SSVRE model same as the one in Li et al. [16]. However, the average weight strategy ignores the differences among the subsampled models. In effect, different subsampled support vector regression models often have different prediction efficacy and can balance each other. Intuitively, subsampled models with better approximation should be placed greater weights in the combination. To improve the forecasting effect of the final SSVRE model, we optimize the weights in a least squares sense. That is,

min_{w_{1}, \dots, w_{m}} \sum_{j = 1}^{N} {[\sum_{i = 1}^{m} w_{i} f_{i} (x_{j}) - y_{j}]}^{2},

(7)

s . t . {\begin{matrix} \sum_{i = 1}^{m} w_{i} = 1, \\ w_{i} \geq 0, i = 1, \dots, m . \end{matrix}

(8)

Owing to the fact that the goal (7) is quadratic and the constraints (8) are linear, the optimization problem can be solved by quadratic programming algorithm. Based on the solution of the objective function with constraints, an optimal weight SSVRE model (OWSSVRE) is derived.

2.3 Cluster-based subsampling

When the data set is highly unbalanced, random sampling may lead to some subsamples unrepresentative and hence affects the performance of the ensemble model. To make subsamples be representative, cluster-based sampling technique [20] can be adopted. There are many ways to achieve data clustering with features, such as hierarchical clustering, K-means clustering, and so on.

2.3.1 K-means clustering

K-means clustering algorithm proposed by MacQueen [21] is a representative of fast clustering algorithm, which automatically categorize similar samples by an iterative clustering procedure. The basic idea of K-means clustering is to assign each sample to the nearest cluster. Specifically, for given data set X = {x₁, x₂, . . . , x_N} and cluster number k, K-means clustering is to minimize the sum of squares of errors (SSE) defined as follows:

\begin{matrix} SSE (Π_{k} | X, k) = \\ arg min_{Π = {C_{1}, C_{2}, \dots, C_{k}}} \sum_{i = 1}^{k} \sum_{x_{j} \in C_{i}} {∥ x_{j} - m_{i} ∥}^{2}, \end{matrix}

(9)

where ∥· ∥ is Euclidean distance, Π = {C₁, C₂, ⋯ , C_k} is the clustering result and m_i is the centroid of cluster C_i (i = 1, 2, ⋯ , k). Its procedure with k clusters can be simplified as follows:

(1)

Divide the samples into k initial clusters.

(2)

Assign each sample to the nearest cluster to the center based on Euclidean distance.

(3)

Recalculate each cluster center based on the following equation.

m_{i} = \frac{1}{| C_{i} |} \sum_{x_{j} \in C_{i}} x_{j}, i = 1, 2, \dots, k,

(10)

where |C_i| denotes the number of samples belonging to C_i.

(4)

Repeat steps (2) and (3) until all the samples can no longer be assigned.

2.3.2 Selection of the number of clusters

The above procedure shows how K-means clustering algorithm works with a given cluster number. However, it does not indicate how to determine the number of cluster. In essence, the number of cluster is an intrinsic nature and has nothing to do with the clustering algorithm. In clustering, Elbow method and Silhouette Coefficient are two most commonly used approaches to determine the proper cluster number for a given data set. Elbow method adopts SSE defined in Equation (9) as the criterion to choose the best cluster number. The core idea is that when the cluster number k increases, the degree of aggregation of each cluster gradually increases and hence SSE becomes progressively small. As k is less than the true value, the drop in SSE

Δ SSE (k, k + 1) = SSE (Π_{k} | X, k) - SSE (Π_{k + 1} | X, k + 1)

(11)

will be large. As k is above the true value, however, the decline ΔSSE (k, k + 1) will slow down because the degree of aggregation within each cluster becomes small. Therefore, the knee point of the SSE curve in cluster number is a good candidate of the number of cluster.

Silhouette coefficient determines the best cluster number based on the cohesion and separation of the clustering. Specifically, for a sample x_i belonging to C_p, its intra-class dissimilarity is measured as follows:

a (i) = \frac{1}{| C_{p} | - 1} \sum_{x_{j} \in C_{p}} ∥ x_{i} - x_{j} ∥ .

(12)

The inter-class dissimilarity of x_i is measured as follows:

b (i) = min_{q \neq p} \frac{1}{| C_{q} |} \sum_{x_{j} \in C_{q}} ∥ x_{i} - x_{j} ∥ .

(13)

Based on the intra-class dissimilarity a (i) and the inter-class dissimilarity b (i), the Silhouette coefficient of the sample x_i can be defined as follows:

s (i) = \frac{b (i) - a (i)}{max {a (i), b (i)}} .

(14)

As we can see, s (i) ranges from -1 to 1 and the clustering of x_i is reasonable if s (i) approaches to 1.

Calculating and averaging the Silhouette coefficients of all the samples, we derive the Silhouette coefficient score of the clustering results. Generally, the best cluster number is determined according to the largest Silhouette coefficient. However, an optimal clustering number resulting from Silhouette coefficient sometimes results in a clustering with large SSE. Therefore, the SSE of the clustering should be taken into account to determine the optimal number of clustering in practice.

2.3.3 Cluster-based subsampling

After clustering, the rest question for cluster-based subsampling is how to allocate sample sizes among clusters for a given sampling size n. In principle, there are at least two factors needed to be considered in sample allocation. On the one hand, the larger objects the cluster contains, the more objects should be selected from the cluster. On the other hand, a cluster with larger variation should be sampled more objects so that more features of the cluster can be drawn. To minimize the variance of the subsamples sampling from clusters subject to the constraint $\sum_{i = 1}^{k} n_{i} = n$ , we solve the following sample allocation problem ([22]):

min \sum_{i = 1}^{k} \frac{W_{i}^{2} σ_{i}^{2}}{n_{i}},

(15)

s . t . \sum_{i = 1}^{k} n_{i} = n,

where σ_i represents the standard deviation of the ith cluster, W_i = N_i/N is the proportion of objects in the ith cluster, N is the number of all the samples and N_i is the number of samples in the ith cluster. By the Lagrange multiplier method, an optimal sample allocation strategy is derived as follows:

n_{i} = \frac{{nW}_{i} σ_{i}}{\sum_{i = 1}^{k} W_{i} σ_{i}}, i = 1, 2, \dots, k .

(16)

In practice, all the sample numbers should be integers and hence we have

n_{i} = [\frac{{nW}_{i} σ_{i}}{\sum_{i = 1}^{k} W_{i} σ_{i}}], i = 1, 2, \dots, k,

(17)

where [·] is the rounding function.

To sum up, to make the objects sampled more representative, we first adopt Elbow method and Silhouette coefficient method to determine the optimal number of clustering k. Then, a series of subsamples are formed by subsampling according to the optimal sample allocation scheme (17).

2.4 KMC-OWSSVRE Model

To improve the forecast accuracy as much as possible, we propose an optimal weight subsampled support vector regression ensemble model integrating K-means clustering (KMC-OWSSVRE). We first determine the cluster number according to Elbow method and use K-means clustering method to categorize the dataset into k clusters. Secondly, subsampling strategy is adopted to form a series of representative subsamples of fixed sample size. Thirdly, a series of subsampled SVR models are built optimized by PSO algorithm. Finally, all the subsampled SVR models are combined with optimal weights to form an optimal weight ensemble model. Specifically, the algorithm of KMC-OWSSVRE model is listed in Algorithm 1.

3 Empirical analysis

To verify the superiority of the KMC-OWSSVRE model, we use half hour electricity price and electricity load data from New South Wales and Queensland respectively for electricity price forecasting. Figure 2 shows half hour electricity prices and electricity loads in New South Wales from May 1, 2007 to June 1, 2007. Figure 3 shows half hour electricity prices and electricity loads in Queensland from August 1, 2007 to September 1, 2007. All results of this study were obtained in R4.1.2 and performed on a laptop with Intel Core(TM) i5-6300HQ CPU@2.30 GHz ×4, 8 GB of RAM.

Fig. 2

Half-hour electricity price and electric load of New South Wales in Australia from May 1, 2007 to June 1, 2007.

Fig. 3

Half-hour electricity price and electric load of Queensland in Australia from August 1, 2007 to September 1, 2007.

3.1 Evaluation criteria

We have used root mean square error (RMSE), mean absolute error (MAE) and mean absolute percentage error (MAPE) in our research experiments to evaluate the performance of the proposed algorithm.

RMSE = \sqrt{\frac{\sum_{i = 1}^{n} (P_{i} - A_{i})^{2}}{N}},

(18)

MAE = \frac{1}{N} \sum_{i = 1}^{N} | (P_{i} - A_{i}) |,

(19)

MAPE = \frac{1}{n} \sum_{i = 1}^{n} | \frac{y_{i} - {\hat{y}}_{i}}{y_{i}} | \times 100 %,

(20)

where A_i and P_i are the i-th real data and the corresponding prediction, respectively, N is the sample size.

3.2 Data preprocessing

Before modelling with stastistical model, we need to implement data preprocessing for the data set including seasonal decomposition, phase space reconstruction and normalization.

3.2.1 Seasonal decomposition

From top panel of Figs. 2 and 3, we see that the data are characteristics of certain business cycle. Intuitively, the data are sampled half-hourly and hence follow certain regularity. Therefore, seasonal decomposition algorithms can be used to remove the seasonality of the data. Classical seasonal decomposition has an important place in time series analysis. However, it also has some drawbacks, such as the inability to derive moving average for the very beginning and last part of the data, the defect that the period is not allowed to change over time, and so on. Dagum and Bianconcini [23] proposed a modified X-11 decomposition to overcome some of the shortcomings of the classical seasonal decomposition by adjusting seasonally for quarterly and monthly time series data.

The additive X-11 decomposition can be expressed as:

Y_{t} = {TC}_{t} + S_{t} + I_{t},

(21)

where Y_t represents the original series, TC_t, S_t and I_t represent the trend, the seasonality and the residuals, respectively. In this paper, the additive X-11 decomposition method is used to decompose the electricity price data of New South Wales and Queensland into the seasonal component, the trend-cycle component and the residual component, which are depicted in Fig. 4 and Fig. 5, respectively.

Fig. 4

Three components based on seasonal decomposition of electricity price data of New South Wales in Australia.

Fig. 5

Three components based on seasonal decomposition of electricity price data of Queensland in Australia.

3.2.2 Phase space reconstruction

After seasonal decomposition, three time series, trend-cycle component ${{trend}_{t}}_{t = 1}^{N}$ , residual component ${{resid}_{t}}_{t = 1}^{N}$ and electric load ${{load}_{t}}_{t = 1}^{N}$ , can be used to predict future electricity price. To better reveal features of electricity data and improve the prediction performance, we employ phase space reconstruction from three times series to generate multi-dimensional input data for later modelling:

x_t = (trend_t , resid_t , load_t )

\begin{matrix} x_{t} = ({trend}_{t}, {resid}_{t}, {load}_{t}) \\ t = 1, 2, \dots, N - d, \end{matrix}

(22)

where

\begin{matrix} {trend}_{t} & = ({trend}_{t}, {trend}_{t + 1}, \dots, {trend}_{t + d - 1}), \\ {resid}_{t} & = ({resid}_{t}, {resid}_{t + 1}, \dots, {resid}_{t + d - 1}), \\ {load}_{t} & = ({load}_{t}, {load}_{t + 1}, \dots, {load}_{t + d - 1}), \end{matrix}

where d is the embedding dimension of trend_t, resid_t, load_t. Therefore, x_t is of dimension 3 × d.

With multi-dimension x_t data in hand, statistical model can be constructed to predict future trend-cycle component y_1,t = trend_t+d, y_2,t = resid_t+d, respectively. Add up the prediction of trend-cycle component ${\hat{y}}_{1, t}$ , the prediction of residual component ${\hat{y}}_{2, t}$ and the seasonal component, then the prediction of electricity price is derived.

In the paper, we simply set d = 3. Hence, the input data ${x_{t}}_{t = 1}^{N - d}$ is of dimension 9.

3.2.3 Normalization

Before the construction of data modeling, it is crucial to perform the standardized data preprocessing. In this paper, Z-score standardization is used to make each element of the feature data calibrate to zero mean and unit variance. That is, for each x, it can be scaled as follows:

x^{'} = \frac{x - μ}{σ},

(23)

where μ and σ are the mean and standard deviation of x, respectively. Based on the built forecasting model, the prediction data can be restored to true value by inverse normalization:

x = σ x^{'} + μ

(24)

3.3 Parameter selection

3.3.1 Case 1: Electricity price data of New South Wales

In the experiment, we randomly divide the dataset into a training set and a test set. The training set consists of 90% of the data, while the test set contains the remaining 10% . The training set is used to determine the parameters for our experiment. In KMC-OWSSVRE model, three parameters, the number of clusters k, the size of subsample n and the number of subsamples B, need to be determined. As can be seen from the left panel of Fig. 6, the knee point of the SSE curve indicates that the optimal cluster number k is 3. From the right panel of Fig. 6, the Silhouette coefficient maximizes at k = 2. However, the SSE at k = 2 is large and hence the secondary maximum point k = 3 is adopted.

Figure 7 illustrates the performance of our KMC-OWSSVRE model with different ensemble size B when the subsample size n = 120. We can see that RMSE declines sharply when the ensemble size B is less than 20 and then decreases slowly as B increases larger than 20. For mean absolute error (MAE), it shows a slow decline and reaches stability when B is larger than 30. Therefore, we choose B = 30 in the experiment. Figure 8 depicts the performance of the KMC-OWSSVRE model with different subsample size under the fixed ensemble number 30. The results shows that RMSE, MAE and MAPE go down slowly as the subsample size n increases and nearly reach stability at 120. Therefore, we take n = 120 in our experiment.

Fig. 6

Left: The sum of squares of errors (SSE) in different cluster number k. Right: Silhouette coefficients in different cluster number k.

Fig. 7

For the New South Wales electricity price dataset, RMSE, MAE and MAPE with respect to ensemble size B at fixed subsample size n = 120.

Fig. 8

For the New South Wales electricity price dataset, RMSE, MAE and MAPE with respect to subsample size n at fixed ensemble size B = 30.

3.3.2 Case 2: Electricity price data of Queensland

Similarly, we randomly divide 90% of the Queensland data set into the training set and the rest into the test set. From the knee point of the SSE curve and different Silhouette coefficients in Fig. 9, we can conclude that the optimal cluster number k is 3.

Fig. 9

Left: The sum of squares of errors (SSE) in different cluster number k. Right: Silhouette coefficients in different cluster number k.

Fig. 10 shows the performance of our KMC-OWSSVRE model with different ensemble size B when the subsample size n takes 80. We can see that three error metrics decline speedily when the ensemble size B is less than 50 and then decreases slowly as B increases larger than 50. Therefore, B = 50 is an appropriate value for the ensemble size. Figure 11 depicts the performance of the KMC-OWSSVRE model with different subsample size at fixed ensemble number 50. The results shows that RMSE, MAE and MAPE decline as the subsample size n increases and nearly reach stability at 80. Therefore, we take n = 80 in our experiment.

Fig. 10

For the Queensland electricity price dataset, RMSE, MAE and MAPE with respect to ensemble size B at fixed subsample size n = 80.

Fig. 11

For the Queensland electricity price data set, RMSE, MAE and MAPE with respect to subsample size n at fixed ensemble size B = 50.

3.4 Model validation

Fig. 12 shows forecast results for half hour electricity prices in New South Wales and Queensland. Table 1 lists the prediction errors of KMC-OWSSVRE model on train set and test set. As we can see, KMC-OWSSVRE model performs better on the test set than train set in term of RMSE, MAE and MAPE. Therefore, it shows that KMC-OWSSVRE model has an outstanding generalization ability. Panel (a) and panel (b) in Fig. 13 depict the 95% confidence interval length of the KMC-OWSSVRE model for the dataset of New South Wales and Queensland, respectively. The relatively small interval length of the KMC-OWSSVRE model suggests that the model has robust performance.

Fig. 12

Electricity price prediction with KMC-OWSSVRE based on the data.

Table 1

Performance of KMC-OWSSVRE model on train set and test set

	Train set			Test set
Area	RMSE	MAE	MAPE(%)	RMSE	MAE	MAPE(%)
New South Wales	10.3643	6.2660	9.90	7.6216	5.5091	9.15
Queensland	4.2588	2.7281	7.13	3.5794	2.7081	7.07

Fig. 13

95% confidence interval length of electricity price prediction with KMC-OWSSVRE.

Fig. 14 and Fig. 15 portray the performance of three models (SSVRE model, KMC-SSVRE model and KMC-OWSSVRE model) on the test data of New South Wales and Queensland, respectively. From the two figures, we can come to the following conclusions: (i) Compared the RMSEs, MAEs and MAPEs of KMC-SSVRE and SSVRE, we see that the K-means clustering algorithm makes the subsamples be more representative and hence contributes to enhancing the prediction ability. (ii) From the results of KMC-OWSSVRE and KMC-SSVRE, optimal weight ensemble strategy can further improve the prediction performance of the ensemble model. To sum up, the K-means clustering subsampling technique and optimal weight program can jointly lift the efficacy of SSVRE model.

Fig. 14

Comparison of prediction performance of KMC-OWSSVRE model, KMC-AWSSVRE model and traiditional SVR model based on New South Wales Electric Utility.

Fig. 15

Comparison of prediction performance of KMC-OWSSVRE model, KMC-AWSSVRE model and traiditional SVR model based on Queensland Electric Utility.

To further demonstrate the forecasting effectiveness of the KMC-OWSSVRE model, we conduct a comparison with a recent study [24] that utilizes the particle swarm optimization algorithm to optimize a BP neural network model (PSO-BP) for short-term power generation forecasting. The results, as shown in Table 2, indicate that the KMC-OWSSVRE model outperforms the PSO-BP model in terms of RMSE, MAE, and MAPE on the same dataset.

Table 2

Performance of KMC-OWSSVRE model and PSO-BP model on test set

	KCM-OWSSVRE			PSO-BP
Area	RMSE	MAE	MAPE(%)	RMSE	MAE	MAPE(%)
New South Wales	7.6216	5.5091	9.15	14.9856	8.2681	14.11
Queensland	3.5794	2.7081	7.07	4.8267	3.2287	8.64

4 Conclusions

Research on effective strategies for short-term electricity price prediction models is currently a hot issue. In this paper, a new effective strategy based electricity price prediction model KMC-OWSSVRE is proposed. In this model, we first explore seasonal decomposition and K-means clustering algorithms to reduce data noise and classify multidimensional the inputs into multiple clusters with similar features. Secondly, subsampling is implemented to create a series of representative subsamples. Thirdly, a series of sub-SVR models, optimized by the particle swarm optimization algorithm, are built using the subsamples obtained from clustering. Finally, an optimal weight strategy is applied to the sub-SVR models for generating the final ensemble model. To demonstrate the effectiveness of the KMC-OWSSVRE model, we conduct experiments using electricity price data of New South Wales and Queensland, respectively. The experimental results show that the model based on seasonal decomposition, clustering subsampling and optimal weighted combination strategy can effectively predict the electricity price series. Therefore, it holds great promise as a practical forecasting method for real electricity markets.

However, our model is not optimal and can be improved in several ways. For instance, some other popular decomposition techniques and deep learning algorithms can be applied to the framework to generate more models with better prediction accuracy and generalization ability. However, these methods were not included in the scope of this study. We aim to enhance the framework in both of these aspects in the near future. In addition, electricity price volatility forecasting will be included in our next extension.

Acknowledgments

The work is partially supported by the Jiangxi Provincial Humanities and Social Sciences Research Project (Grant No. JJ21215), National Natural Science Foundation of China(Grant No. 71971105, 12361096, 12161058)

References

Weron

, Electricity price forecasting: A review of the stateof-the-art with a look into the future, International Journal of Forecasting30(4) (2014), 10301081.

Derk Swider

Weber

Extended ARMA models for estimating price developments on day-ahead electricity markets, Electric Power Systems Research77(5-6) (2007), 583593.

Bowden

Payne

J.E.

, Short term forecasting of electricity prices for MISO hubs: Evidence from ARIMAEGARCH models, Energy Economics30(6) (2008), 31863197.

Inglesi-Lotz

, The evolution of price elasticity of electricity demand in South Africa: A Kalman filter application, Energy Policy39(6) (2011), 36903696.

Ioannidis

Kosmidou

Savva

Theodossiou

, Electricity pricing using a periodic GARCH model with conditional skewness and kurtosis components, Energy Economics95 (2021), 105110.

Anbazhagan

Kumarappan

, Day-ahead deregulated electricity market price forecasting using neural network input featured by dct, Energy Conversion and Management78 (2014), 711719.

Yang

Lian

, Electricity price forecasting by a hybrid model, combining wavelet transform, ARMA and Kernel-Based Extreme Learning Machine Methods. Applied Energy190 (2017), 291305.

Shao

Zheng

Liu

Gao

Wang

Chu

, A feature extraction-and ranking-based framework for electricity spot price forecasting using a hybrid deep neural network, Electric Power Systems Research200 (2021), 107453.

Ghayekhloo

Azimi

Ghofrani

Menhaj

M.B.

Shekari

, A combination approach based on a novel data clustering method and bayesian recurrent neural network for day-ahead price forecasting of electricity markets, Electric Power Systems Research168 (2019), 184199.

10.

Gollou

A.R.

Ghadimi

, A new feature selection and hybrid forecast engine for day-ahead price forecasting of electricity markets, Journal of Intelligent & Fuzzy Systems32(6) (2017), 40314045.

11.

Yang

Sun

Hao

Wang

, A novel machine learning-based electricity price forecasting model based on optimal model selection strategy, Energy238 (2022), 121989.

12.

Zhang

Tang

Chen

, Short term electricity price forecasting using a new hybrid model based on two-layer decomposition technique and ensemble learning, Electric Power Systems Research205 (2022), 107762.

13.

Meng

Q.N.

, Price forecasting using an aco-basedsupport vector regression ensemble in cloud manufacturing, Computers Industrial Engineering125 (2018), 171177.

14.

Kalpana

Reshmy

A.K.

Pandi

S.S.

Dhanasekaran

, Oesv-krf: Optimal ensemble support vector kernel random forest based early detection and classification of skin diseases, Biomedical Signal Processing and Control85 (2023), 104779.

15.

Politis

D.N.

Joseph Romano

Wolf

Subsampling. Springer Science & Business Media, 1999.

16.

Che

Yang

, Subsampled support vector regression ensemble for short term electric load forecasting, Energy164 (2018), 160170.

17.

Schmitt

L.M.

, Theory of genetic algorithms, Theoretical Computer Science259(1-2) (2001), 161.

18.

Smm

Grey wolf optimizer, Advances in Engineering Software (2014), pp. 4661.

19.

Kennedy

Eberhart

Particle swarm optimization. In Icnn95-international Conference on Neural Networks, 1995.

20.

Yen

S-J.

Lee

Y.-S.

Cluster-based sampling approaches to imbalanced data distributions. In Data Warehousing and Knowledge Discovery: 8th International Conference, DaWaK 2006, Krakow, Poland, September 4-8, 2006. Proceedings 8, pages 427436.Springer, 2006.

21.

MacQueen

Classification and analysis of multivariate observations. In 5th Berkeley Symp. Math. Statist. Probability, pp. 281297. University of California Los Angeles LA USA, 1967.

22.

Zhao

Liang

Dang

, A stratified sampling based clustering algorithm for large-scale data, Knowledge-Based Systems163(JAN.1) (2018), 416428.

23.

Dagum

E.B.

Bianconcini

Seasonal adjustment methods and real time trend-cycle estimation. Springer, 2016.

24.

Zhou

Gao

Yang

Han

Lian

, Shortterm power generation forecasting of a photovoltaic plant based on PSO-BP and GA-BP neural networks, Frontiers in Energy Research9 (2022), 824691.