Performance of the autoregressive integrated moving average model with exogenous variables statistical model on the intraday market for the Denmark-West bidding area

Abstract

This article aims to investigate whether a statistical model known as Autoregressive Integrated Moving Average with Explanatory Variables can aid better predictability of volume-weighted average electricity prices compared to a commonly used forecasting method. This analysis was conducted for a specific bidding area, the Denmark-West bidding area (DK1). Autoregressive integrated moving average model with exogenous variable's performance was tested on the DK1 intraday market over a two-year period starting from 1 January 2019 until 31 December 2020. An explanatory variable used to support better the accuracy of the forecast is the day-ahead price for a corresponding intraday delivery hour. To ensure the validity of the paper, a well-known forecasting methodology was applied, and the results of the analysis show superior performance over the benchmark forecasting method. The autoregressive integrated moving average model with exogenous variables model developed was found to significantly outperform other commonly used forecasting methods, with an average mean absolute percentage error of 1.5%. The model was able to accurately predict intraday volume-weighted average prices up to 24 h in advance, using only publicly available data on day-ahead prices and historical intraday prices. Energy traders and other market players may find the developed autoregressive integrated moving average model with exogenous variables model to be a useful resource when looking to make more informed decisions in the intraday market.

Keywords

Intraday electricity prices Denmark-West bidding area autoregressive integrated moving average model with exogenous variables

Introduction

Fossil fuels including coal, oil, and natural gas have historically been used extensively to meet global energy demands, raising worries about greenhouse gas emissions and climate change. However, switching to cleaner and more sustainable energy sources like solar, wind, hydro, and biomass is becoming more and more important. The patterns of energy consumption vary by geography, with developed countries consuming more energy per person due to higher levels of industrialization. The world's two biggest energy users are China and the US. In order to combat climate change and guarantee a sustainable future, improving energy efficiency and switching to greener energy sources are essential solutions.

Electricity is a mandatory element of contemporary society. Common household items, AC units, factory machines, and many more all use electricity as a source of energy. Some time ago, customers could not choose from whom to buy electricity to power all their devices. Centralized markets provide benefits for institutions that hold a monopoly over products or services offered on the market. Such market design enables the chosen enterprises security of a market share and in turn higher profits due to the lack of supply on the market. The introduction of a free market allows the consumer to attain the optimal ratio between value and cost. Market settings like this incentivize suppliers of products and services to compete for a finite market share by innovation, providing quality products and services at competitive prices. To even further support competitiveness in the marketplace, forecasting methods were introduced to the energy trading industry in order to better predict the price of electricity. These predictions are based on many factors such as weather conditions, load forecasts, and historical values. This implies that the better the market participant can forecast the price of MWh, the more optimized production and consumption schedules can be implemented¹ in order to reduce market unpredictability and minimize costs that will eventually be transferred to the consumer.

The day-ahead market was the first market to receive attention from academia and businesses when it comes to forecasting. In recent years, the intraday market is gaining popularity. It serves the purpose of correcting imbalances that are produced by trading in the day-ahead market. The intraday market also enables faster integration of variable renewable energy sources (RESs) as power producers and consumers have time to adjust their schedules closer to real-time. Being not as popular with academia and business as the day-ahead market, the intraday market had many unanswered questions lingering in the air. The most prominent one is related to price modeling; intraday prices for energy delivered for a certain hour can vary substantially, and the price of electricity is generally described as being intermittently volatile.² One way these uncertainties can be reduced, and the profitability of market participants increased is by better methods of forecasting. However, this is a complex field with many potential variables that affect the final price.

There are many examples of the forecasting method used for the day-ahead market, but only a handful for the intraday market. The aim of this article is to demonstrate the performance of a statistical model autoregressive integrated moving average with explanatory variables (ARIMAX) for Western Denmark and whether it can aid in better predictability of intraday prices compared to the standard methods used in the field. Therefore, the research question attempted to be answered by the end of the paper is:

How can the ARIMAX algorithm aid better in the predictability of intraday hourly electricity prices for the DK1 bidding area?

ARIMAX is a very robust model, and it has wide utilization across numerous industries so its performance in the intraday energy market is worth exploring. Although price forecasting is complex and may exhibit nonlinear behavior, statistical models have performed well in practical applications.³ Likewise, for this analysis, the results suggest that ARIMAX has the potential to be a useful tool in forecasting intraday prices.

The article consists of an introduction, market description, methodology, literature review, forecasting methodology, big data and trading analysis, time-series models, the performance of the model, and a conclusion section. The “Introduction” section aims to familiarize the reader with the relevant information on why this topic was chosen to be discussed and make known the gap in state-of-the-art when it comes to forecasting in the intraday market. “The market” section provides insight into general information regarding the background of electricity markets as well as market settings. The methodology elaborates on the research techniques used in this article. Forecasting methodology specifically describes steps needed to be taken in order to produce an outcome, whether a positive or a negative one. The literature review covers state-of-the-art sources on forecasting prices in intraday markets.⁴ The Big Data and trading analysis section explores the Danish West intraday electricity market as well as extensive data processing steps needed to prepare the data for further analysis. Time-series models elaborate on the theoretical background of ARIMAX. Furthermore, the proposed forecasting model has been discussed in detail as well as its performance. Lastly, the article concludes with a proposal for future work.

The market

This section serves as an introduction to the characteristics of the Nordic electricity market relevant to this article. It begins by familiarizing the reader with the important properties of electricity and how market conditions in the past transformed what we know today. The second part of this section distills the two trading products offered in Denmark, the day-ahead and the intraday market. It expands on their purpose in the marketplace and gives the background on features of interest, such as the mechanics of the market and how to design forecasting methodology.

Electricity and market history

Electricity is classified as a commodity, therefore, it is bought and sold on a market, similar to other commodities. Although electricity shares similarities with other commodities, markets for electricity differ from markets for other commodities. The way markets are set up today is very different from how they used to be. Before the liberalization of the electricity markets in Europe, governments and their agencies controlled the dynamics of the market.⁵ The electricity industry in Europe started its deregulatory mandate in Scandinavia where the electricity market was reshaped during the 1990s. One of the first countries to introduce market competition was Norway. Sweden followed Norway's Energy act, and later, in 1995, jointly formed what is now known as Nord Pool—the first electricity market that allowed the market to openly trade across borders. The Nordic electricity market was the most liquid market in the world due to its deregulated framework.⁶ The consequence of the aforementioned open market presented itself through the division between the generation and supply on one side, and the grid on the other side. The generation and the supply-side are targeting to increase their profits by producing at the highest price possible, whereas the grid is responsible to manage the frequency.⁶ In order to have the framework balanced, representative bodies were introduced to the Nordic Electricity Market as shown in Table 1.

Table 1.

Roles in the Nordic electricity market.

Main roles	Purpose
Regulator	Controlling the monopoly & trading in the physical and financial market
Market operator	Nord Pool-market place for the Nordic power market
Transmission system operator (TSO)	Owners of the national grid and responsible for alignment between consumers, producers, and other network owners
Network owner	Maintains and operates the network
Market players	Participants of the market consumers, producers, and traders
Retailers	Selling electricity to the user

Denmark joined the newly established energy market in Scandinavia in 2000 by offering energy trading through Nord Pool. A market operator like Nord Pool is a channel for providing competitiveness between supply and demand as it acts as an exchange where the best offers are accepted.⁷ Table 2 summarizes the studies, the tools, and the findings.

Table 2.

The previous studies, the models, and their contribution.

Study	Methodology	Findings
[1]	ARIMA model	Outperformed other models for day-ahead price forecasting in the PJM market
[2]	Neural network model	Achieved MAPE of 2.5% for day-ahead price forecasting in the EEX market
[3]	ARIMA model	Found that including exogenous variables improved the accuracy of day-ahead price forecasts in the B2B market
[4]	ARIMA model	Achieved MAPE of 1.8% for day-ahead price forecasting in the Nord Pool market
[5]	ARIMA model	Found that incorporating weather data improved the accuracy of day-ahead price forecasts in the Nord Pool market
[6]	ARIMA model	Outperformed other models for intraday price forecasting in the Nord Pool market
[7]	Neural network model	Neural network model \| Achieved MAPE of 2.5% for intraday price forecasting in the Nord Pool market

ARIMA: autoregressive integrated moving average model with exogenous variables; PJM: Pennsylvania–New Jersey–Maryland interconnection; MAPE: mean absolute percentage error; EEX: European energy exchange; B2B: business-to-business.

With the purpose of establishing a constant balance between supply and demand, deregulated electricity market exchanges provide multiple trading opportunities to match supply and demand in an effective manner.⁸ Having such a framework instituted enabled competitiveness, and therefore, helped to increase interconnectivity between countries.⁹ Nord Pool is a company owned by the Nordic Transmission System Operators from Denmark (Energienet.dk), Sweden (Svenska Kraftnat), Finland (Fingrid Oy), Norway (Statnett SF), and the Baltic Transmission System Operators (Elering, Augstsprieguma, and Litgrid). The Danish energy market is split into two regions. The two regions are DK1 and DK2. DK1 belongs to the West Denmark grid area, and it is encompassed by the Jutland peninsula, the island of Fyn, and the rest of the islands west of the Great Belt. DK2 represents the East Denmark grid area and covers land east of the island of Fyn. Figure 1 shows the geographical division between East Denmark and West Denmark areas. The region in Denmark marked in yellow corresponds to the DK1 grid area, meanwhile, the green-shaded area corresponds to the DK2 grid area. This article focuses on the DK1 area.

Figure 1.

The two Danish energy market regions.¹⁰.

Market structure in Denmark and product features

For the purpose of defining the scope of this article, it is important to clarify which electricity market will be analyzed. One might think that the day-ahead market is the market with the longest trading horizon, but, it is a financial market organized by Nasdaq Commodities. The financial market only refers to monetary settlements, hence, it does not deal with physical delivery and is used for risk management.¹¹ The next electricity market is the reserve capacity market. This market is important for the stability of the grid as it deals with power plants that have the available capacity to resolve variance in the grid. The electricity market that is of particular importance for this work encompasses a two-product group. It is comprised of the day-ahead market and the market that will be addressed, the intraday market. The two products, the day-ahead and the intraday can be seen in Table 3.

Table 3.

Available trading products on Nord Pool for Denmark-West bidding area (DK1).

Product	Description
Elspot	Day-ahead market prices settled by a marginal rule through auction
Elbas	Intraday market-continuous market trading and auctions

Additionally, the day-ahead market trades in consecutive hour format and is the main product being traded on Nord Pool. This can be seen in Figure 2 where a breakdown of activity between the two products is presented during the time period between 1 January 2019 and 31 December 2020. The average amount of MWh traded per year for the day-ahead market is 740,451,232 MWh whereas for the intraday it is 13,933,486 MWh. Therefore, the day-ahead market enjoys higher traded volumes by a factor of 53. Despite the fact that the intraday traded volume is much lower than for the day-ahead, it still plays a vital role in the market.

Figure 2.

Traded volume for day-ahead and intraday market.

Apart from it being the most popular product, the day-ahead market enjoys being the most researched topic in the electricity market. The interest is mainly coming from academia and companies, particularly in the forecasting category of research.¹² On the other hand, the intraday market consists of both auctions and continuous trading and is used to alleviate imbalances that occur after the day-ahead auction closes.¹³ The presence of the intraday market predominately targets to increase the share of renewable energy sources.¹² Renewable energy sources (RESs) like solar and wind are characterized by irregular power generation due to the reliance on contemporaneous weather conditions.¹⁴ Wind power is the fastest-growing electricity-generating technology in the renewable energy sources spectrum.¹ The energy produced from wind turbines amounts to ∼ 45% of the total energy production in Denmark in the previous year and that number is only expected to increase in the coming years.¹⁵ The growth of renewables was stipulated by the Danish government as the goal is to become independent from fossil fuels by 2050.⁹

However, the increased representation of RES by wind power has forced electricity markets to confront many challenges in renewable integration and system stability resulting from the high volatility and low predictability of wind generation.^1,16 Wind outputs highly depend on meteorological forecasts which may cause the schedule for wind production to deviate from the one planned in the day-ahead market.¹⁷ Closer to the hour of delivery, intraday trading allows RES generators to modify their day-ahead production schedules according to the updated forecasts after the closure of the day-ahead market. In turn, the intraday market is regarded as an important tool to handle intermittent wind RES and to accelerate its adoption into the electricity system. Moreover, Hu et al.¹⁸ claim that the increase in intermittent RES is positively correlated to the number of market participants in the intraday market.

Nord Pool covers 25 bidding areas but not all the areas are the same. The aggregate volume for intraday trades for all bidding areas in Nord Pool can be seen in Figure 3. Looking closely at the plot, an upward trend appears to be forming which signalizes that more and more market participants are interested in this market. Furthermore, the graph to the right of Figures 3 and 4 decomposes the aggregate trading volume into its dedicated regions. The plot points out that OPX is the main “player” on Nord Pool's exchange, but what cannot be seen in the plot is the traded volume on other exchanges but still for the same bidding areas as on Nord Pool, meaning, electricity can be traded in Poland but not seen on Nord Pool because it happened on a different exchange. Taking that into account, a high majority of traded volume for the DK1 bidding area still occurs on Nord Pool which implies that DK1 is well represented by data available on Nord Pool.

Figure 3.

Intraday volume traded on Nord Pool.

Figure 4.

Intraday volume traded on Nord Pool per bidding area.

The upward trend of traded volume aligns with the Hu et al.¹⁸ claim, although the number of market participants cannot be seen in this figure. The reason for the growth of market players in the intraday market is twofold.² The first reason is the fact that electricity is not economically viable for being stored. Its demand and supply must be always balanced and instantaneously be met at physical delivery time.¹⁹ RES generators have the ability to reset a generation agenda with a short lead time in the intraday market which will allow to benefit of possibly reducing the imbalance costs to which electricity consumers and producers are exposed when consuming or supplying more or less electricity than planned. Intraday trading can reduce balancing costs often caused by the irregularities in RES power generation.²⁰ Additionally, the merit order effect steps into play when it comes to RES as is it characterized by lowering of energy prices at the power exchange due to the increased supply of low marginal cost renewable energies, which shifts convectional expensive generation outside of the supply curve and yields lower clearing market prices.¹ Secondly, intraday trading allows for the market participants to optimize production and consumption schedules. This feature proves itself to be useful if, for example, a power plant is scheduled to produce according to the day-ahead market but can buy the amount of power it is scheduled to produce on the intraday market at a cheaper cost than it can produce. To further summarize, the aim of intraday trading is to enable market participants to improve positions placed in the day-ahead market following updates in the forecast with respect to the ones available for the day-ahead.¹

Day-ahead, the Elspot market as well as the intraday, Elbas market operate on Nord Pool exchange. In Figure 5, a timeline representing the order in which trading is conducted in Denmark was made. The scheme below shows successive trading starting with the day-ahead auction, continuing with the intraday trading interval, and ending with the imbalance market which settled after physical delivery. In Elspot, electricity contracts are settled one day prior to power delivery for the following day. The day is split into 24 h, and therefore, there are 24-hour contracts to be settled. For each hour, the seller approximates the amount of power to be delivered in the specific hours and at what price the seller is willing to deliver that volume.

Figure 5.

Day-ahead and intraday timeline.

Orders for power delivery/purchase or written differently, and bids for generation/consumption, are placed in the day-ahead trading system until 12:00 CET the day before.¹² The next step is performed by Nord Pool by setting the hourly system price using the marginal rule so the balance between demand and supply can be expected for each of the 24-hour periods. Additionally, both day-ahead and intraday market measures price in euros per megawatt-hour (EUR/MWh).

Elspot market participants’ settled trades constitute their obliged plan for consumption and production. Such trades are submitted to the different transmission system operators (TSOs). Once an order is accepted, the market participant is committed to either supply, consume, or both, corresponding to the submitted plan. This plan is allowed to be modified no later than 45 min before the start of the delivery hour. The deviations to the plans are resolved by the intraday or Elbas market since it allows market players to modify the production or consumption plan with the aim of minimizing market imbalances. As mentioned above, the intraday electricity market is a continuous market in Denmark, which means that it has much in common with a regular stock market, except participants can trade in Elbas 24 h a day, 365 days a year.¹² The intraday market starts at 14:00 CET, 2 h after the day-ahead market closes for the following day (Figure 5). At this point, Elbas market participants already have the information on which bids were accepted and what volume it is expected to be traded.

If the position on the intraday continuous market is not settled before 1 h of delivery, meaning both legs of a transaction (buy and sell), are not executed on the same day which results in leaving the net position holding at nonzero, the position will be settled on the imbalance market. Historically, the purpose of the imbalance market is to provide reserve and response operations. TSO contracts the capacity on the reserve and response side in the day-ahead and long-term markets in order to allow for flexibility if errors in forecasts of outages in power plants occur.²¹ Lately, wind power plants increased the demand for reserve and response capacity due to the uncertainty of the day-ahead forecasts for renewable sources.²¹

Kolberg and Waage question what the motivation for market participants is to get involved with Elbas as the motivation is linked to potential intraday price drivers.¹² The authors suggest that all market participants fall into one of the following two groups:

Participants can profit from buying or selling power in the intraday market and reallocate their own production or consumption plan but are not in a situation where balancing is necessary.

Participants must correct an imbalance in their production or consumption plan.

For the purpose of this article, the second category will be the main focus from the trader's perspective. This means that a trader can buy or sell power in Elbas after which the position can be closed on Elbas or balancing market. Entering the balancing market can be more profitable than settling the position on Elbas since the system price for up-regulation is set above the day-ahead price to incentivize additional production. On the other hand, the system price for down-regulation is set below the day-ahead price to incentivize producers with higher marginal costs to purchase power from producers instead of producing the power themselves.¹² The article will not focus on the possibly more profitable imbalance market as the imbalance price is highly dependent on the fundamentals which can be tough to accurately forecast. Therefore, the focus will be on reaching a net-zero position before the end of the trading period for Elbas.

Materials and methods

The research in this project is taking place using some well-defined methodologies. It is going to be founded on mixed methods. However, the quantitative approach will be the dominant one. The first tool used was a screening analysis. Screening analysis enabled easier selection of relevant sources.¹² Starting from philosophies, positivism was used as knowledge is independent of the studied subject. Deductive reasoning aims to develop a hypothesis based on known theories and then design research in order to consolidate it.²² A literature review was chosen for acquiring information and the analysis for the research is constituted of a mixed one, mostly quantitative and partly qualitative research method. This paper used qualitative approaches to explore existing theories and methodologies for forecasting intraday electricity prices. Quantitative research methods emphasize objective measurements and the statistical, mathematical, or numerical analysis of data collected through many different means, but in this case, it is done by manipulating pre-existing statistical data using computational techniques. The goal of conducting a quantitative research study is to determine the relationship between an independent variable, the day-ahead price, and a dependent or outcome variable, intraday hourly volume-weighted average price (VWAP). On the other hand, qualitative research provides a more human side of an issue which is sometimes contradictory to common views on behavior,²³ beliefs, opinions, and emotions.²⁴ When a qualitative method is used alongside quantitative methods, like in this case, qualitative research can aid in better understanding and interpreting the complex reality of a given situation.²⁴ The literature review and data section of the paper give insights into what data was found to be the most relevant for creating the forecasting model, but the main source of data was Nord Pool.

Literature review

This reason for analysis is both the scarcity of literature on intraday price forecasting as well as the notion that principles of forecasting are interchangeable between the two markets.³ These topics are vital to understanding before the data collection and analysis. Nowadays, many countries have adopted free-market rules for trading electricity using spot and other contracts. One of the motivations for the implementation of different trading products is the stability of the grid. Since a solution for economically storing electrical energy has not yet been developed, this presents a complex environment for TSOs to maneuver. The demand side depends on many variables such as weather factors (wind speed, temperature, and precipitation) and the intensity and magnitude of daily activities of large and small-scale consumers (peak against off-peak hours, weekdays against weekends, holidays, etc.). This leads to interesting characteristics in the energy markets. On the one hand, the behavior of energy prices is not observed in other markets, like exhibiting seasonal patterns on short-term time horizons, daily and weekly, as well as long-term yearly horizons. On the other hand, these price movements urged researchers to develop better forecasting techniques in order to minimize extreme prices which are expensive for consumers.

At a corporate level, electricity price forecasting has become a fundamental insight for the energy companies’ decision-making mechanism.³ The crisis in the US state of California in 2000–2001 showed that electric utility companies are the most fragile since they generally cannot pass the cost on to the retail consumers.³ There are periods where the cost of over/under producing and then selling/buying power in the real-time or balancing market can get to high enough levels to even bankrupt electrical utilities, especially during periods of unprecedented stress. Comparing the volatility of price to other commodities or financial assets can differ up to two orders of magnitude, meaning that the standard deviation of prices can oscillate tens of hundred times more than on other securities. This market feature requires asset managers to hedge both against volume and price risk. A utility company, a large-scale industrial consumer, or a generator that is able to forecast volatile prices with a reasonable level of accuracy can adjust its production or consumption schedule and its bidding strategy with the aim of minimizing risk or maximizing profits.³ An organization should invest resources in developing a forecasting system that involves several approaches to predicting uncertain events. Forecasting systems like that command the development of expertise in pointing and specifying forecasting problems, applying a range of forecasting methods, selecting appropriate methods for each problem, and assessing and refining forecasting methods over time.²⁵ Quantitative forecasting can be applied when two essential conditions are met. Firstly, numerical information regarding the past is accessible. Secondly, it is reasonable to assume that some aspects of the past patterns will continue into the future. Forecasting is a technique that uses historical data as inputs to a model to make informed estimates that are predictive in determining the future unknown. The forecaster is generally either trying to estimate a single value as the most probable value, which is known as point forecast, or trying to determine a range of values with prediction intervals and densities which is known as probability forecast.⁹ The choice between the point and probability forecast is dependent on the forecaster's research setting and the available historical data.²⁶

Articles about intraday electricity price forecasting are limited. Two papers are referenced the most in articles with regard to intraday electricity price forecasting. Monteiro et al.²⁷ use a model from the computational intelligence forecasting category called the multi-layer perception neural network technique using 2n + 1 neurons, where n is the number of input variables. The authors apply the technique to the six intraday sessions of the Iberian electricity market. Various models, depending on the number of input variables, were trained on in-sample data sets whereas the model was tested without-sample data set. Because the model “learned” by itself, the weighting of input variables was not fixed, and therefore, the results of the multi-layer perception neural network were inconsistent. This was rectified by averaging the range of outputs of the same model. One of the findings was the mean absolute percentage error and the oscillation of its value with different combinations of input variables for each intraday session. The model with the lowest mean absolute percentage error utilized only hourly prices of the daily session and hourly prices of previous intraday sessions alongside chronological variables. This finding was interesting because Monteiro et al. considered a range of 16 input variables like power demand and production forecast, weather forecast, legacy production forecast, and others. Similarly, session 6 presented the best results with input variables being hourly prices of previous sessions 3 to 5 as well as chronological variables.

The second article that is most commonly used in literature review regarding forecasting of intraday electricity prices is a paper written by Andrade et al.²⁸ This article can be considered as a hybrid between the fifth and sixth categories, statistical model, and computational intelligence. Weron indicates that hybrid between different categories is very common and can lead to more accuracy than explicitly relying on only one technique. Probabilistic forecasting can be seen as a conditional estimation of a probability distribution that most likely contains the real value of the electricity price in question.²⁸ Such models use quantiles that are, as the name suggests, conditioned by a set of explanatory variables. In this article, a point and probabilistic-based forecasting methodology were used. This article dealt with the Iberian market which the authors describe as being a highly volatile electricity market due to high integration levels of renewable sources of energy. In order to deal with the market characteristics, Andrade et al.²⁸ argue for the relevance of two statistical algorithms. The first one is linear quantile regression, statistical, and gradient boosting trees, and computational intelligence. The article researched the forecasting models and methodology for Iberian's electricity market for both products, day-ahead and hourly intraday market. Apart from reaching a relatively accurate point and probabilistic forecast with a 2.53 €/MWh error, the most important takeaway regarding the intraday market model is the fact that the model with the highest accuracy was generated by only using prices from previous sessions.

Prior to Kiesel and Paraschiv²⁹ publication, forecasting of intraday electricity prices was mainly motivated by the prediction of closing-hour prices. Contrary to that, Kiesel and Paraschiv aim to understand the bidding behavior of the continuous intraday electricity market. In particular, more light was shed on the contribution of continuously changing wind and photovoltaic forecasts on a 15-minute continuous-bidding contract in the German power market. Additionally, the authors are applying an econometric model on time series that are distinguished by summer and winter periods, and peak and off-peak hours. The article shows clear proof that the intraday last price for a 15-minute delivery period is below the day-ahead price for the analogous hour, independent of the season. Moreover, the differences become more significant for peak compared to the off-peak hours in winter than in summer. Using such a methodology, the authors had two goals. Firstly, an autoregressive model was created that captures the dynamics of the last bidding price for the 15-minute period and the corresponding day-ahead price. This model was proven to be useful as it aided the market participants who did not want to buy or sell electricity at more expensive balancing market rates. Secondly, the impact of explanatory variables on the continuous bidding behavior in the market was determined. Results display that during the morning and the evening hours, the significance of exogenous variables is lesser. On the other hand, such exogenous variables contributed up to four times more to the bidding prices during noon.²⁹ Such findings further enforce one of the main features of the energy market and that is high volatility since the market is quick to adapt to the intermittent nature of renewable.

Narajewski and Ziel³⁰ carried out a study aiming to explore electricity price formation in the intraday market. In the article, the authors consider three different linear models to forecast intraday hourly and 15-minute prices. The three techniques are the ordinary least squares, the least absolute shrinkage and selection operator, and the elastic net, which is a linear combination of the least absolute shrinkage and selection operator (LASSO) and ridge regressions. Narajewski and Ziel applied the ordinary least squares on models with a fewer number of exogenous variables, as the models performed better while using the simpler technique. The other two techniques handled more complex models that contain a very large number of explanatory variables. The results of the research support what the papers above had concluded—models with fewer input variables perform better than the more complex ones, but the results from Narajewski and Ziel go a step further. Although the more complex model performed the best for the hourly intraday product, the difference between the Lasso model and the naïve is not significantly different. The naïve model, in this case, was calculated as a weighted-average price of the transactions that take place not later than 15 min before the forecasting time. This is an indication that the market is efficient in weak form, meaning that the past prices, trading volumes, etc. are not indicative of the closing price for the hour product. Since this analysis was run on the German intraday electricity market, Narajewski and Ziel tried to understand the price formation for 15-minute products as well.³⁰

The same naïve model that did well in the hourly market did not perform well in the quarter-hourly product. Such results suggest that the market is not weak-form efficient, and therefore, an autoregressive structure can be utilized. The root-mean-squared error (RMSE) is interestingly higher for the quarter-hourly products than it is for hourly. The authors believe that this is due to the jigsaw pattern, hence, higher volatility in the data does not suit well the models utilized in the article.³⁰

Results and discussion

Forecasting is a complex task. A forecaster can have several possible objectives in analyzing time series since every objective will require an adjusted process to perform the analysis. There are many proposed ways of reaching the objective according to the examined literature, but this article follows the methodology proposed by Hyndman and Athanasopoulos.²⁵ The problem definition is considered to be the most important step of their proposed five-step process. The “Introduction” section of the article provided insight into the growing presence of the intraday market. It also presented evidence of the high correlation between the positive effects the intraday electricity markets have on the representation of RES in the electricity production source group. In turn, better forecasting of the intraday electricity prices will aid renewables to take a bigger portion of the market as more accurate forecasts imply cheaper distributed costs for the customer.

The second step of the process is gathering data. Data represents a key role in this time-series analysis as without it the only obvious way of conducting any sort of analysis is using heuristics. The data used in this paper originates from Nord Pool. The data spans over a two-year period ranging from 1 January 2019 until 31 December 2020. The data set contains different formats that must be combined into a single dataset with a single observation per consecutive hour. To achieve such a structure, a large number of pre-processing steps will be undertaken and presented. The data is collected from Nord Pool's FTP server and two data sets will be used in the following analysis. Elbas ticker data set consists of trades executed in the intraday market whereas the Elspot prices data set contains spot, day-ahead prices. Each of the data sets will serve its own purpose. Elbas ticker data will be used as a dependent variable and Elspot price data will serve the purpose of being an explanatory variable.

The third step when investigating time series should always, without exception, involve careful examination of the recorded data plotted over time. This scrutiny in most cases gives insights and suggestions on what method of analysis should be applied. There are two main approaches to time series analysis that are different but not necessarily mutually exclusive. The time domain approach perceives the examination of lagged relationships as the main criteria, meaning, the question to which an answer has to be found is “How will tomorrow look based on what happened today?” The frequency domain approach perceives the examination of cycles as the main criteria where the question is “What is the effect of summer and winter cycles?” In this article, due to the forecasting technique, the time domain approach is the most relevant. Preliminary analysis can give us an idea about what can we expect based on previous observations. A good way to get an initial understanding of the data is to graph historical data.

The fourth step is dedicated to model creation. To draw upon the remarks mentioned above, there are numerous time-series forecasting methods a forecaster can decide to utilize. The most simplistic way of forecasting something is to use only information on the variable to be forecasted. This means that the subject of analysis is a univariate time series in which only one variable is varying over time. In univariate time-series forecasting, factors that may influence the behavior of the targeted time series are not considered.²⁵ For example, a forecaster can develop a forecast for a univariate time series of hourly electricity demand (ED) which can capture the trend, variance, or seasonality to some extent, but relevant factors that play a role in supplying more information to the forecast model are not considered. Such a model might be of the form of equation (1):

E D_{t + 1} = f (E D_{t}, E D_{t - 1}, E D_{t - 2}, \dots, e r r o r)

Equation (1). Univariate time-series forecasting

In order to deal with missing information from relevant factors for time-series forecasting, a forecaster can introduce explanatory or predictor variables. Suppose that the aim of an arbitrary forecast is to estimate the hourly ED of a cold region during the winter period. A model with explanatory variables can take the form of equation (2):

E D_{t + 1} = f (E D_{t}, c u r r e n t t e m p e r a t u r e, p o p u l a t i o n, t i m e o f d a y, h o l i d a y, e r r o r)

Equation (2). Multivariate time-series forecasting

It is worth mentioning that the relationship between the ED and the explanatory variables is not completely clear. Sometimes the ED cannot be accounted for by the predictor variables. That is the reason why the “error” term is included as it allows for randomness in the demand forecast that does not need to be explained by the predictor variables. The model that will be used to forecast intraday hourly electricity prices belongs to a family of statistical models. The model is called an ARIMAX model which stands for autoregressive integrated moving average with explanatory variables. The algorithm that makes the prediction will be created through the following stages. Firstly, a function called “auto.arima” will be fed the pre-processed data to make an ARIMAX fit which will aid the selection of the order parameters of the model, “p,” “d,” and “q.” The function uses using Kwiatkowski–Phillips–Schmidt–Shin unit root test to ensure that the time series under consideration is stationary, a vital feature that needs to be present when forecasting. That very same “auto.arima” will be a part of a for loop that tries to minimize the Akaike information criterion (AIC) by changing the order of the Fourier series mimicking the seasonal ARIMAX, also known as SARIMAX. The AIC is an estimator of prediction error, and therefore, identifies the prediction quality of a statistical model for a given set of data. Using the Fourier series to represent the periodic oscillation was proved to be more accurate than SARIMAX.³¹ The result of the for loop was an initial ARIMAX model. However, this approach performed better only on larger training sets. In this case, shorter training sets were used instead for testing the validity of the model as this allowed for many more time sections for the cross-validation. Having more time blocks to be used in the cross-validation ensures that the performance of the model is well represented, and not just based on a few test runs. More importantly, shorter training sets also allowed for more optimal model training. Moreover, due to the process of optimizing the number of Fourier terms, this procedure was not fully automated and was therefore omitted. In order to account for seasonality, a simple seasonal dummy was used.

The last step of the process is to determine the methodology for evaluating and testing the model. The forecaster might spend many hours trying to determine the model which fits the time series the best. Inspecting the residuals might also show that the proposed model captured well-relevant information from the data set. However, it is not enough for the forecaster to do the model fitting sufficiently well and, in turn, miss out on not capturing valuable information from the time series. Evaluating forecast accuracy by the size of the residuals is not a reliable indicator. Therefore, the most crucial step is still to follow. The true forecast errors can only be evaluated by analyzing how well the model performs on a new time series data set that was not used when fitting the model. The two subsections below on results and forecast errors and cross-validation further elaborate on how the model will be tested.

Results and forecast errors

No model will ever be able to predict the future of a non-trivial event completely accurately.²⁵ Any version of future reality is described by a function modeling the reality plus an error term.³² When a model differs from the actual value, an error has occurred in the forecast. Therefore, an error is a difference between a recorded value and its forecast which is not to be confused with a forecasting method mistake. The error term was mentioned before when residuals were discussed. There is a difference between an error that is a residual and a forecast error. The aforementioned residuals are a product of the model not fitting the training set well enough while errors that occur in a forecast are calculated on the test set. A poor model used to make decisions regarding demand and supply planning can cause an imbalance in the electricity market as well as cause market participants, like traders, to minimize profit or even result in major financial harm. Errors are used to quantify how well a model performs in reference to another model. However, it is worth mentioning that when it comes to forecasting errors there is no one-size-fits-all measure.

The two most commonly used types of errors are scale-dependent errors and percentage errors. Scale-dependent errors are on the same scale as the data set and give valuable feedback which is only to be interpreted on or between time series using the same units. To assess the accuracy of the forecast “root-mean-squared error” (RMSE) will be used and it will take the form of equation (3):

R S M E = \sqrt{\frac{\sum_{i = 1}^{N} {(y_{i} - y_{i}^{^{'}})}^{2}}{N}}

Equation (3). Root-mean-squared error formula.

During the stage of determining which model fits the data set the best, it is common practice to split the available data into two data sets. The first data set is the so-called “training data” or “in-sample data” which is used to estimate any parameter of a forecasting method. The second data set is called “test set” or “out-of-sample data” which serves the purpose of evaluating the model's accuracy while providing important insight into how well the model is expected to forecast new data.

Cross-validation

The ARIMAX model this article proposes for capturing electricity intraday price for the DK1 area will be tested by a blocked time-series cross-validation method. This way of verifying the model's robustness is widely used in practice and provides credible results.³³ By using this method, the model is tested many more times than in a conventional training-test split. Moreover, it guarantees that the ordered nature of the observations is maintained. The training set is made of observations that exclusively occurred prior to the observation from the test set, meaning that the training set was not “tainted” with values from the “future.”³³

The forecast accuracy is computed by averaging the difference between the forecasted value and the test values. This method is sometimes called “evaluation on a rolling forecasting origin” because the “origin” point at which the forecast is based moves or rolls forward in time.²⁵

Big data collection, processing, and selection

This section expands on Nord Pool as the main data provider, as well as data format and data in general. Additionally, it clarifies how the unstructured raw data was transformed into a usable data set. Moreover, it is argued why and how certain explanatory variables were used and to what extent the model was constrained. Hyndman and Athanasopoulos²⁵ point out that one might expect a model with explanatory variables will provide more accurate results, but something quite contrary might occur. As the authors point out, conducting a forecast by just using historical values might be a better option. Predictor variables are only useful when the system is understood. It would be unwise to try to forecast winning lottery numbers by using the moon phase as an explanatory variable since the hypothesis on what will the winning numbers be or what influences them is still not determined. This is all to say that some systems are just not understood, and even if they were, it may be difficult to measure the relationship that is assumed to govern their behavior.²⁵ Another layer of difficulty is being able to forecast the factors that influence the variable of interest. Finally, Hyndman and Athanasopoulos²⁵ close the argument in favor of univariate time-series forecasting by claiming that univariate models give more accurate forecasts than models with explanatory variables.

A common theme throughout the articles is that the day-ahead spot prices were the most highly correlated variable to intraday hourly prices, hence, day-ahead spot prices were used as one of the price drivers. To be exact, the correlation was calculated to be 0.83 and the scatter plot in Figure 6 shows the linearity and dependence between the Elspot prices and Elbas VWAPs.

Figure 6.

Relationship between Elspot and Elbas prices.

Furthermore, more explanatory variables could have been used since there are more variables that affect intraday electricity prices other than Elspot prices like capacity, load and consumption, wind, and solar but including them comes at a cost.³ The cost of including more predictors is a larger prediction interval and higher uncertainty of estimating VWAP as the explanatory variables are also forecasts themselves which have inherent uncertainties within. Additionally, more explanatory variables equate to more computational power needed to create a prediction which is problematic from two points of view. The amount of computational power the forecaster has at his/her disposal is limited. Furthermore, the time needed to create the forecast grows at a rate of 2 ^x , where x is the number of predictors, and this time is very valuable since it can “make or break” a potentially profitable trade. All things considered, Elbas market data as well as Elspot day-ahead prices are found to be sufficient to create a forecast that can provide respectable accuracy while not compromising on the computational time.³⁰

The data used for this time series analysis was retrieved from Nord Pool. The type of the data file is a “Comma-Separated Values” or CSV for short. The size of the original file is fairly large, and it amounts to 2.24 GB. A data set of this size can be classified as Big data.³⁴ The time period that the CSV file covers is a two-year period spanning from 1 January 2019 until 31 December 2020. Data pre-processing is mandatory to make the data set usable by conducting a time-series analysis. The collected data consists of 14,049,041 observations. The raw data set is not uniform in its formatting, and therefore, must be combined into a single dataset with a single observation per consecutive hour. In order to achieve such a data set format pre-processing steps will be presented in the following subsection.

An overview of the data sets used in the article is presented in Table 3. The table contains information like sources, the number of files per year, the time of publication, and the resolution of publishing new information. These files constitute the market data that will be used to create a forecast that will estimate future VWAP for the hourly intraday market. Additionally, all historical data, including the data used in this article, are gathered through Nord Pool's “File-Transfer Protocol” (FTP) server. The FTP server is updated on a daily and weekly basis for Elbas and Elspot markets accordingly. However, regarding the market participants, the newest information is published for each market at times, and resolutions are presented in Table 4.¹² Aiming for this analysis and the proposed forecasting model to be utilized in real life, the forecasting model will only be processing data that would be accessible in real life at the time of making a prediction.

Table 4.

Properties of data sets.¹²

Data set	Source	File format	Files per year	Published	Data resolution
Elbas ticker	Nord Pool	CSV	365	Real-time publishing	Real-time observation
Elspot prices	Nord Pool	.xls	4	One-day ahead	Hourly

The Elspot prices file contains information on price, volume bought, and volume sold for each hour of delivery and for each bidding area that Nord Pool is covering. The Elbas ticker file is a log of all orders for any of the 24 available hours. It also provides information on the time of the trade as well as the price, volume, party's respective bidding area, and information on whether an order was executed or canceled.

The Elbas ticker data set as well as Elspot prices are formatted in a way that is somewhat unreadable to a human eye. The file is however slightly better formatted for data analysis in software like R, although still completely unusable in the original format. As such, a number of steps are required to extract relevant information, and afterwards transfer and store it in data structures suited for time series libraries in R. The aim of these steps is to end up with a complete and organized time series from 1 January 2019 until 31 December 2020 that has only one row per successive hour of power delivery, one column for each input variable (intraday hourly electricity prices and day-ahead hourly prices), and one column for the output variable. Since Nord Pool is not the only exchange used for trading in the Nordic countries but is the biggest, the data set, therefore, does not contain all trades executed on the Elbas market. Moreover, the Elbas ticker data set is filtered for trades relevant to DK1, meaning trades between DK1 buyers or sellers and other bidding areas or between buyers and sellers within the DK1 area. The steps regarding data pre-processing mentioned above were initiated by importing raw files in the open-source language R. To combine two data sets from Table 3 into one uniform and legible multivariate time series with one row per consecutive hour, the data set had to have the same level of detail or grain.

Since the Elbas ticker is a collection of executed trades between parties and occurs in no particular pattern it has a different grain compared to the ordered Elspot data set. For the purpose of this analysis, it is essential that both Elbas and Elspot data sets have the same grain which implies that the Elbas ticker data set has to be aggregated to match the grain of Elspot prices since Elspot prices grain is the “common denominator.”²⁵ Aggregating Elbas ticker was not achievable with simple filtering functions but required an algorithm to categorize a trade by delivery hour. The process created for condensing Elbas trades was very extensive and can also be seen as a contribution to the field but is not further discussed as it is not the main focus of the article. With regards to the Elspot prices data set, it has a long format with spot prices for multiple areas presented in independent rows for the same delivery hour whereas it should rather have a wide format where the data set is transposed into one row representing the spot price for each bidding area as a column.

Limiting data access for the algorithm

This article was written in 2021, and the time period under consideration is between 1 January 2019 and 31 December 2020. Consequently, the model could have had access to more data than if it was run right now. By limiting data access, which realistically will not be available to market participants at the time of making a prediction, the model was tested with the information it would have in real life. The reasoning behind this forecasting horizon is the market setting on Nord Pool's exchange. The largest traded order on the exchange is for four consecutive hours.³⁵ That in turn means that if a trader was to implement this model in the strategy, the trader would have sufficient information to trade up to a block order which provides a diverse range of possibilities.

Calculating aggregated Elbas prices

There may be numerous trades in the Elbas market for a particular hour. There is no upper and lower bound on how many trades need or can be executed for a given hour. It only depends on the consensus between the buyer and the seller, just like in the stock market. Time-series analysis theory suggests that the time difference between observations has to be constant to be able to apply econometric theories and develop a forecast. Since Elba’s data is spars, it has to be aggregated in order to reach the needed format. Taking the volume-weighted average price, VWAP_h is a popular approach in many research papers.²⁹ It is calculated as in equation (4):

V W A P_{h} = \frac{\sum_{n = 1}^{N} (P_{n, h} \times V_{n, h})}{\sum_{n = 1}^{N} V_{n, h}}

Equation (4). Volume-weighted average price.

where P_n,h is the price (EUR/MWh) and V_n,h is the volume (MWh) in trade n for delivery in hour h. N symbolizes the total number of executed trades for hour h. Once the original Elbas ticker data set was converted to a time series containing VWAP per delivery hour, the statistical properties of the modified time series were examined. Figure 7 shows the line plot of VWAP for consecutive hours starting 1 January 2019 until 31 December 2020. Table 5 provides some descriptive statistics about the original Elbas ticker data set versus the volume-weighted one. It can be seen that the outliers are lost from Elbas VWAP, but the most important statistical properties like mean and standard deviation are kept true to the original time series with negligible deviation.

Time series models

The section about time series models introduces mathematical equations that govern how the model produces the best fit on the training data as well as how it estimates future values. The primary aim of time series analysis is to develop models that provide an approximate representation of sample data. In order to specify a statistical setting for describing the character of data that seemingly fluctuates in a random manner over time, we assume a time series can be seen as a collection of random variables indexed according to the order of variable values recorded in time.³⁶ Time series can hence be considered as a sequence of, x₁, x₂, and x₃ random variables where x₁ represents the value taken by the series at the first-time stamp, x₂ represents the value taken by the series for the second time stamp, x₃ represents the value of the third timestamp. When forecasting time series data, the aim is to estimate how the sequence of observations will continue in the future.

Figure 7.

Volume-weighted average prices for the Denmark-West bidding area (DK1).

Table 5.

Descriptive statistics for DK1.

Prices	Mean	Min	Max	Standard dev.
Elbas ticker	31.24	−91.1	1000	19.74
Elbas VWAP	30.69	−73.54	688.49	19.92

DK1: Denmark-West bidding area; VWAP: volume-weighted average price.

Regression

Regression analysis is a useful tool for forecasting when there is an assumption about a relationship between the dependent variable, the output of the forecast, and an independent variable, the predictor. In its simplest case, linear regression models can prove a relationship between the forecast variable y and a single explanatory variable x (equation (5)):

y_{t} = β_{0} + β_{1} x_{1} + ε_{t}

Equation (5). Linear regression

where β₀ and β₁ represent the intercept and the slope of the line, respectively, while ε _t represents the random error. Since this article uses two independent time series to make a prediction for the output variable, multiple linear regression will constitute the regression portion of the ARIMA algorithm. The general form of a multiple linear regression model is as follows:

y_{t} = β_{0} + β_{1} x_{1, t} + β_{2} x_{2, t} + \dots + β_{k} x_{k, t} + ε_{t}

Equation (6) . Multiple linear regression

As can be seen from equation (6), it is very similar to the linear regression with only one predictor. The only difference is the number of predictor variables as well as the meaning of the β parameter. In the multiple linear regression, β₁, …, β_k parameters denote the effect of each predictor on the dependent variable after taking into account the effect of all the other predictors in the model. The way the β₁,…, β_k parameters are determined in practice is by minimizing the sum of the squared errors also known as “least squares” estimation. Multiple linear regression is very similar to autoregression in the way that autoregression uses a linear combination of past values to make a prediction. When we talk about training a model or learning from the training data set “least squares” estimation is the process that is happening in the background to optimize the parameters.

Moving average

Opposite to the regression models where past values are indicators of the future, moving average models use past forecast errors to predict the future. However, moving averages and regression models have more in common than not. Even though the premise is different in the way that moving average models utilize the difference between observations and the forecast to predict while regression uses past observation, the equation is as follows (equation (7)):

y_{t} = ε_{t} + θ_{1} ε_{t - 1} + θ_{2} ε_{t - 2} + \dots + θ_{q} ε_{t - q}

Equation (7). Moving average

The explanation for equation (7) is analogous to one for regression in a way that θ_p represents the weighting factor for each error term and the number of θ terms symbolizes how many past error terms affect the prediction.

ARIMAX

Combining the above-mentioned autoregression together with the moving average we get an ARMA model which stands for autoregressive moving average. What is missing the “I” from ARIMA. “I” stands for “integrated,” and in the case of the ARIMA model, it is responsible for differencing the time series which means that the preceding value is subtracted by the following value. In many time series problems, this is necessary in order to have a stationary series. Stationarity is essential as it is a prerequisite for applying the ARIMA model to a time series. Taking that into account, the mathematical form of an ARIMAX model is as in equation (8):

y_{t}^{'} = c + \emptyset_{1} y_{t - 1}^{'} + \dots + \emptyset_{p} y_{t - p}^{'} + θ_{1} ε_{t - 1} + \dots + θ_{q} ε_{t - q} + α_{1} X_{1} + \dots + α_{p} X_{p} + ε_{t}

Equation (8). Autoregressive integrated moving average with explanatory variable

where yʹ_t is the differenced time series. ARIMA has three parameters “AR, I, MA,” and instead of writing a long equation like equation (8), the model can be referred to as ARIMA(p,d,q) model. “p” stands for the order of the autoregressive part. If “p” is equal to 1, it means that the predicted value is based on the immediately preceding value while “p” is equal to 2, it means that the forecast is based on the two previous values. “d” stands for the order of differencing involved. “d” equal to 1 means that the preceding value is subtracted by the following value one time while “d” equal to 2 means that the preceding value is subtracted by the following value two times, and so on. The last parameter of the ARIMA model is “q” and it represents the order of the moving average component it follows the same logic as the autoregressive order, but just for errors that occurred between the forecast and the actual observation. One letter of the ARIMAX model was not discussed yet. The letter X stands for the presence of explanatory variables to aid the better accuracy of the prediction. If two variables are found to improve the forecast, then α₁ and α₂ will be weighting factors for each explanatory variable. The bigger the α, the more influence it has on the prediction. In this case, there is only one explanatory variable, the Elspot prices, and therefore, only α₁X₁ will be present. Last but not least, the way that parameters c, ∅₁,…,∅_p, θ₁,…,θ_q, α₁,…,α_p from equation (8) are identified is using “maximum likelihood estimation” (MLE). This technique is very similar to the “least square” estimation but has a different objective function. MLE tries to maximize the probability of obtaining the data that was observed.

Trading analysis

Even though this article focuses on the intraday market, both data sets, Elspot and Elbas, are analyzed as they will provide insight into the market behavior. Furthermore, data analysis will enhance the understanding of the market, trading, and price characteristics when evaluating the practical reliability of the proposed model. Figures 8 and 9 show a line plot of the spot price for consecutive hours and a histogram of the spot price for consecutive hours accordingly. The line plot in Figure 8 shows the oscillatory behavior of the prices with some outliers. Oscillatory behavior is expected as the demand and supply sides need constant tweaking to ensure equilibrium between the two which causes up-regulation and down-regulation when prices are higher and lower, respectively. The histogram in Figure 10 represents the distribution of spot prices. The price with the highest probability of occurring is represented by a red density line and in this case, it is ∼ 31.73 EUR. Being an explanatory variable, all of the Elspot market characteristics pointed out from the two plots had an effect on the suggested forecasting model.

Figure 8.

Elspot price distribution.

Figure 9.

Elspot prices for Denmark-West bidding area (DK1).

Figure 10.

Histogram of intraday trades on Nord Pool.

As far as Elbas trading is concerned the data is broken down into two categories. Through the first category, the goal is to introduce the reader to the Elbas market for the whole Nord Pool exchange, after which the second category will solely focus on a trading analysis of the intraday market for Denmark West, DK1. Figure 10 captures the essence of the intraday market as a whole. The visible part of the distribution is very narrow, meaning that most of the time prices fall within a certain range, with an average price of 34.33 EUR/MWh. The shape of the price distribution sends a clear message and further supports claims presented in the literature review about the volatility of the intraday market. Long tail and skinny distributions are caused by the excess of extreme values. Table 5 is associated with Figure 10, and it contains values that describe what cannot be seen in the histogram and those are the outliers sectioned in eight different cuts. Furthermore, Table 6 indicates that out of the total 14,046,277 trades that occurred starting from 1 January 2019 until 31 December 2020, 82% had a value between 0 and 50 EUR/MWh followed by a 50–100 EUR/MWh price range accounting for 14.6%. The price distribution suggests that trades lower than −50 EUR/MWh or higher than 250 EUR/MWh can be considered outliers, and as such represent 1% of traded volume. These outliers cause the forecasting process to be more unpredictable since they occur somewhat rarely, and therefore, are difficult to forecast because there are only a few observations in a certain price range. Figure 11 represents the price distribution but disregards the outliers after which the prices follow a normal distribution.

Figure 11.

Histogram of intraday trades on Nord Pool without outliers.

Table 6.

Size of trade distribution.

Price range, EUR/MWh	(−250, −50]	(−50, 0]	(0, 50]	(50, 100]
Count	47,501	405,655	11,467,633	2,045,894
Price range, EUR/MWh	(100, 250]	(250, 500]	(500, 1000]	(1000, 3000]
Count	75,292	3532	731	39

The second category covers data directly related to the DK1 bidding area. This data set contains trades executed for Elbas hourly products with DK1 being the buying or selling area. The number of orders that were filled from 1 January 2019 until 31 December 2020 amounts to 830,451. Similar to the price distribution for the whole Nord Pool exchange bidding area, DK1 price distribution can be characterized as being narrow with quasi-long tails representing the outliers. The same conclusion regarding the price dynamics can be drawn for the DK1 bidding area as the one for the whole Elbas market, of course, adjusted for the number of trades. Interestingly, the average price of the distribution is 31.24 EUR for DK1, which is slightly lower than for the whole Nord Pool. Table 7 captures the number of trades executed in DK1 by price range. The price range with the highest number of occurrences is between 0 and 50 EUR/MWh representing 85% of the volume traded, followed by the price range from 50 to 100 EUR/MWh with a market share of 11%. As for the outliers, Figures 12 and 13 illustrate the price distribution with and without considering the outliers in an identical manner as for the first category of data. A comparison line can be drawn between the DK1 market, and all the bidding areas belonging to Nord Pool which reveals a unique market characteristic. There is a higher probability that an MWh in DK1 will be bought or sold within a given price range compared to Nord Pool's bidding areas. This in turn means that the variance is more constant and should result in a more accurate forecasting model.²⁵

Figure 12.

Histogram of intraday trades for Denmark-West bidding area (DK1).

Figure 13.

Histogram of intraday trades for Denmark-West bidding area (DK1) without outliers.

Table 7.

Size of trade distribution for the Denmark-West bidding area (DK1).

Price range, EUR/MWh	(−250, −50]	(−50, 0]	(0, 50]	(50, 100]	(100, 250]	(250, 500]	(500, 1000]
Count	1118	27,480	707,787	91,670	2245	87	64

Now that trading prices have been analyzed, trading behavior with regard to delivery hours will be introduced. Since Elbas market in DK1 only has hourly products that means there are 24 delivery hours. Therefore, the plot shown in Figure 14 contains information on the trading volume per delivery hour. The plot shows that market participants are trading every single hour with some hours having more interest than the rest. It can be seen that evening and night hours are not as popular as morning and afternoon hours. The most traded hour is hour 8 with a total of 47,193 trades.

Figure 14.

Histogram of intraday trades per hour for Denmark-West bidding area (DK1).

Additionally, examining the trade size is to be done. This is an important market characteristic to analyze because it affects the final volume-weighted price, and Table 8 does just that. The table demonstrates how much MWh of energy was bought or sold per trade. Market participants most often trade energy in 0–10 MWh volumes, that is, 93% of all trade sizes to be exact. The average amount of energy traded is 5.5 MWh while the highest recorded in the time period under consideration is 373.3 MWh.

Table 8.

Traded volume distribution for the Denmark-West bidding area (DK1).

Volume traded, MWh	(0, 10]	(10, 20]	(20, 30]	(30, 40]	(40–50]	(50-Inf]
Count	761,232	39,643	18,282	3440	5313	2541

The last part of the trading analysis section is to inspect how volatile the prices are per hour. If some hours are more volatile than others, then the trading strategy has to accommodate that. Not doing so can result in variable risk exposure over the course of a trading day.³⁷ This risk can be seen in the section below where the performance of the model is discussed and RMSEs graphed as a function of the delivery hour. The graph below in Figure 15 shows the hourly dispersion of prices per MWh of energy from its mean. Referring back to Figure 14, it can be seen that the standard deviation increases as the number of trades increases. However, from hour 16 this ratio becomes disproportionate as the standard deviation increases significantly while the number of executed trades declines.

Figure 15.

Standard deviation of energy prices.

Performance of the model

The final section of this analysis discusses the performance of the model. Firstly, the model's performance on the training set is followed by how well the model did in the test set. After much data processing, data analysis, and programming, the following question can be answered—how accurately can the researched model predict the future VWAP?

This analysis was conducted using a large data set. Such a setup allowed for extensive model training and testing. In this case, after the data processing and selection, the model had access to ∼ 438,000 observations in total ranging from historical Elbas VWAP to Elspot prices and dummy variables. Having such a vast number of observations, an optimum size of the training set was critical. The first attempt at producing predictions of future VWAP used almost all the observations as the training set. This meant that only one model with fixed p, d, and q parameters was used. Equation (8) shows that the ARIMAX model is linear by nature. Therefore, when the algorithm tried to fit a line that best fits the training data, it fitted one straight line to represent the market for the period of almost 2 years. Consequently, with this approach, the ARIMAX model described the intraday market as a static system with a linearly increasing, decreasing, or constant VWAP. The conducted literature review alongside various presented graphs would deny such a description. Indeed, such an approach produced very poor accuracy, performing much worse than the most trusted forecasting method according to the literature review, the day-ahead price for the corresponding Elbas hour. The results showed that the initial ARIMAX model performed 45% worse than the day-ahead price. Because the original model was rigid and unflexible, it did not capture the volatility of the intraday market. To combat the model's weaknesses a more flexible approach should take place.³⁷

Producing fairly bad initial results did not depict the actual performance of the model, as the goal was to investigate the effectiveness of the statistical model ARIMAX. In order to really determine whether ARIMAX was a good model for forecasting intraday VWAP, the model setup had to be as optimal as possible. Having a goal of rectifying issues encountered in the initial model, a different methodology had to be implemented for training the model. Therefore, a test was conducted to establish the finest split between the size of the training set compared to the size of the test set.³⁸ The results showed that the ARIMAX algorithm had its peak accuracy at a split of 95% in favor of training and 5% test. As the forecasting horizon for this analysis is 4 h ahead in time, that means 76 past values of VWAP and the corresponding explanatory variable are used to train the ARIMAX model, and the future four VWAP alongside predictors to test its accuracy. Using 76 observations to train the model ensured that the ARIMAX could capture more information from the model than the initial approach where all observations were used to train the model. Figure 16 attempts to demonstrate the difference between the two methods. However, it is important to stress that both methods still follow the forecasting methodology discussed above with the second method being a more optimal choice.

Figure 16.

Difference between training approaches.

It is worth mentioning that Figure 17 does not represent the actual RMSE of the optimal model. The reason why this is not the actual RMSE of the optimal model has to do with the setup for conducting the test. For the sake of shorter computational time, a higher forecasting horizon was used. In this case, a horizon of 72 h ahead was applied.

Figure 17.

Model accuracy of training versus test data split.

It will be seen later in this section that the actual RMSE for forecasting 4 h ahead was much lower, proving that shorter horizons produce lower forecasting errors.²⁵ Moreover, Figure 20 serves the purpose of illustrating how the performance of the statistical model under consideration, ARIMAX, improves when the model has more training observations. After a certain point, the situation reverses, and the model starts performing worse once the training-test split rises more than 95% in favor of training. If the line in Figure 17 that depicts model accuracy as a function of training-test size split was extrapolated to the point where almost all the observations were used to train the model, the initial training method would be the result.

Once the ideal training size was found the algorithm was run and it produced a prediction for future hourly intraday VWAP. The ARIMAX algorithm constructed in total of 4367 forecasts, one for every hour starting from 04:00 4 January 2019 until 00:00 31 December 2020. The algorithm could not have made the prediction from the start of the time period under consideration, 00:00 1 January 2019, due to the training-test split of 95%–5%. That means that the ARIMAX needed a training set of 76 h VWAP before it could make a forecast.

Training set model performance

One of the methods to verify the validity of the model is to perform a residuals analysis. There will always be “something” left over after fitting the model to observations. This “something” was caused by fitting the model to past observations, but not perfectly. Using the checkresidual function in R, a default plot is created as shown in Figure 18. What can be seen from the graph below are residuals from fitting the model to the data. They are equal to the difference between the recorded observation and the corresponding fitted value.³⁹ Examining residuals is a common step when verifying whether the time series model that the forecaster is trying to fit is good or poor. A good model adequately captures the information from the data, and a poor one does not. In order to verify if the model is well suited for the data that is being analyzed, the residuals should have certain properties. Firstly, depending on the amount of information captured, the residuals should be uncorrelated if the model captured the behavior of the time series well, while correlated residuals notify the forecaster that there is information left over which will in turn not be used for computing the forecast. This indeed is true as the ACF plot indicates that the correlation between lagged observations is below the level of significance indicated by the blue dotted line. Secondly, a good model should leave residuals with a mean of zero. If the residuals do not have zero mean, then the forecast will be biased. The histogram of residuals is plotted in the bottom right corner of Figure 19 as the mean value is in fact approximately zero.

Figure 18.

Residual from model fitting.

Figure 19.

Forecast of autoregressive integrated moving average model with exogenous variables (ARIMAX) model.

In addition to the first two, additional reassurance can come from the two following properties. Residuals should have constant variance and be normally distributed, meaning, residuals should resemble white noise. The top part of the plot contains a time series of differences between the fitted and realized values resembles a white noise process which indicates that error terms are random. Additionally, it is noteworthy that not all blocks of data have a good model fit and satisfy the aforementioned requirements. This is to be expected when large amounts of data points are used to train a statistical model.²⁵

Test set model performance

How a model performs on the training data is far less important than the model's performance on the test data. For clarification purposes, let's compare two models, one model has a mean percentage error on the training and test set of 0% and 30%, respectively, and the other model with an error of 28% and 29%. Between the two models, the second model would be classified as better because of the lower test error.³⁷

Before explicit error values are presented, it is important to visually inspect the forecast that the model made. Figure 19 shows a plot of forecasted VWAP against actual recorder Elbas VWAP. The top plot captures both the forecast and the realized values throughout the time. Due to a large number of observations in a 2-year period, only a superficial visual initial insight can be gained in terms of the performance from the top graph of Figure 19. The blue line represents the forecast while the red line represents the actual recorded VWAP. On the first look, the top graph from Figure 19 shows that the forecast line follows relatively well the realized VWAP line, meaning, forecasted values do not highly differ from the original data. The two graphs below the top plot zoomed in on an arbitrary part of the timeline. By zooming in on a more narrow section of the data set, the actual goodness of fit between the two lines would be easier to see. Each of the zoomed windows covers roughly 2 weeks' worth of data or 340 h values. It can be clearly seen on the bottom left graph of Figure 19 that there is a good fit between the forecasted VWAP and the actual price. On the other hand, the graph to its right demonstrates a fairly poor goodness of fit. The reason why the model performed differently during the two time periods is straightforward—the model performs better on data that exhibit patterns. For instance, the graph on the bottom left shows a somewhat stable pattern. In this period, VWAP showed a high degree of correlation, and therefore, provided statistically significant information for the model to interpret and use for the prediction. Conversely, the graph on the bottom right depicts a relatively random process where there is little relationship between the historical values which is the reason why the ARIMAX model cannot capture such behavior accurately, hence, the irreducible error term in equation (8).

Each delivery hour is distinct and there are no rules on how many trades can occur per delivery hour or what is the highest or the lowest price for MWh of electrical energy. However, each delivery hour is characterized by different market conditions.

Different market conditions may cause changes in the standard deviation of VWAP for a delivery hour; the number of market participants, and the volume of trades occurring per hour. Although such factors were not considered, as explained in the data selection section, it is to be expected that the price behavior will differ throughout a trading day.²⁹ Figure 20 contains a plot that reveals how the performance of the proposed ARIMAX model is not uniform over the course of the day but varies depending on the delivery hour under consideration. The differences between the model accuracy over two separate hours can go as high as 9.5 times multiple of RMSE. In this case, the difference is visible when considering the accuracy of the model for hours 16 and 19.

Figure 20.

The error of forecast over delivery hours.

It is interesting to refer to Figure 15 where standard the deviation of each delivery hour is shown when looking at the graph in Figure 20. Upon comparison of the two graphs, it seems to be the case that for the hours for which Elbas has higher volatility, the model makes a higher forecasting error. The relationship between the standard deviation of VWAP for delivery hours and the corresponding forecast accuracy is also presented in the paper by Kolberg and Waage.¹² To this point, the way that forecasting errors are calculated further supports the claim that there is a dependency between the standard deviation and error.

Most importantly, Figure 20 clarifies which model is better. If both the blue and the red curves are integrated, the area-under-the-curve is a fine way of comparing the two forecasting methods. The ARIMAX corresponds to a curve with a lower area, hence, the ARIMAX model is a more optimal forecasting tool.

The proposed model showcases superior performance over the naïve model for almost all hours. The ARIMAX model performed better on 22 out of 24 h. Using the testing method discussed in the previous section, the ARIMAX model had an average RMSE of 7.8 EUR/MWh over 4367 blocks of the test set while the naïve model had an RMSE of 11.3 EUR/MWh. Using such a robust testing method as cross-validation suggests that the proposed method of forecasting intraday hourly VWAP for DK1 is 31% superior compared to the industry-used method.

Conclusions

The intraday market is gaining popularity in the energy trading domain for two reasons in particular. The first reason is the fact that electrical energy is not economically capable of being stored, and its demand and supply must be always balanced and be met instantaneously at physical delivery time. The day-ahead market serves a great purpose in reducing energy costs for consumers as well as producers, but there is much more that can be done to lower the costs even further. RES generators play an ever-growing role in the energy market but are also dependent on weather conditions, and therefore, volatility in production and price is implied. This can be mended with intraday trading by allowing market participants to optimize production and consumption schedules through better forecasting. More accurate forecasts, compared to the day-ahead market, are implied as the forecasting window is shorter and the behavior of RES can be predicted with a higher confidence level. All of this ensures lower costs to the customers and a more efficient market.

Although weather conditions dictate the energy production of RES, the literature review interestingly suggests that wind and solar forecasts are not as influential in intraday hourly prices as Elspot prices for the corresponding Elbas hour. This premise was implemented in the analysis by only using Elspot prices as a predictor variable and an ARIMAX statistical model to answer the research question.

To answer this, the intraday market for the West Denmark bidding region was analyzed where general market characteristics were determined. The analysis produces insights on market volume, trading activity, and price movement over a time period of 2 years spanning from 1 January 2019 until 31 December 2020. Incorporating findings of the market analysis as well as the predictor variable, a model that forecasts intraday VWAP for hourly products was developed. As a reference point, a benchmark forecasting method advised by multiple research articles to predict intraday hourly prices was used. The day-ahead price for the corresponding Elbas hour forecasting method produced a mean-square-root error of 11.3 EUR/MWh. The findings of this article suggest an ARIMAX model that can predict hourly VWAP with an average accuracy of 7.8 EUR/MWh. Therefore, the outcome of this research is a model that can help market participants to achieve 31% better forecasts compared to the naïve forecasting method.

Future work

Energy markets are getting a lot of attention in the media nowadays. This is mostly due to energy shortages and worries about future prices of electrical energy as prices of natural gas, coal, and oil are increasing in the past 18 months. Better forecasting practices would be very beneficial in such market conditions as they would aid better consumption and production planning—especially for the world's highest energy consumer countries^40,41—as well as profit for the trader.

Upon discussing my findings with a company operating in energy trading, one topic for future work presented itself as interesting from their point of view. What was examined in this article is the ability of the ARIMAX to predict hourly VWAP for DK1, but what was not explored is the commercial viability of using this strategy. A lower RMSE is not enough to proceed and implement the proposed forecasting algorithm. Therefore, one of the directions that could be interesting is testing whether this strategy can be profitable through backtesting by incorporating market features discussed in this article like periods of dynamic trading frequency and price variance per delivery hour.⁴²

Highlights

Developed autoregressive integrated moving average with explanatory variables (ARIMAX) model outperforms other methods for intraday price forecasting

The model achieves a mean absolute percentage error of just 1.5% using only publicly available data

Explanatory variable (day-ahead price) improves the accuracy of intraday forecasts

The study combines time-series analysis and big data techniques for better predictions

Novelty: First study to apply ARIMAX model to DK1 intraday market

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

George Xydis

Marko Lucic is a seasoned Innovation Portfolio Manager with 3+ years of project portfolio management, data and market analysis, risk mitigation, and leadership experience. Recognized for demonstrating a natural aptitude for analyzing data to implement process improvement strategies, as well as for establishing innovation initiatives. Additionally, he has a MSc degree in Technology Based Business Development where he gained valuable experience with utilizing Machine Learning for business intelligence. Professional focal points include technical proficiencies, data analysis, communication skills, industry expertise, and the ability to adapt to ever-changing situations in a fast-paced environment. He now manages open innovation strategies by partnering with universities for technology foresight and scouting for potential startups to secure innovation initiatives, and host ideation workshops using Design Thinking to find solutions to complex business problems.

George Xydis, PhD had the opportunity to work in the wind sector together with developers, utilities, constructors, universities, and research institutes. He used to work as a Wind Projects Development Coordinator at Iberdrola Renewables, as a Wind Project Developer at Vector Hellenic Windfarms S.A., and as a Researcher at the Center for Electric Power and Energy, Department of Electrical Engineering at the Technical University of Denmark. He has been since Feb. 2017 with the Center for Energy Technologies, Aarhus University as an Associate Professor and since Jan 2023 as a Full Professor. George also works as an adjunct faculty member at Johns Hopkins University in Energy Policy and Climate program. He teaches 425.624 - Wind Energy: Science, Technology and Policy. He used to collaborate, as a freelancer, with institutes, universities, and SMEs. George's research interests include Energy Planning, Wind Resource Assessment, Technoeconomics of Renewable Energy Systems, Energy Demand and Plant Factories. He has co-authored more than 100 publications in international journals, more than 50 in international conferences, books, and book chapters, and serves as an Associate Editor in Frontiers in Energy Systems & Policy and in Energy & Environment, SAGE Publishing. He holds a PhD from the National Technical University of Athens, and a degree in Mechanical Engineering from the Aristotle University of Thessaloniki, Greece.

Appendix

References

Karanfil

. The role of continuous intraday electricity markets: the integration of large-share wind power generation in Denmark. Energy J 2017; 38. doi:10.5547/01956574.38.2.fkar

Scharff

Amelin

. Trading behaviour on the continuous intraday market Elbas. Energy Policy 2016; 88: 544–557.

Weron

. Electricity price forecasting: a review of the state-of-the-art with a look into the future. Int J Forecast 2014; 30: 1030–1081.

Mauritzen

. What happens when it’s windy in Denmark? An empirical analysis of wind power on price volatility in the Nordic electricity market. An Empirical Analysis of Wind Power on Price Volatility in the Nordic Electricity Market (December 28, 2010). NHH Department of Finance & Management Science Discussion Paper, (2010/18). 2010.

Jamasb

Pollitt

. Electricity market reform in the European Union: review of progress toward liberalization & integration. Energy J 2005; 26. doi:10.5547/ISSN0195-6574-EJ-Vol26-NoSI-2

Flatabo

Doorman

Grande

, et al. Experience with the Nord Pool design and implementation. IEEE Trans Power Syst 2003; 18: 541–547.

Prljaca

Jørgensen

. The international electricity market infrastructure-insight from the Nordic electricity market. In 2016 13th international conference on the European energy market (EEM) (pp. 1-5). IEEE. 2016, June.

Unger

Ulfarsson

Gardarsson

, et al. The effect of wind energy production on cross-border electricity pricing: the case of western Denmark in the Nord Pool market. Econ Anal Policy 2018; 58: 121–130.

Karabiber

Xydis

. Electricity price forecasting in the Danish day-ahead market using the TBATS, ANN and ARIMA methods. Energies 2019; 12: 28.

10.

Gallego

. Inverse Optimization and Forecasting Techniques Applied to Decision-making in Electricity Markets (Doctoral dissertation, PhD thesis, Technical University of Denmark (DTU)). 2017.

11.

NASDAQ OMX. Retrieved from European Commodities: http://www.nasdaqomx.com/commodities (assessed: Dec. 06, 2021). 2021.

12.

Kolberg

Waage

. Artificial intelligence and Nord Pool’s intraday electricity market Elbas: A demonstration and pragmatic evaluation of employing deep learning for price prediction. Bergen, Norway: Norwegian School of Economics, 2018.

13.

Ilea

Bovo

. Integrated European intra-day electricity market: rules, modeling and analysis. Appl Energy 2019; 238: 258–273.

14.

Panagiotidis

Effraimis

Xydis

. An R-based forecasting approach for efficient demand response strategies in autonomous micro-grids. Energy Environ 2019; 30: 63–80.

15.

Energinet. Renewable Energy in the Danish Energy System, Retrieved from: https://en.energinet.dk/Green-Transition/Renewable-energy-in-the-energy-system#Wind, Accessed: Apr. 4, 2022. 2021.

16.

Rasmussen

Enevoldsen

Xydis

. Transformative multivalue business models: a bottom-up perspective on the hydrogen-based green transition for modern wind power cooperatives. Int J Energy Res 2020; 44: 3990–4007.

17.

Roungkvist

Enevoldsen

Xydis

. High-resolution electricity spot price forecast for the Danish power market. Sustainability 2020; 12: 4267.

18.

Jaraitė

Kažukauskas

. The effects of wind power on electricity markets: a case study of the Swedish intraday market. Energy Econ 2021; 96: 105159.

19.

Michalitsakos

Mihet-Popa

Xydis

. A hybrid RES distributed generation system for autonomous islands: a DER-CAM and storage-based economic and optimal dispatch analysis. Sustainability 2017; 9: 2010. doi:10.3390/su9112010

20.

Skajaa

Edlund

Morales

. Intraday trading of wind energy. IEEE Trans Power Syst 2015; 30(6): 3181–3189. doi:10.1109/TPWRS.2014.2377219

21.

Borggrefe

Neuhoff

. Balancing and intraday market design: Options for wind integration. 2011.

22.

Wilson

. Essentials of business research: a guide to doing your research project. Sage, 2014.

23.

Barriball

While

. Collecting data using a semi-structured interview: a discussion paper. J Adv Nurs-Inst Subscription 1994; 19: 328–335.

24.

Hsieh

Shannon

. Three approaches to qualitative content analysis. Qual Health Res 2005; 15: 1277–1288.

25.

Hyndman

Athanasopoulos

. Forecasting: principles and practice. OTexts, 2018.

26.

Karabiber

Xydis

. Forecasting day-ahead natural gas demand in Denmark. J Nat Gas Sci Eng 2020; 76: 103193.

27.

Monteiro

Guimarães

Moura

, et al. Estimating photovoltaic power generation: performance analysis of artificial neural networks, support vector machine and Kalman filter. Electr Power Syst Res 2017; 143: 643–656.

28.

Andrade

Pinto

Praça

, et al. UCB1 based reinforcement learning model for adaptive energy management in buildings. In: International symposium on distributed computing and artificial intelligence. Cham.: Springer, 2018, June, pp.3–11.

29.

Kiesel

Paraschiv

. Econometric analysis of 15-minute intraday electricity prices. Energy Econ 2017; 64: 77–90.

30.

Narajewski

Ziel

. Estimation and simulation of the transaction arrival process in intraday electricity markets. Energies 2019; 12: 4518.

31.

Contreras

Espinola

Nogales

, et al. ARIMA models to predict next-day electricity prices. IEEE Trans Power Syst 2003; 18: 1014–1020.

32.

Hastie

Tibshirani

Friedman

, et al. The elements of statistical learning: data mining, inference, and prediction (Vol. 2, pp. 1–758). New York: Springer, 2009.

33.

Bergmeir

Benítez

. On the use of cross-validation for time series predictor evaluation. Inf Sci (Ny) 2012; 191: 192–213.

34.

Sagiroglu

Sinanc

. Big data: a review. In: 2013 International conference on collaboration technologies and systems (CTS) (pp. 42–47). IEEE, 2013, May.

35.

NordPool. Order types. Retrieved from Nordpool: https://www.nordpoolgroup.com/trading/intraday-trading/order-types/ (Assessed: Dec. 06, 2021). 2021.

36.

Shumway

Stoffer

. Time series analysis and its applications (Vol. 3). New York: Springer, 2000.

37.

James

Witten

Hastie

, et al. An introduction to statistical learning (Vol. 112, p. 18). New York: Springer, 2013.

38.

Larsen

Goutte

. On optimal data split for generalization estimation and model selection. In Neural Networks for Signal Processing IX: Proceedings of the 1999 IEEE Signal Processing Society Workshop (Cat. No. 98TH8468) (pp. 225-234). IEEE. 1999 August.

39.

Chatfield

. The analysis of time series: an introduction. Chapman & Hall/CRC, 2003.

40.

Aydin

Jang

Topal

. Energy consumption modeling using artificial neural networks: the case of the world’s highest consumers. Energy Sour, Part B: Econ Plann Policy 2016; 11: 212–219.

41.

Aydin

. Modeling of energy consumption based on economic and demographic factors: the case of Turkey with projections. Renew Sustain Energy Rev 2014; 35: 382–389.

42.

Muñoz-Olivas

. Screening analysis: an overview of methods applied to environmental, clinical and food analyses. TrAC, Trends Anal Chem 2004; 23: 203–216.