Spatially Explicit Prediction of Wholesale Electricity Prices

Study Area

PJM is a regional transmission organization (RTO) that coordinates the movement of wholesale electricity in all or parts of thirteen states and the District of Columbia. Figure 1 provides a map of the different zones within the interconnection. Our study is not the first to use spatial panel data models to analyze and exploit the spatial nature of LMPs. Namely, Douglas and Popova (2011) contributed to the literature by developing a spatial weighting matrix based upon the constraints within the transmission system and by offering an explanation of different types of spatial panel data models. Although certainly a contribution to the literature, the authors did not offer a theoretical explanation or justification for using the spatial panel data model and their focus of analysis was on only one particular type of spatial panel data model.

OPEN IN VIEWER

Figure 1.

Zones within the PJM (2013) interconnect.

Literature Review

Previous analyses of wholesale electric markets have generally either (1) abstracted away from the details of the transmission system and focused on the strategic behavior among generators or (2) given more detailed examinations of the transmission system including the strategic behavior among generator. The latter often included sophisticated engineering or operations research models to analyze the transmission system constraints. Examinations of the former include the seminal work by Green and Newbery (1992) who modeled a supply function equilibrium with applications to the UK market. Despite the article’s contribution, the authors almost completely ignored the transmission system constraints. Two follow-up works expanded the models to consider strategic behavior among asymmetric firms (Green 1996, 1999). In response to Green and Newbery’s (1992) findings, von der Fehr and Harbord (1993) developed a game theoretic model to explain strategic interaction at different times of demand throughout the day. The authors found different equilibria outcomes based upon the congestion constraints within the transmission system, for example, they found that during periods of low demand a standard Bertrand equilibrium is the optimal solution. ¹ The contribution by the latter study was in understanding that the constraints within the system have an effect on prices and behavior among independent operators.

The latter type of analyses includes applications of transportation models to analyze the exogenously determined transportation costs between markets (Schmalansee and Golub 1984). Other models have offered an explicit treatment of the network including some that incorporate the engineering principles governing the flow of electricity within the system (Uri 1976; Hobbs 1986a, 1986b; Hobbs and Kelly 1992; Borenstein et al. 1995; Borenstein and Bushnell 1999; Borenstein, Bushnell, and Stoft 2000). Haldrup and Nielsen (2006) developed a model that explicitly incorporates the dependence of congestion probabilities, and how it affects multilateral price behavior.

More specifically, Haldrup and Nielsen (2006) used a regime switching model to reflect changes in wholesale electricity price behavior between periods of congestion and noncongestion in the transmission system. Guerci and Sapio (2012) employed an agent-based model to investigate the increase of geographically concentrated wind capacity into a transmission system and its effect on wholesale electricity prices. As in Haldrup and Nielsen (2006), Guerci and Sapio’s (2012) approach explicitly models congestion probabilities in the transmission system.

Until recently, no previous works (to the authors’ knowledge) have used spatial econometric models to analyze wholesale electric markets. Spatial econometrics is an applied field of econometrics that deals with sample data that are collected with reference to location measured as points in space. What distinguishes spatial econometrics from traditional econometrics is that the locational data, such as nodes within a transmission system, may be characterized by spatial dependence or spatial heterogeneity (LeSage and Pace 2009). The idea of spatial dependence, or technically spatial autocorrelation, is similar to the concept of temporal autocorrelation found within the time series literature. As in time series, if this autocorrelation is present and unaccounted for, then it could lead to inefficient estimates and biased estimates of standard errors of parameter estimates. Traditional econometrics had largely ignored spatial autocorrelation until the development of spatial econometrics. Recent advances in spatial econometrics have led to the development of longitudinal or panel data models that control for spatial autocorrelation. Longitudinal data are simply cross-section observations collected over time. These models offer the dual benefit of potentially controlling for province level unobserved or heterogeneous fixed effects (FEs) and spatial dependence.

Based upon this idea of spatial dependence, Douglas and Popova (2011) exploited the geographic nature of spot prices within a transmission system to analyze a wholesale electricity market with spatial panel data econometric models. The major thrust of the article was with a spatial error model (SEM; to be discussed in Empirical Strategy subsection) estimated by a general method of moments (GMMs) approach. Using this approach, the authors found statistically significant spatial patterns in electricity prices. The major contribution of Douglas and Popova (2011) was in using the spatial weighting matrix (discussed in greater detail subsequently) to serve as instrument for the constraints within the transmission system.

Theoretical Model

In an electricity market, a node is simply the physical or geographic location on a transmission grid where the electricity is delivered or withdrawn. The grid is a vast, synchronized network of electricity generators and transmission lines. A zone within the grid is an aggregation of all the nodes within a distinct geographic region. Since electricity is largely nonstorable, it must be generated and dispatched immediately to meet demand at any one time in the network or at any one node. In order for the grid to operate safely and efficiently, demand and supply must be in balance at all times, otherwise the grid system is subject to failures. The challenge for the RTO (or independent system operator [ISO]) ² is to balance the supply and demand within the physical limits of the system, which is subject to certain thermal and contingency constraints. Thermal constraints refer to the amount of power that can flow through a given transmission or distribution line at one time. Too much power can overheat lines and melt the physical wires within the line. Contingency constraints refer to restricting the transmission network to operate at levels that can instantaneously withstand the loss of key elements within the network (Bushnell and Stoft 1996).

We will assume that at any node in the market electricity is injected into the transmission system, and the electricity flows through the system in a linear network. We assume that the network contains n nodes indexed by i ∊ (1, …, n) and m lines indexed by l ∊ (1, …, n) × (1, …, n). We will also take one arbitrary node to be “swing bus,” which has a level of injection that is determined by the net injections at the other nodes in the system (Schweppe et al. 1988). A bus in network topology is simply a shared line between different nodes in the system.

After defining the swing bus, one can calculate the flow of electricity through all the lines as a function of the injections in the other n – 1 nodes in the system. More specifically, the connectivity within the network allows one to calculate an (n – 1) × m “transfer admittance matrix,” which we denote by H. In the electrical engineering literature, a transfer admittance matrix or nodal admittance matrix, in its simplest form, is an n × n matrix describing a power system with n buses—it is simply a power flow model. Let us denote z_l as the thermal limit within each transmission line, y_i as the amount of electricity injected into the line from node i, and h_li as an element of the H matrix. This element represents the fraction of power injected at node i that flows through line l. Given these assumptions, the total flow of line l can be expressed as:

z l = ∑ i = 1 n − 1 h l i y i or z = H y .

1

The second term in equation (1) represents equation in matrix notation. For a more detailed explanation of the admittance matrix presented here, the reader is referred to Schweppe et al. (1988).

Equation (1) indicates the power flows through the system given any set of injections (dispatch), but it does not include thermal constraints or limits imposed on the transmission lines. If the line is at its maximal thermal limit, then dispatch is infeasible. Therefore, we define line limits as follows:

− z _ l ≤ z l ≤ z ¯ l ∀ l ,

2

where z _ l and z ¯ l represent the lower and upper thermal limits of the line, respectively. When these limits are combined with the transfer admittance matrix we have

− z _ ≤ H y ≤ z ¯

3

where z _ and z ¯ now denote column vectors of the lower and upper thermal limits within all the lines, l. (For illustrative purposes, we will drop the lower thermal limit, z _ , in the cost minimization problem subsequently. Theoretically, the transmission line is constrained when it reaches either of the thermal limits. However, in the context of our current problem, the line is generally constrained by the upper thermal limit, z ¯ , so we will only consider the upper limit as a constraint in the proceeding cost minimization problem.)

Optimal nodal pricing problem

Following Schweppe et al. (1988) and Bushnell and Stoft (1996), we provide a generalized approach for computing the optimal spot prices at each node in a wholesale electricity market. To help motivate the model, we provide a definition of all the variables in Table 1.

OPEN IN VIEWER

Table 1.

Nomenclature.

Symbol or Acronym	Definition
General
RTO	Regional transmission organization
LMP	Locational marginal (or nodal) prices
PJM	Pennsylvania–Jersey–Maryland interconnect system
Theoretical Model
n	Number of nodes in the transmission system
yi	Injection of electricity at node i
Ci (yi)	Cost of supply at node i
zl	A power flow on line l
z ¯ l	The maximum power flow on line l
H	Transfer admittance matrix which computes the flow on lines from injections of electricity
Regression Models
ρ	Spatial autocorrelation coefficient (spatially lagged dependent variable)
τ	Spatial autocorrelation coefficient (temporally lagged dependent variable)
δ	Spatial autocorrelation coefficient (spatially lagged error term)
W	Spatial weights matrix
β	(Nonspatial) coefficient on the explanatory variable(s)
γ	(Nonspatial) coefficient on the temporally lagged dependent variable
BE	Two-way panel effects (i.e., fixed effects and temporal effects)
SAR	Spatial autoregressive model
SEM	Spatial error model
SDPD	Spatial, dynamic panel data model
Forecasting Metrics
MAE	Mean absolute error
MAPE	Mean absolute percentage error
RMSE	Root mean square error
Theil’s U	Forecasting error statistic
DM stat	Diebold Mariano statistic

We assume that there are clearly defined cost functions at each node in the network. We let C_i (y_i ) denote the cost of supplying y megawatts of electricity at node i. Abstracting away from transmission losses, the optimal dispatch problem is

min y i ∑ i C i ( y i ) , Subject to , H y ≤ z ¯ ∑ i y i = 0 .

4

The objective function in equation (4) can be interpreted as a cost minimization problem subject to two constraints. Intuitively, the first constraint implies that the amount of electricity flowing into the lines cannot exceed the upper thermal limits of the lines. The second constraint is an expression of Kirchhoff’s Current Law, which is a statement about the conservation of energy. Formally, the law states that the vectorial sum of all currents at a node or bus is equal to zero (Behrendt, Costello, and Zocholl 2010). More intuitively, Kirchhoff’s law implies that the total charge flowing into a node must be the same as the total charge flowing out of the node. Therefore, the sum of all the currents is zero. (Please note that equations [1]–[4] do not directly incorporate Kirchhoff’s Current Law explicitly into the objective function. It is possible to do so, as in Schweppe et al. 1988. However, to keep the model tractable, we abstract away from explicit inclusion of the law into the objective function.)

We expand the model to compute prices when transmission constraints are binding within the system. For this, we formulate a Lagrangian, and let p denote the Lagrange multiplier on the energy balance constraint (the second constraint in equation [4]) and λ_l denote the multiplier for the flow constraint (the first constraint in equation [4]) on line l. Given these the assumptions, the Lagrangian can be specified as follows:

min y i L = ∑ i C i ( y i ) − p ⋅ ( ∑ i y i ) − ∑ l λ l ⋅ ( ∑ i = 0 n − 1 h l i y i − z ¯ l ) .

5

The optimal solution for equation (5) is obtained by differentiating the Lagrangian with respect to the nodal injection y_i :

C i / ( y i ) = p + ∑ l = 1 m λ l h l i i = 1 , … , n − 1 C n / ( y n ) = p swing bus.

6

The power system depends on the flows in all the lines and thus on the net injections as represented by transfer admittance matrix in equation (3). The admittance matrix cannot depend on injections at all the nodes according to Kirchhoff’s Current Law (Kirschen and Strbac 2004). To get around this difficulty, the n bus in the system is designated as the swing or slack bus and the injection at this bus is omitted from the first equation in equation (6). Given all the other net injections in equation (6), the injection of the swing bus can be adjusted to satisfy the optimization condition (Kirschen and Strbac 2004). The concept of the swing bus is purely mathematical and has no physical implications (in terms of one node or bus in the grid system being designated as the swing bus), so the choice of the swing bus is purely arbitrary, and we have chosen bus n as the swing bus.

The Lagrange multiplier p thus represents the marginal cost or marginal benefit of an injection of power at the swing bus. In a purely competitive network of finite capacity, p represents in the nodal price at the swing bus. The nodal prices at the other buses are related to the price at the swing bus by combining the equations in equation (6):

C i / ( y i ) = C n / ( y n ) + ∑ l = 1 m λ l h l i , i = 1 , … , n − 1 .

7

If an increase in the net injection at node i adds to losses in the system (i.e., assuming h_li > 0), then

C i / ( y i ) > C n / ( y n ) ,

8

and the nodal price paid to generators at node i is smaller than the nodal price at the swing bus to penalize the generators for the additional losses they would cause by injecting an increment of power in the network at that node (Kirschen and Strbac 2004). Consequently, consumers at node i pay a lower price because an increase in load at that bus would reduce the losses. The opposite holds true if an increase in the net injection at node i reduces the losses in the system. If all losses are neglected, then the nodal prices at all buses are equal (Kirschen and Strbac 2004).

The Kuhn–Tucker necessary conditions are ³

λ l ⋅ ( ∑ i = 1 N − 1 h l y i − z ¯ l ) = 0 ∀ l , λ l ≤ 0.

9

The optimal solution for the cost minimization problem, when transmission constraints are binding, implies that the marginal cost of injecting y_i at any node is equal to the sum of Lagrange multiplier, p, and the constraints on the transmission lines. Another way to interpret the first Lagrange multiplier, p, is as the optimal nodal spot price, which implies that marginal costs should equal the spot price plus the constraints within the system. This implies that if independent operators had perfect knowledge, then the market clearing prices for supplying electricity within the system should reflect the theoretical spot price while taking into consideration the transmission constraints within the system. The line constraints, λ _l , then can be interpreted as the shadow prices of the transmission lines within the system (Li, Liu, and Salazar 2006). These shadow prices represent the marginal benefit to the system of increasing the thermal limit on a line.

The shadow prices can alternatively be interpreted as congestion costs in the system. At times of low demand, for example, in the late evening hours, there is little or no congestion within the system (i.e., the transmission lines are not at their thermal limit), and so the shadow prices are generally negligible. Hence, during periods of low demand, the prices at each node should be nearly identical. The competing generators rely upon the transmission network to schedule and dispatch their produced electricity, and the sale of electricity is organized through spot and forward markets and through bilateral contracts with the end-use customer or marketing intermediaries (Joskow and Tirole 2000).

Bushnell and Stoft (1996) offer the important observation that the “optimal spot price” at a node is the “average of the prices at all other nodes” weighted by their relative change in supply when power is injected at that node. Although the authors’ observation is not directly obvious from the solution in equation (6) above, their statement can be understood from intuition. The solution in equation (6) is for the optimal spot price at one single node in the system. However, there are multiple nodes in the system. So as additional operators inject electricity into the system then congestion is created, which in turn make the thermal constraints binding. Therefore, if all operators behave optimally, as in equation (6), then they will consider these constraints within the system and offer a price to the ISO that reflects this information, and the price then at one node is a reflection of the average of prices at the other spatial nodes. In the current study, we use this spatial information in our empirical approach to better predict prices in a wholesale electricity market.

The correspondence between the optimal solution in equation (6) and the spatial relationship in nodal spot prices may not be immediately obvious. One of the underlying assumptions in the above optimization problem is that an alternating current (AC) is being used in the transmission network, which corresponds with empirical reality in PJM. The problem with AC is that the electrical current can move in multiple directions, and, therefore, operators cannot entirely control the flow of electricity through the grid system. An additional assumption involves the sensitivity of the flow of electricity along line l and in relation to the injection at node i—this is not necessarily explicitly modeled in the optimization problem but more on this subject can be found in Kirschen and Strbac (2004) and Stoft (2002). Solving equation (6) can be computationally difficult because it implicitly involves the solution of the power flow equations and potentially can be nonlinear (Kirschen and Strbac 2004). Consequently, the spatial dependence within the underlying system is not modeled explicitly in the optimization problem above because it would only further complicate the model. That is, the nodal price is affected by the flow constraint on a single line, the thermal constraint (represented by the transfer admittance matrix), and the sensitivity of the flow at bus m to the net injection at node i. Instead, we discuss the spatial dependence by appealing to the intuition derived from equations (7) and (8) above.

Spatially Weighted Matrix

In spatial econometrics, the spatial weight matrix is simply a compact reflection of the geographical relationship among different points in space. For example, a geographical contiguity matrix is a binary matrix with a one indicator if two points are neighbors and a zero indicator otherwise. The spatial weight matrix can be written as follows:

∑ i ≠ j w i , j = W ,

10

where w_i,j denotes the indicator value of whether referenced point i shares some sort of neighboring relation with point j. For ease of notation, spatial weight elements can be represented as a matrix W.

There are many different specifications for a spatial weighted matrix, such as distance weighted matrix, inverse-distance weighted matrix, and k-nearest neighbors matrix. One may argue that the specification of the spatial weight matrix will substantially affect the results of the estimated spatial autocorrelation coefficient. However, LeSage and Pace (2010) demonstrate that there is little theoretical basis for this commonly held belief. Therefore, we will rely upon a simple binary continuity matrix as described above in this study.

Data

Our data consist of PJM’s “Real-time Market,” which are a spot market for current LMPs (or nodal prices) that are calculated at five-minute intervals based on actual grid operating conditions (PJM 2013). Transactions are settled hourly and PJM issues invoices to market participants monthly. PJM provides hourly aggregated values of the real-time LMPs. The LMPs then are historical values, which are listed hourly by zone for each day of market operation. For proprietary reasons, PJM takes an average of the nodal prices to represent the prices by zone. A map of the zones is provided in Figure 1 above.

In addition to the real-time market, PJM also provides a day-ahead energy market, which is a forward market for hourly LMPs that are calculated for the next operating day based on generation offers, demand bids, and scheduled bilateral transactions (PJM 2013). The day-ahead markets provide accurate price signals for market participants to make informed decisions about the next day’s market operations. For example, if there are planned outages or transmission disruptions, then such information may be reflected in the day-ahead market prices.

The participation in day-ahead electricity markets opens arbitrage opportunities for power producers. When the producer’s real-time imbalance with respect to the day-ahead contract is in the opposite direction compared to the overall system imbalance, power producers received a more favorable price at the balancing market (Morales et al. 2014). Specifically, producers can sell excess energy compared to their day-ahead position at a higher price than the day-ahead price when there is a deficit of power production in the system as a result of deviations from producers and consumers with respect to their day-ahead positions (Morales et al. 2014). Conversely, when the grid system has a surplus of power, producers can purchase their production deficit at a lower price.

Borenstein et al. (2008) and Saravia (2003) have examined the day-ahead energy market and found a spread between the day-ahead and real-time hourly price on the US power markets. Both authors attribute the spread to market power and speculative activity. In a similar study, the authors analyzed the difference between the day-ahead price and expected real-time price, the risk premium, and found that premiums are affected by demand, sales, and price variation (Longstaff and Wang 2004). Forbes and Zampelli (2011) examined the day-ahead markets in California and found that day-ahead price reflected processed information and expectations of all market participants regarding day-ahead demand. Huisman, Huurman, and Mahieu (2007) found that day-ahead prices are mean reverting, the speed of mean reversion differs by hour of day, and prices exhibited a block structured cross-sectional correlation pattern.

In this study, we collected the hourly real-time spot and day-ahead prices in the eighteen zones that existed in PJM interconnection from January 1, 2012 to October 31, 2013. For each of the 670 days in the total sample, we have downloaded price data for each of the twenty-four hours of the day from PJM website.

Regression Models

There are N = 18 cross-sectional zones, observed over T = 670 days in our data set. In this study, we will observe the estimation and prediction of each current hour. Our empirical model is

p i t = x it β + µ i + η t + ∊ i t

11

where p_it is the real-time price for the hour of the day t in the zone i, and x_it is a vector of explanatory variables including a constant term and the day-ahead forward price for the hour of the day and region corresponding to p_it . The term µ _i denotes the individual effect (or heterogeneity) for each zone within PJM. The term η _t denotes the time FEs, which control for shocks that occur to all zones simultaneously through time; an example of such a shock may include a sudden climatic event, such as a really hot or cold day of the year that stimulates electricity demand in all zones.

In this study, we treat the individual effect as fixed meaning that we assume that this variable is correlated with the explanatory variable and approximately “fixed” over time for each zone within the sample. If we allow for FEs terms to enter into the error term and estimate equation (11) without controlling for it, then the estimates will result in omitted variable bias (i.e., if the FE is correlated with the explanatory variables). To control for FEs we can either (1) estimate µ _i and η _t directly in the model by creating dummy variables for these parameters as in a least squares dummy variable model or (2) we could demean the data as in a FEs or within estimator. In this study, we use the FE estimation method. The individual and time FE can be eliminated by transforming the data as follows (Elhorst 2012):

q i t = q i t * − 1 T ∑ t = 1 T q i t − 1 N ∑ i = 1 N q i t + 1 N ⋅ T ∑ t = 1 T ∑ i = 1 N q i t ,

12

where q i t * denotes any of the nontransformed variables (dependent or independent) within the study.

Empirical Strategy

Spatial econometric models have come under criticism recently for problems associated with identification and for a lack of appeal to theoretical foundations (Partridge et al. 2012). According to these criticisms, the problem of identification is similar to Manski’s (1993) “reflection problem,” where group average characteristics (neighboring electricity prices) affect individual outcomes (local electricity prices), but the parameters in the model are not identifiable. That is, it is hard to separate out the effects of what causes prices to fluctuate locally versus what causes price fluctuations in neighboring regions.

We agree that there may be potential problems with the exogeneity of the spatial weighting matrix (among other things), so rather than appealing to causality we focus primarily on the alternative validation strategy of prediction, which is less dependent on prior theory. In other words, we take the spatial panel models as a black box and test them against empirical reality (Freedman 1991).

Given our assumptions of the spatial weighting matrix, we now demonstrate the empirical model used to identify the optimal theoretical solution provided in equation (6) above. We assume that independent operator (electricity producer) has constant marginal costs, α, so equation (6), excluding the swing bus, can be rewritten as:

p = α − ∑ l = 1 m λ l h l i i = 1 , … , n − 1 .

13

As discussed above, λ _l is a parameter that represents the shadow prices of the thermal constraints within the system. Equation (13) represents how the transmission system is managed. Due to the complexity of the transmission system, it would be incredibly challenging to develop a (structural) statistical model that captures the thermal limits within the system, and it would be difficult if not impossible to acquire data on the thermal limits. However, because of the thermal constraints, we know that there are spatially dependent relationships between prices at each node in the system. One way to model these relationships is to impose a structure on these spatially dependent relations. Ord (1975) proposed a parsimonious parameterization for the dependence relations—namely, the spatially weighted matrix discussed above. In other words, the second term on the right-hand side of equation (13) can be proxied by a simple continuity spatial weight matrix. For example, we instead could use a spatial econometric model to estimate the relationship in equation (13), such as ⁴

p i t = ρ ∑ j = 1 N w i , j p j t + β x i t + ∊ i t , ∊ i t ∼ N ( 0 , σ 2 ) .

14

Equation (14) is a reduced-form estimable model corresponding to equation (13) or its structural counterpart in equation (6). The structure of equation (14) gives rise to a generating process known as the spatial autoregressive (SAR) process, which captures Bushnell and Stoft’s (1996) observation that the optimal nodal price at one location is a reflection of the average nodal price at neighboring locations. The spatial weight matrix then is simply a compact representation of the transmission system (as represented by the price behavior within the grid). Note that equation (14) does not explicitly capture the management aspects of the transmission system but rather implicitly captures management decisions as reflected in the price behavior of all the nodes within the system.

In addition of the SAR model, we examine different specifications of spatial econometric models as well. The grouped spatial econometric model can be written as:

p i t = ρ ∑ j = 1 N W i j p j t + γ p i , t − 1 + τ ∑ j = 1 N W i j p j , t − 1 + β x i t + μ i + η t + φ i t φ i t = δ ∑ j = 1 N W i j φ j t + ∊ i t , ∊ i t ∼ N ( 0 , σ 2 ) ,

15

where ρ denotes the scalar SAR parameter on the dependent variable, γ is a scalar parameter on the temporally lagged dependent variable, τ is the spatial autocorrelation coefficient on the temporally lagged dependent variable, and δ is the spatial autocorrelation coefficient on the error term.

The restriction of the parameters within equation (15) defines the specific type of spatial panel data model used. The SAR model could be obtained by restricting γ, τ, and δ equal to zero—this model exhibits spatial dependence within only the dependent variable. The SEM is obtained by restricting φ, γ, and τ equal to zero—this model exhibits spatial dependence within only the error term. The spatial dynamic panel data (SDPD) model is obtained by restricting δ equal to zero—this model allows for spatial dependence within both the dependent variable and the temporally lagged dependent variable. If all the parameters except γ and β are restricted, then we derive the dynamic panel data model with FEs. Finally, if all the parameters except for β are restricted, then the model reduces to the traditional pooled OLS model with both FEs.

Note that our modeling approach differs paradigmatically from Douglas and Popova (2011) who focus on a spatial econometrics model estimated by the GMMs approach of Kapoor, Kelejian, and Prucha (2007). Further, Douglas and Popova primarily focus on a SEM specification. The advantage of the GMM approach is that it is distribution free, whereas the quasi-maximum likelihood approach, adopted here, assumes that the pricing data are normally distributed. Is this a problem for our analysis? Perhaps, but since our focus is on prediction, not causality, this is not as much of an issue. The quasi-maximum likelihood approach is based on the model developed by Yu, de Jong, and Lee (2008, 2012), and Matlab code was provided by the authors (Yu, de Jong, and Lee) for our particular application. The benefit of using maximum likelihood is that it allows us to experiment with several different spatial specification models, and the examination of the different specifications presents a contribution to the literature. The potential drawback with Yu, de Jong, and Lee’s method is that the asymptotic theory relies upon a relatively large cross section (N) and a relatively large sample along the time dimension (T). Our empirical approach has a very large sample along the time dimension but a relatively small sample (eighteen zonal regions within PJM) within the cross section. Despite this potential deficiency, Yu, de Jong, and Lee argue that when T is large relative to N, their model still produces consistent and asymptotically normal estimates. If our empirical approach were to focus more on the interpretation of parameters and make appeals to causality, then the relatively small cross-section sample may present a problem. However, since we are focusing on the forecasting ability of this particular model, the relatively small cross section should not be as problematic.

Forecasting Metrics

Three common metrics are used to evaluate forecast accuracy: mean absolute error (MAE), mean absolute percentage error (MAPE), and root mean square error (RMSE). These metrics are defined as

MAE = ∑ t = 1 T ∑ i = 1 N 1 N ⋅ T | F ( t ) − A ( t ) | ,

16

MAPE = ∑ t = 1 T ∑ i = 1 N 1 N ⋅ T | F ( t ) − A ( t ) A ( t ) | ,

17

RMSE = { ∑ t = 1 T ∑ i = 1 N 1 N ⋅ T [ F ( t ) − A ( t ) ] 2 } 1 / 2 .

18

The symbol T denotes the total number of time periods, and N denotes the total number of zones within the system. The symbol F(t) denotes the forecasted value, and A(t) denotes the actual empirical observation. The difference between the forecasted value and the empirical observation denotes the forecast error. Therefore, the smaller the forecasted error, the better the model predicts future values. These metrics are often called ex post forecasts because the independent variables do require forecasts themselves (Greene 2003). According to Kennedy (2008), MAE is appropriate when the cost of forecast errors is proportional to the absolute size of the forecast error. MAPE is the average of the absolute values of the percentage forecast errors, and it has the advantage of being dimensionless. MAPE is more appropriate when the cost to forecast error is more closely related to the percentage error than to the numerical size of the error (Kennedy 2008). A problem with MAPE is that it often performs underforecasting. The errors in the RMSE metric are squared before averaging, so the RMSE gives a relatively higher weight to large errors—ergo, RMSE represents a “quadratic loss function.” RMSE is one of the most popular metrics in use.

Another metric for evaluating prediction is based on the Theil U statistic (Theil 1966; Greene 2003). The potential problem with the ex post forecast metrics above is that those metrics have scaling problems. The Theil U statistic is scale free and measured as follows:

U = { ∑ t = 1 T ∑ i = 1 N 1 N ⋅ T [ F ( t ) − A ( t ) ] 2 } 1 / 2 { ∑ t = 1 T ∑ i = 1 N 1 N ⋅ T F ( t ) 2 } 1 / 2 + { ∑ t = 1 T ∑ i = 1 N 1 N ⋅ T A ( t ) 2 } 1 / 2 .

19

The ex post forecast metric measures the “fit” of the predictive models, whereas the Theil U statistic measures how well the models perform against “naive” models, such as a random walk. The Theil U statistic is similar to the measure for R ² (Greene 2003). With the specification in equation (19), the statistic is bounded between zero and one, and a larger value indicates poor forecasting performance. For the sake of robustness, we consider all the ex post and Theil U forecasting metrics in the current study.

Forecasting Results

In order to carry out the forecasts, we shorten the full sample of observations by omitting the last s days of observations, where s denotes different prespecified lengths of time. Specifically, we will explore three out-of-sample specifications: (1) short term (s = one week), (2) the medium term (s = one month), and (3) the near term (s = three and six months, respectively).

We define the s number of observations as the “out-of-sample” observations. Note that we still observe the last s number of observations, only we exclude these observations from the regression so that can use the s observations to compare against forecasted values over the same period. In other words, we run regressions on the shorter “within sample” and then use the forecasts against empirical reality to see which model provides the best predictions for the out-of-sample data.

We compute the prediction for the ith individual zone at a future period t + s, in which t is the number of days within sample, t + s is the total number of days of full sample, which equals 670. The term s denotes the number of days for the forecasting period. The forecasts are conducted by regressing the model on the entire initial within sample, and then forecasting over the entire out-of-sample period using the empirical observations of the independent variable within the out-of-sample period.

We used our PJM price data set to calculate and compare the accuracy of pooled OLS, nonspatial FEs, dynamic panel data model with FEs, SAR, SEM, and SDPD forecasts. The results of the forecast error performance, in the context of MAE, MAPE, RMSE, and the Theil U statistic, of each of the models are presented in Tables 4–7. Weron and Misiorek (2007) forecast electricity spot prices with a series of Gaussian and heavy-tailed innovations to improve the power of the forecasts. In the current study, we did not experiment with heavy-tailed innovations because it would significantly complicate the estimation of the spatial econometric forecast models. Instead, we leave such extensions for future research.

OPEN IN VIEWER

Table 4.

Short-run Forecast Metrics for the First Half-day of Business Operation.

Time of Day	Metrics	One Week						One Month
		Nonspatial Models			Spatial Models			Nonspatial Models			Spatial Models
		Pooled OLS	BE	Dynamic BE	SAR + BE	SEM + BE	SDPD + BE	Pooled OLS	BE	Dynamic BE	SAR + BE	SEM + BE	SDPD + BE
1	MAE	0.2570	0.2521	0.2564	0.2606	0.2451	0.2097	0.2443	0.2308	0.2435	0.2306	0.2310	0.1874
	MAPE	1.6454	1.5752	1.6612	1.0652	1.7168	0.8456	26.5328	16.1593	25.4624	19.6416	17.9501	12.3777
	RMSE	0.3100	0.2989	0.3104	0.3034	0.2994	0.2454	0.3091	0.2934	0.3090	0.2779	0.2964	0.2274
	Theil’s U	0.2096	0.2116	0.2095	0.1892	0.2014	0.1556	0.2473	0.2440	0.1000	0.2033	0.2343	0.1695
2	MAE	0.2456	0.2634	0.2756	0.1979	0.2570	0.1514	0.2402	0.2261	0.2376	0.1830	0.2258	0.1365
	MAPE	1.2083	1.0640	1.1994	1.4193	1.1156	0.8380	1.6471	1.5054	1.6323	1.5719	1.5954	1.1167
	RMSE	0.3426	0.3306	0.3415	0.2629	0.3277	0.2099	0.2960	0.2818	0.2929	0.2350	0.2854	0.1786
	Theil’s U	0.2543	0.2492	0.2536	0.1834	0.2392	0.1506	0.2717	0.2607	0.2406	0.2030	0.2554	0.1596
3	MAE	0.4820	0.2159	0.4738	0.2045	0.2222	0.1595	0.4743	0.1948	0.4327	0.1734	0.2006	0.1394
	MAPE	2.2485	1.1498	2.2073	1.2946	1.2138	1.0054	4.6201	1.8277	4.1738	1.7412	1.9409	1.4943
	RMSE	0.5375	0.2941	0.5301	0.2594	0.3056	0.2116	0.5262	0.2527	0.4874	0.2186	0.2626	0.1784
	Theil’s U	0.4491	0.2346	0.4440	0.1977	0.2363	0.1644	0.5475	0.2624	0.6876	0.2184	0.2641	0.1814
4	MAE	0.4163	0.2019	0.3983	0.2001	0.2211	0.1649	0.3995	0.1936	0.3604	0.1749	0.2059	0.1433
	MAPE	2.5693	1.5740	2.4297	1.2519	1.7028	1.0913	15.6638	6.8959	14.3093	4.4987	7.4380	4.2064
	RMSE	0.4650	0.2851	0.4459	0.2553	0.3099	0.2135	0.4523	0.2566	0.4117	0.2195	0.2740	0.1824
	Theil’s U	0.4096	0.2327	0.3957	0.2038	0.2429	0.1729	0.5102	0.2750	0.3204	0.2324	0.2821	0.1956
5	MAE	0.3303	0.2068	0.3299	0.2038	0.2081	0.1717	0.3099	0.2019	0.2992	0.1818	0.2047	0.1591
	MAPE	2.3132	1.7849	2.3350	1.9323	1.8908	1.6439	2.5348	2.2896	2.5128	1.9275	2.4232	1.7158
	RMSE	0.3895	0.2861	0.3902	0.2633	0.2896	0.2335	0.3649	0.2629	0.3539	0.2299	0.2688	0.2044
	Theil’s U	0.3274	0.2433	0.3275	0.2168	0.2390	0.1927	0.4103	0.2963	0.1242	0.2543	0.2939	0.2266
6	MAE	0.3962	0.2958	0.3904	0.2779	0.3087	0.2109	0.3840	0.3077	0.3687	0.2612	0.3174	0.2039
	MAPE	5.9199	2.7897	5.7617	3.6062	2.9083	2.5086	3.6039	2.7721	3.4504	2.3971	2.8854	1.8130
	RMSE	0.4747	0.3794	0.4641	0.3351	0.3966	0.2540	0.4730	0.3981	0.4524	0.3241	0.4119	0.2538
	Theil’s U	0.3641	0.2900	0.3604	0.2620	0.2964	0.2053	0.4470	0.3711	0.0355	0.3144	0.3754	0.2539
7	MAE	1.1804	0.6422	1.1391	0.5400	0.6484	0.4874	1.1017	0.4383	0.9987	0.3524	0.4424	0.3244
	MAPE	4.8226	1.7827	4.6150	1.9157	1.8071	1.7029	8.8895	3.1258	7.9295	1.9074	3.1673	1.7966
	RMSE	1.5453	1.0440	1.5062	0.8363	1.0466	0.7922	1.2908	0.6534	1.1886	0.5181	0.6573	0.4877
	Theil’s U	0.6190	0.4741	0.6104	0.3752	0.4725	0.3521	0.6443	0.4114	0.1616	0.3408	0.4111	0.3179
8	MAE	0.5580	0.4187	0.5860	0.4085	0.3994	0.3900	0.5700	0.4107	0.5808	0.3584	0.3933	0.3410
	MAPE	2.7895	1.6907	2.9069	1.5727	1.5708	1.4944	2.8076	2.1977	2.8636	1.6305	2.0404	1.5288
	RMSE	0.7577	0.6109	0.7852	0.5366	0.5982	0.5199	0.7089	0.5441	0.7184	0.4723	0.5267	0.4543
	Theil’s U	0.3818	0.3101	0.3960	0.2765	0.3153	0.2670	0.4252	0.3296	0.0040	0.2999	0.3318	0.2878
9	MAE	0.5288	0.3894	0.5025	0.3590	0.3870	0.3165	0.4277	0.3280	0.3907	0.2895	0.3289	0.2553
	MAPE	2.3984	1.5865	2.2143	1.6012	1.5019	1.3863	2.4315	1.6042	2.2057	1.2508	1.5243	1.1116
	RMSE	0.6286	0.5180	0.6059	0.4351	0.5222	0.3940	0.5618	0.4317	0.5202	0.3696	0.4306	0.3308
	Theil’s U	0.2992	0.2945	0.2927	0.2374	0.3056	0.2157	0.3185	0.2989	0.2401	0.2509	0.3073	0.2252
10	MAE	0.4318	0.4053	0.4277	0.3738	0.4141	0.3309	0.4039	0.3701	0.3983	0.3344	0.3860	0.2948
	MAPE	2.7344	2.2143	2.7211	2.1001	1.9383	1.8549	5.3529	3.3984	5.2364	2.2641	2.9618	1.9840
	RMSE	0.5430	0.5280	0.5404	0.4769	0.5597	0.4331	0.5208	0.4750	0.5141	0.4247	0.4896	0.3806
	Theil’s U	0.2518	0.2687	0.2513	0.2297	0.3048	0.2089	0.2793	0.2796	0.0395	0.2421	0.3092	0.2172
11	MAE	0.7345	0.4431	0.7003	0.3932	0.4814	0.3699	0.8489	0.4216	0.7683	0.3880	0.4719	0.3656
	MAPE	6.3911	1.4151	6.0908	2.6525	1.1832	2.4683	6.7701	3.1208	6.1118	3.9927	2.5030	3.7142
	RMSE	0.8775	0.5528	0.8447	0.5063	0.6194	0.4785	0.9776	0.5514	0.8950	0.4941	0.6153	0.4678
	Theil’s U	0.3917	0.2687	0.3802	0.2304	0.3359	0.2171	0.4542	0.2841	0.0803	0.2389	0.3539	0.2249
12	MAE	0.3935	0.4126	0.3924	0.3258	0.4105	0.3067	0.4587	0.4713	0.4489	0.3743	0.4688	0.3485
	MAPE	1.3398	0.9494	1.3460	1.4451	0.9543	1.3813	1.8178	1.1468	1.7757	1.5591	1.1536	1.4870
	RMSE	0.4853	0.5320	0.4822	0.4227	0.5284	0.3946	0.5709	0.5926	0.5612	0.4778	0.5890	0.4440
	Theil’s U	0.2442	0.3281	0.2414	0.2231	0.3245	0.2049	0.2832	0.3591	0.0097	0.2502	0.3554	0.2289

Note: OLS = ordinary least square; SAR = spatial autoregressive; SEM = spatial error model; SDPD = spatial dynamic panel data; MAE = mean absolute error; MAPE = mean absolute percentage error; RMSE = root mean square error. The bolded values indicate the method with the lowest forecast error.

OPEN IN VIEWER

Table 5.

Short-run Forecast Metrics for the Second Half-day of Business Operation.

Time of Day	Metrics	One Week						One Month
		Nonspatial Models			Spatial Models			Nonspatial Models			Spatial Models
		Pooled OLS	BE	Dynamic BE	SAR + BE	SEM + BE	SDPD + BE	Pooled OLS	BE	Dynamic BE	SAR + BE	SEM + BE	SDPD + BE
13	MAE	0.4175	0.5201	0.4066	0.3754	0.4580	0.3364	0.4499	0.5619	0.4548	0.4179	0.5112	0.3750
	MAPE	2.2018	1.4347	3.0642	7.2981	1.6267	7.0403	2.9555	1.6976	3.1704	3.1651	1.9433	3.0890
	RMSE	0.5430	0.6869	0.5361	0.5152	0.6148	0.4666	0.5763	0.7188	0.5807	0.5540	0.6536	0.4983
	Theil’s U	0.2548	0.4408	0.2572	0.2801	0.3656	0.2461	0.2767	0.4675	0.0006	0.3098	0.3955	0.2694
14	MAE	0.3647	0.4538	0.3343	0.3325	0.4070	0.2864	0.4501	0.5333	0.4362	0.4011	0.5005	0.3488
	MAPE	1.4384	0.9877	1.2990	1.1739	1.0307	1.0461	3.5444	2.1385	3.1142	1.8881	2.3571	1.7215
	RMSE	0.4673	0.5645	0.4123	0.4368	0.5096	0.3777	0.5815	0.6694	0.5592	0.5286	0.6276	0.4619
	Theil’s U	0.2126	0.3393	0.1966	0.2305	0.2904	0.1950	0.2633	0.3969	0.0022	0.2812	0.3538	0.2395
15	MAE	3.0385	0.4098	2.1535	0.3086	0.3944	0.2674	3.2011	0.4262	2.1193	0.3220	0.4158	0.2785
	MAPE	19.7274	1.2125	13.9032	1.1952	1.2909	0.9989	162.7939	21.8246	115.8059	27.7675	24.1346	25.0841
	RMSE	3.0736	0.4988	2.1968	0.3927	0.4773	0.3403	3.2423	0.5347	2.1733	0.4130	0.5231	0.3566
	Theil’s U	0.6736	0.2961	0.5921	0.2102	0.2714	0.1792	0.6927	0.3274	1.0423	0.2359	0.3064	0.1995
16	MAE	2.3646	0.3990	1.6236	0.2977	0.4003	0.2665	2.4596	0.4477	1.5784	0.3234	0.4482	0.2879
	MAPE	9.4233	1.1864	6.4981	0.8764	1.1809	0.7963	9.1411	1.0631	5.9694	0.8881	1.0576	0.8106
	RMSE	2.4102	0.4860	1.6811	0.3766	0.4876	0.3353	2.5263	0.5900	1.6643	0.4191	0.5908	0.3708
	Theil’s U	0.6118	0.2751	0.5190	0.1969	0.2769	0.1726	0.6259	0.3369	0.6144	0.2266	0.3385	0.1968
17	MAE	4.4028	0.4988	3.7156	0.3166	0.1983	0.2915	4.5965	0.5278	3.6445	0.3635	0.5275	0.3351
	MAPE	17.9788	1.9164	15.0379	1.1737	1.9186	1.0988	19.9295	1.4257	15.7974	1.0977	1.4273	1.0561
	RMSE	4.4369	0.6006	3.7524	0.4109	0.6001	0.3786	4.6451	0.7275	3.6996	0.4950	0.7271	0.4571
	Theil’s U	0.7309	0.3331	0.6954	0.2010	0.3326	0.1835	0.7356	0.4034	2.2243	0.2442	0.4030	0.2223
18	MAE	2.4368	0.3419	1.9096	0.3196	0.3808	0.2809	2.4792	0.4811	1.8079	0.3428	0.4712	0.3016
	MAPE	116.4708	12.3561	86.2436	6.3920	9.5147	4.8962	32.7470	3.6195	24.5817	2.2049	2.8248	1.7866
	RMSE	2.4811	0.4881	1.9590	0.3943	0.4622	0.3473	2.5660	0.6772	1.9107	0.4378	0.6657	0.3864
	Theil’s U	0.5882	0.2139	0.5265	0.1776	0.2310	0.1567	0.6079	0.3177	0.6819	0.2085	0.3529	0.1829
19	MAE	0.6619	0.8417	0.6582	0.5177	0.8822	0.4583	0.5983	0.6473	0.5908	0.4237	0.6709	0.3767
	MAPE	0.9945	0.6594	0.9835	0.8559	0.6695	0.7192	1.1885	0.7514	1.1541	0.8846	0.7471	0.7797
	RMSE	0.8965	1.2062	0.8932	0.7302	1.2477	0.6578	0.7781	0.8899	0.7707	0.5593	0.9171	0.5004
	Theil’s U	0.2414	0.4066	0.2406	0.2032	0.4290	0.1814	0.2623	0.3861	0.0043	0.2081	0.4065	0.1844
20	MAE	0.5554	0.6206	0.5461	0.4175	0.6105	0.3574	0.6693	0.7090	0.6598	0.4691	0.7006	0.4020
	MAPE	1.0964	0.7425	1.0941	1.0094	0.7451	0.8409	1.4149	0.9686	1.4064	1.3698	0.9771	1.1823
	RMSE	0.8395	0.9770	0.8342	0.6572	0.9659	0.5812	0.9343	1.0359	0.9224	0.6451	1.0263	0.5590
	Theil’s U	0.2886	0.3990	0.2871	0.2278	0.3911	0.1974	0.3257	0.4262	0.0001	0.2256	0.4187	0.1915
21	MAE	0.5322	0.4534	0.5111	0.3318	0.4554	0.2630	0.5878	0.4497	0.5314	0.3363	0.4506	0.2741
	MAPE	1.5064	1.5226	1.4406	1.2094	1.5138	1.1325	13.8998	2.2498	12.3145	7.8093	2.2336	6.3562
	RMSE	0.6552	0.5706	0.6255	0.4283	0.5732	0.3530	0.7082	0.5587	0.6447	0.4221	0.5598	0.3485
	Theil’s U	0.2849	0.2793	0.2739	0.1892	0.2816	0.1549	0.3562	0.3166	0.0000	0.2233	0.3185	0.1824
22	MAE	0.4649	0.4296	0.4606	0.3189	0.4467	0.2651	0.4995	0.4156	0.4841	0.3056	0.4151	0.2605
	MAPE	1.2671	1.0864	1.2579	1.1154	1.0445	0.9517	1.6584	1.3973	1.6253	1.3329	1.3139	1.1393
	RMSE	0.5654	0.5047	0.5608	0.3897	0.5182	0.3279	0.6190	0.5228	0.6018	0.3771	0.5240	0.3233
	Theil’s U	0.2559	0.2372	0.2538	0.1674	0.2523	0.1416	0.3424	0.2995	0.0009	0.2038	0.3112	0.1751
23	MAE	0.2706	0.3166	0.2736	0.2643	0.3337	0.2515	0.3312	0.3414	0.3340	0.2541	0.3498	0.2394
	MAPE	0.9409	0.8802	0.9492	0.8912	0.8691	0.8523	2.6642	2.2639	2.7003	2.3653	2.1631	2.2449
	RMSE	0.3496	0.3840	0.3533	0.3151	0.4012	0.2991	0.4323	0.4370	0.4365	0.3114	0.4462	0.2929
	Theil’s U	0.2046	0.2461	0.2070	0.1743	0.2633	0.1635	0.2973	0.3265	0.0166	0.2041	0.3417	0.1895
24	MAE	0.2868	0.2924	0.2856	0.2601	0.2892	0.2357	0.3115	0.3008	0.3049	0.2380	0.2992	0.2178
	MAPE	0.8903	0.8511	0.8846	0.8834	0.8574	0.8029	1.8906	1.5448	1.8198	1.7945	1.5622	1.6668
	RMSE	0.3660	0.3654	0.3643	0.3085	0.3624	0.2804	0.3925	0.3805	0.3853	0.2905	0.3788	0.2663
	Theil’s U	0.2438	0.2634	0.2425	0.1933	0.2589	0.1747	0.3081	0.3203	0.0673	0.2146	0.3166	0.1945

Note: OLS = ordinary least square; SAR = spatial autoregressive; SEM = spatial error model; SDPD = spatial dynamic panel data; MAE = mean absolute error; MAPE = mean absolute percentage error; RMSE = root mean square error. The bolded values indicate the method with the lowest forecast error.

OPEN IN VIEWER

Table 6.

Medium-run Forecast Metrics for the First Half-day of Business Operation.

Time of Day	Metrics	Three Months						Six Months
		Nonspatial Models			Spatial Models			Nonspatial Models			Spatial Models
		Pooled OLS	BE	Dynamic BE	SAR + BE	SEM + BE	SDPD + BE	Pooled OLS	BE	Dynamic BE	SAR + BE	SEM + BE	SDPD + BE
1	MAE	0.2942	0.2742	0.2925	0.2310	0.2727	0.1883	0.3508	0.3057	0.9371	0.2839	0.3006	0.2290
	MAPE	11.0718	6.2450	13.7246	7.4864	6.9233	4.9749	8.1450	3.5878	10.5762	4.4618	3.9818	2.9943
	RMSE	0.3873	0.3701	0.3938	0.2812	0.3635	0.2305	0.4635	0.4221	1.2127	0.3559	0.4126	0.2886
	Theil’s U	0.2764	0.2739	0.3091	0.1772	0.2564	0.1476	0.2603	0.2440	0.6677	0.1716	0.2266	0.1424
2	MAE	0.2644	0.2420	0.2581	0.1820	0.2397	0.1388	0.3391	0.2778	0.8194	0.2109	0.2731	0.1564
	MAPE	1.7604	1.5649	1.7564	1.6209	1.6573	1.1788	2.7652	1.3431	8.0759	1.6326	1.4213	1.1985
	RMSE	0.3363	0.3121	0.3281	0.2296	0.3088	0.1764	0.4291	0.3653	1.0539	0.2768	0.3585	0.2033
	Theil’s U	0.2808	0.2619	0.3021	0.1737	0.2509	0.1385	0.2783	0.2355	0.6682	0.1574	0.2237	0.1212
3	MAE	0.6278	0.2089	0.4421	0.1679	0.2127	0.1365	1.0525	0.2512	0.7579	0.2107	0.2542	0.1649
	MAPE	7.9236	2.2362	5.5272	2.0841	2.3733	1.5238	12.9055	2.2401	5.9714	2.2067	2.3728	1.5648
	RMSE	0.6822	0.2738	0.5107	0.2107	0.2778	0.1721	1.1001	0.3319	0.9755	0.2806	0.3393	0.2205
	Theil’s U	0.6051	0.2674	0.5302	0.1896	0.2632	0.1592	0.6667	0.2447	0.7612	0.1861	0.2428	0.1522
4	MAE	0.5124	0.2084	0.3735	0.1702	0.2169	0.1418	0.8274	0.2489	0.6562	0.2163	0.2603	0.1765
	MAPE	0.0000	0.0000	10.7869	0.0000	0.0000	0.0000	0.0000	0.0000	19.5107	0.0000	0.0000	0.0000
	RMSE	0.5724	0.2759	0.4393	0.2134	0.2871	0.1786	0.8844	0.3331	0.8442	0.2880	0.3534	0.2358
	Theil’s U	0.5829	0.2913	0.5059	0.2130	0.2918	0.1818	0.6372	0.2675	0.7440	0.2135	0.2729	0.1810
5	MAE	0.3888	0.2151	0.3204	0.1727	0.2162	0.1532	0.5984	0.2338	0.6376	0.1871	0.2343	0.1658
	MAPE	0.0000	0.0000	3.8547	0.0000	0.0000	0.0000	0.0000	0.0000	6.6157	0.0000	0.0000	0.0000
	RMSE	0.4556	0.2833	0.3908	0.2184	0.2844	0.1961	0.6655	0.3058	0.8219	0.2408	0.3072	0.2142
	Theil’s U	0.4886	0.3162	0.4431	0.2298	0.3086	0.2093	0.5359	0.2677	0.7165	0.1928	0.2610	0.1763
6	MAE	0.4644	0.2795	0.3930	0.2261	0.2835	0.1763	0.6916	0.2820	0.7458	0.2381	0.2864	0.1813
	MAPE	0.0000	0.0000	3.4538	0.0000	0.0000	0.0000	0.0000	0.0000	98.6208	0.0000	0.0000	0.0000
	RMSE	0.5449	0.3661	0.4796	0.2864	0.3730	0.2252	0.7727	0.3676	0.9673	0.3057	0.3744	0.2334
	Theil’s U	0.4839	0.3309	0.4266	0.2500	0.3303	0.2019	0.5331	0.2710	0.7245	0.2094	0.2704	0.1658
7	MAE	1.4387	0.3670	1.0333	0.2928	0.3685	0.2658	2.2357	0.3396	1.2378	0.2806	0.3409	0.2550
	MAPE	15.7112	2.7788	6.9903	2.1462	2.8135	1.8873	28.9558	2.8248	13.2256	2.2129	2.8588	1.9728
	RMSE	1.5426	0.5282	1.1903	0.4087	0.5295	0.3788	2.2985	0.4711	1.5455	0.3771	0.4724	0.3487
	Theil’s U	0.7066	0.3600	0.5503	0.2685	0.3587	0.2475	0.7810	0.2947	0.8007	0.2207	0.2938	0.2042
8	MAE	0.6256	0.3629	0.5656	0.3062	0.3550	0.2940	0.9973	0.3734	1.1032	0.3064	0.3760	0.2960
	MAPE	0.0000	0.0000	3.7912	0.0000	0.0000	0.0000	0.0000	0.0000	8.0582	0.0000	0.0000	0.0000
	RMSE	0.7423	0.4720	0.6932	0.3989	0.4654	0.3848	1.1159	0.5283	1.4586	0.4017	0.5403	0.3884
	Theil’s U	0.4477	0.2957	0.4237	0.2431	0.3030	0.2334	0.5216	0.2732	0.7406	0.1873	0.2902	0.1798
9	MAE	0.4256	0.3304	0.3822	0.2830	0.3370	0.2494	0.5358	0.4008	1.1693	0.3073	0.4162	0.2707
	MAPE	2.7097	1.8834	2.4625	1.7420	1.7904	1.4867	2.6852	1.3981	5.7616	1.4158	1.3393	1.2235
	RMSE	0.5394	0.4416	0.4945	0.3611	0.4525	0.3221	0.6757	0.5842	1.5064	0.4141	0.6071	0.3699
	Theil’s U	0.2821	0.2830	0.2663	0.2141	0.2980	0.1913	0.2734	0.2938	0.6510	0.1839	0.3134	0.1645
10	MAE	0.4154	0.3803	0.4227	0.3502	0.3977	0.3089	0.4920	0.4404	1.3384	0.3685	0.4818	0.3257
	MAPE	3.6404	2.5848	4.0418	2.1051	2.2567	1.8538	4.4295	2.7183	6.3826	3.0641	2.3810	2.7138
	RMSE	0.5323	0.5009	0.5467	0.4386	0.5281	0.3932	0.6359	0.5973	1.7361	0.4822	0.6542	0.4325
	Theil’s U	0.2631	0.2722	0.2864	0.2236	0.3074	0.2009	0.2539	0.2598	0.6848	0.1885	0.3044	0.1696
11	MAE	1.0249	0.4576	0.7532	0.4002	0.5014	0.3774	1.8415	0.5788	1.7206	0.4495	0.6676	0.4246
	MAPE	8.2851	3.0765	5.7996	3.1602	2.4551	2.9648	14.0138	2.4425	8.9607	3.5108	1.9856	3.3114
	RMSE	1.1682	0.5947	0.8910	0.4998	0.6674	0.4727	2.0046	0.8954	2.3493	0.6066	1.0258	0.5741
	Theil’s U	0.4716	0.2811	0.4132	0.2181	0.3516	0.2052	0.5554	0.3154	0.7445	0.1844	0.3999	0.1732
12	MAE	0.5564	0.5364	0.5332	0.4211	0.5330	0.3907	0.7757	0.7267	1.7108	0.4774	0.7197	0.4448
	MAPE	2.9712	1.9062	3.6719	1.8652	1.9266	1.7660	2.8887	1.5308	4.8639	1.8603	1.5472	1.6917
	RMSE	0.7305	0.7550	0.7163	0.5425	0.7503	0.5033	1.0879	1.1280	2.3243	0.6845	1.1190	0.6404
	Theil’s U	0.3045	0.3863	0.3419	0.2348	0.3818	0.2149	0.3300	0.4167	0.6945	0.2038	0.4108	0.1870

Note: OLS = ordinary least square; SAR = spatial autoregressive; SEM = spatial error model; SDPD = spatial dynamic panel data; MAE = mean absolute error; MAPE = mean absolute percentage error; RMSE = root mean square error. The bolded values indicate the method with the lowest forecast error.

OPEN IN VIEWER

Table 7.

Medium-run Forecast Metrics for the Second Half-day of Business Operation.

Time of Day	Metrics	Three Months						Six Months
		Nonspatial Models			Spatial Models			Nonspatial Models			Spatial Models
		Pooled OLS	BE	Dynamic BE	SAR + BE	SEM + BE	SDPD + BE	Pooled OLS	BE	Dynamic BE	SAR + BE	SEM + BE	SDPD + BE
13	MAE	0.5682	0.6534	0.5792	0.4809	0.5959	0.4317	0.7534	0.9662	1.8777	0.6114	0.8673	0.5373
	MAPE	2.3936	1.3689	2.8691	1.9835	1.5525	1.9005	2.5555	1.5702	7.9996	2.7278	1.7992	2.6887
	RMSE	0.8241	0.9700	0.8863	0.6692	0.9011	0.6046	1.4465	1.8247	2.8749	1.0500	1.7023	0.9291
	Theil’s U	0.3216	0.5088	0.4172	0.2957	0.4404	0.2588	0.3606	0.5872	0.7092	0.2646	0.5143	0.2258
14	MAE	0.8017	0.7891	0.7829	0.5694	0.7621	0.5000	0.9053	1.0198	2.1333	0.6660	0.9628	0.5792
	MAPE	9.5084	5.0959	11.3126	3.4691	5.7886	3.1876	5.4945	3.0752	7.7610	2.2597	3.4471	2.0514
	RMSE	1.7611	1.8276	2.0092	1.1184	1.8012	1.0043	2.1897	2.4503	3.6321	1.4107	2.3808	1.2584
	Theil’s U	0.5063	0.6388	0.8280	0.3261	0.6063	0.2834	0.4346	0.5913	0.7139	0.2755	0.5530	0.2387
15	MAE	3.9632	0.7004	2.4048	0.5094	0.6995	0.4498	5.4708	0.8364	2.7893	0.5814	0.8192	0.5159
	MAPE	84.3965	9.5567	69.6832	10.5323	10.5007	9.4570	64.6670	5.2707	42.6497	5.8016	5.7849	5.2071
	RMSE	4.1792	1.7749	2.9439	1.0320	1.7708	0.9386	5.6193	1.7255	3.5669	1.0616	1.7040	0.9764
	Theil’s U	0.6467	0.5987	0.5330	0.3007	0.5790	0.2656	0.6510	0.4612	0.6383	0.2461	0.4385	0.2219
16	MAE	3.0465	0.7477	1.8371	0.5334	0.7473	0.4786	4.0788	0.9250	2.6003	0.6292	0.9258	0.5685
	MAPE	14.9847	2.1903	9.4591	1.8823	2.1719	1.7319	17.1598	1.9307	9.8647	1.8454	1.9139	1.7239
	RMSE	3.2385	1.4677	2.1917	1.0058	1.4680	0.9217	4.2922	1.6978	3.4861	1.1324	1.6994	1.0502
	Theil’s U	0.6011	0.5104	0.4775	0.3272	0.5123	0.2920	0.5737	0.4094	0.6445	0.2501	0.4116	0.2280
17	MAE	5.5842	1.0729	4.0141	0.7040	1.0736	0.6491	7.5760	1.1114	4.1488	0.7141	1.1127	0.6585
	MAPE	41.1912	2.9731	17.3257	2.9361	2.9609	2.5775	40.7707	2.1507	28.2786	2.1943	2.1422	1.9891
	RMSE	5.9072	2.6428	4.5587	1.6133	2.6432	1.5036	7.7852	2.2755	4.9264	1.3879	2.2768	1.2995
	Theil’s U	0.6651	0.6947	0.5342	0.3537	0.6956	0.3211	0.6882	0.5200	0.6480	0.2732	0.5212	0.2519
18	MAE	3.1116	0.8066	2.0937	0.5213	0.7704	0.4651	4.3761	0.9458	2.6715	0.5939	0.9106	0.5292
	MAPE	26.3160	3.9258	17.7557	2.3232	3.0402	1.9899	24.7646	2.8979	11.3138	1.8109	2.2518	1.5948
	RMSE	3.3707	1.7544	2.5963	1.0273	1.7019	0.9427	4.5978	1.8531	3.5530	1.0361	1.7733	0.9375
	Theil’s U	0.6111	0.5405	0.5150	0.3116	0.5816	0.2804	0.5989	0.4063	0.6419	0.2271	0.4402	0.2037
19	MAE	0.7868	0.7259	0.8320	0.4845	0.7393	0.4331	0.8833	0.7859	1.7777	0.5145	0.8053	0.4584
	MAPE	3.0208	1.6161	3.0595	1.3172	1.5500	1.2384	5.3396	2.4665	11.1029	2.7172	2.3448	2.5492
	RMSE	1.6637	1.6283	1.9433	0.9197	1.6338	0.8400	1.5343	1.4969	2.5945	0.8631	1.5152	0.7852
	Theil’s U	0.5074	0.6142	0.7109	0.2931	0.6278	0.2633	0.3862	0.4826	0.6691	0.2392	0.4997	0.2153
20	MAE	0.7742	0.7498	0.8553	0.5031	0.7454	0.4363	0.7861	0.7416	1.6883	0.4795	0.7345	0.4150
	MAPE	1.9312	1.4289	2.2278	1.9138	1.4475	1.6993	1.8830	1.3130	5.0512	1.5780	1.3300	1.4021
	RMSE	1.6309	1.6169	1.9029	0.9320	1.6140	0.8165	1.3885	1.3879	2.3466	0.7968	1.3808	0.6997
	Theil’s U	0.5115	0.5868	0.7329	0.2866	0.5822	0.2435	0.4133	0.4845	0.6891	0.2340	0.4784	0.2012
21	MAE	0.8011	0.6077	0.7090	0.4383	0.6087	0.3616	1.0981	0.6798	1.7470	0.4590	0.6825	0.3790
	MAPE	8.7183	1.8347	7.4321	3.9047	1.8194	3.0234	9.1276	1.7520	8.4901	2.7396	1.7382	2.1753
	RMSE	1.2452	1.1120	1.2351	0.7103	1.1136	0.6152	1.4656	1.1530	2.3640	0.7080	1.1568	0.6100
	Theil’s U	0.4446	0.4485	0.5378	0.2552	0.4509	0.2169	0.4354	0.3940	0.7075	0.2091	0.3968	0.1780
22	MAE	0.5886	0.4515	0.5217	0.3316	0.4530	0.2829	0.9260	0.5835	1.5090	0.4074	0.5969	0.3496
	MAPE	2.4799	1.7098	1.9835	1.4731	1.6007	1.2122	4.5315	1.7242	6.8484	2.1050	1.6182	1.7850
	RMSE	0.7792	0.6533	0.7237	0.4427	0.6613	0.3870	1.1972	0.9116	2.0389	0.5779	0.9351	0.5045
	Theil’s U	0.3700	0.3246	0.4129	0.2028	0.3401	0.1770	0.4145	0.3326	0.7079	0.1864	0.3530	0.1620
23	MAE	0.3587	0.3497	0.3474	0.2600	0.3570	0.2456	0.4248	0.4032	1.0468	0.3013	0.4146	0.2887
	MAPE	2.3131	2.0175	2.9748	1.9789	1.9239	1.8807	2.8587	2.3381	6.3016	1.8813	2.2349	1.8032
	RMSE	0.4721	0.4729	0.4530	0.3183	0.4837	0.3002	0.5606	0.5500	1.3554	0.3801	0.5650	0.3638
	Theil’s U	0.2980	0.3234	0.3246	0.1885	0.3393	0.1757	0.2851	0.3017	0.6788	0.1767	0.3173	0.1670
24	MAE	0.3485	0.3215	0.3186	0.2456	0.3205	0.2236	0.4233	0.3710	0.9523	0.2865	0.3681	0.2665
	MAPE	2.0111	1.5522	2.1507	1.7320	1.5679	1.5837	2.1175	1.4436	4.4810	1.6545	1.4695	1.5284
	RMSE	0.4623	0.4419	0.4234	0.2971	0.4403	0.2710	0.5595	0.5180	1.2371	0.3570	0.5136	0.3327
	Theil’s U	0.3383	0.3450	0.3508	0.1996	0.3418	0.1805	0.3149	0.3105	0.6834	0.1795	0.3046	0.1659

Note : OLS = ordinary least square; SAR = spatial autoregressive; SEM = spatial error model; SDPD = spatial dynamic panel data; MAE = mean absolute error; MAPE = mean absolute percentage error; RMSE = root mean square error. The bolded values indicate the method with the lowest forecast error.

Based on the forecasting error metrics in Tables 4–7, it is clear that the SDPD model provides the best forecasts across all of the different specifications, times of day, and forecast window. This can be observed by recalling that the smaller the forecast error, the better the forecasting ability of the particular estimator. However, the small forecasting errors of the SDPD model are not necessarily indicative of superior predictive ability, unless the SDPD model forecasts are statistically different from the other models’ forecasting errors. Therefore, we next turn to the panel DM to statistically test the difference in forecasts between the SDPD and the other models.

In order to accurately estimate the panel DM statistic, we restrict its calculation to one-week-ahead forecasts to reduce the chance of having serial correlation within the underlying data, which could possibly invalidate our ability to infer from the DM tests. We made no adjustments for serial correlation because we are only forecasting seven days in advance; therefore, it is reasonable to assume that the differentials are serially uncorrelated. For forecasts greater than seven days, the panel DM statistic can be modified to deal with the serial correlation by using a Newey–West type estimator. We do not pursue this extension here. According to Pesaran, Schuermann, and Smith (2009), the degree of cross-sectional dependence of the forecast errors has to be sufficiently weak for N − 1 ∑ i = 1 N y ¯ i to tend to a normal distribution as N → ∞. Unfortunately, due to the limited number of markets within our analysis, we are not able to pursue such an extension in the current article. The panel DM is a one-sided test, so the relevant 1 percent and 5 percent critical values are −2.326 and −1.634, respectively. Assuming that the SDPD forecast is defined as A, a positive value of the panel DM statistic will present evidence against it. As a sensitivity analysis, we use the panel DM tests to determine if the SDPD forecasts are statistically distinguishable from the other models, since the forecasting error metrics imply that the SDPD model provides the best predictions of spot prices. The results of the panel DM statistic are listed in Table 8.

OPEN IN VIEWER

Table 8.

Diebold Mariano Statistics for the Spatial Dynamic Panel Data Model Relative to a Select Number of Benchmarks.

Benchmark Models	Hour of Operation
	1	2	3	4	5	6
POLS	−3.5992	−6.7895	−10.5147	−9.5611	−7.1788	−7.2687
BE	−3.3010	−6.6329	−4.3821	−3.4890	−3.9453	−5.8207
Dynamic FE	−3.5981	−6.9127	−10.3682	−9.1651	−7.2061	−7.0829
SAR	−12.1878	−6.2719	−8.1613	−7.8252	−6.8448	−8.8579
SEM	−3.1949	−5.7678	−4.3443	−3.9717	−3.8274	−5.9465
	7	8	9	10	11	12
POLS	−4.6730	−3.3212	−6.5100	−2.7410	−6.0651	−2.4598
BE	−2.3273	−1.8911	−3.3374	−2.7789	−2.7573	−4.7515
Dynamic FE	−4.4836	−3.5495	−6.1381	−2.6396	−5.6415	−2.4524
SAR	−3.2299	−3.1896	−9.4166	−7.8764	−7.9643	−6.7789
SEM	−2.3681	−1.5620	−3.3288	−3.3559	−3.9059	−4.6689
	13	14	15	16	17	18
POLS	−2.2772	−2.6520	−38.5474	−29.9097	−46.4728	−27.7892
BE	−6.8338	−7.3110	−7.2292	−6.6956	−9.0054	−2.6970
Dynamic FE	−1.8910	−1.1832	−28.2233	−21.1826	−40.0744	−22.8371
SAR	−7.4740	−8.8349	−9.7165	−9.9828	−9.4874	−9.7155
SEM	−5.3059	−6.2225	−6.2337	−6.7767	−8.9836	−4.8009
	19	20	21	22	23	24
POLS	−5.0653	−4.3504	−6.6950	−7.0034	−2.0574	−3.1181
BE	−5.7000	−4.1157	−6.5182	−7.5380	−3.7166	−3.6112
Dynamic FE	−5.0088	−4.1495	−6.5958	−6.9399	−2.1771	−3.0717
SAR	−5.5374	−4.6235	−8.5791	−8.6479	−7.3534	−10.4428
SEM	−5.9266	−4.0706	−6.5101	−7.8963	−4.4377	−3.4774

Note: SAR = spatial autoregressive; SEM = spatial error model; POLS = pooled ordinary least squares model. The DM test statistic presented in the table is based on a one-sided test, where the 1 percent and 5 percent critical values are −2.326 and −1.645, respectively.

The results of the panel DM tests largely corroborate the findings in Tables 4–7. That is, the DM test results imply that the spatial dynamic panel model provides forecasts that are statistically distinguished from the benchmark forecasts at conventional significance levels. The only exception is for the one-week forecasts for the fourteenth hour of operation during the business day. During that time, the forecasts of dynamic FEs model cannot be statistically distinguished from the dynamic panel data model with FEs. These tests also seem corroborate our claim that the SDPD controls for congestion within the transmission grid, which leads to a spatial relation in the spot prices. This is revealed through the fact that congestion increases during peak demand. Table 8 reveals that the SDPD forecasts are highly statistically distinguished from the benchmarks models during peak demand (fifteenth to twenty-second hours of operation).