Comparative analysis of AI-based models for short-term photovoltaic power forecasting in energy cooperatives

Abstract

Energy communities can support the energy transition, by engaging citizens through collective energy actions and generate positive economic, social and environmental outcomes. Renewable Energy Sources (RES) are gaining increasing share in the electricity mix as the economy decarbonises, with Photovoltaic (PV) plants to becoming more efficient and affordable. By incorporating Artificial Intelligence (AI) techniques, innovative applications can be developed to provide added value to energy communities. In this context, the scope of this paper is to compare Machine Learning (ML) and Deep Learning (DL) algorithms for the prediction of short-term production in a solar plant under an energy cooperative operation. Three different cases are considered, based on the data used as inputs for forecasting purposes. Lagged inputs are used to assess the historical data needed, and the algorithms’ accuracy is tested for the next hour’s PV production forecast. The comparative analysis between the proposed algorithms demonstrates the most accurate algorithm in each case, depending on the available data. For the highest performing algorithm, its performance accuracy in further forecasting horizons (3 hours, 6 hours and 24 hours) is also tested.

Keywords

Energy forecasting machine learning deep learning short-term prediction artificial intelligence energy communities photovoltaic

ï»¿

1. Introduction

As consumer-empowerment and community-driven initiatives, energy communities can play a key role for social innovation as they reflect a fundamental shift in citizen’s behaviour and their role as a consumer [1]. Engaging citizens through collective energy actions can reinforce positive social norms and support the energy transition. Towards this direction, the Clean Energy Package of the European Commission (EC) recognises and offers an enabling legislative framework for ‘Citizen Energy Communities’ and ‘Renewable Energy Communities’ [2].

By 2030, EU will have to increase renewables to 32% share of the energy supply and in order to reach this binding target, an explicit role for citizens and communities is foreseen [3, 4]. This is an important step towards the ‘energy democracy’, as not only it acknowledged the role of democratically controlled communities in the energy transition, but it will also help European citizens to set up their own renewable energy projects and protecting them from the big players of the energy market. Successful renewable energy cooperatives generate positive economic, social and environmental outcomes while accelerating the social and psychological dimensions of the global transition towards clean energy sources [5].

One major source of renewable energy is harvesting solar power by PV solar plants. The technology is becoming more widely used and year on year PVs make up a bigger part of the energy mix in the European Union (EU). In 2018, the EU output of PV electricity reached the 127 TWh, amounting to 3.9% of the EU’s gross electricity output [6]. The coming decade continued growth is foreseen, mostly driven by increased self-consumption and more rooftop PV installations as a path towards a post-lignite era [7].

In order to maximise efficiency and optimise production for energy supply and demand [8], the use of Machine Learning (ML) and Deep Learning (DL) algorithms is being implemented in several application of Artificial Intelligence (AI). Predicting energy production values, plays an important role in PV applications, for both short-term and long-term forecasting horizons, as the prediction accuracy is the main factor in performance applications such as fault prediction and anomalies detection, load balancing and performance monitoring of the system.

The levels of applicability of AI applications on PV plants vary, depending on the specific solutions required to be applied and the available resources for data-driven solutions. AI Applications on PV plants in energy communities, from the simplest to the more advanced, require a significant amount of available infrastructure, interconnections and expertise in order to perform efficiently and leverage the added value. Utilising relevant infrastructure and sensor-based systems, are of major importance for PV plants in order to monitor energy efficiency [9], optimise supply and demand, and perform energy management in general [10]. However, the technological maturity and the available infrastructure in energy communities varies, depending on the activities of the community, its funding capabilities and its participants needs. This poses an issue when a data gathering process takes place in order to produce data-driven applications which can enhance the PV plant’s energy efficiency. Hence, the solutions and the services for energy communities, may be fragmented, in need of redesign and not easily replicable. More specifically, the availability of smart meters in consumers, the several types of sensors (e.g., temperature, humidity, etc.) installed on the PV plants’ sites, their communication interface, and their impact on the implementation of AI applications are factors to consider when developing AI applications for PV plants in energy communities.

Forecasting the future performance of a PV plant in several scales has been examined in literature, by applying several algorithms and architectures in order to achieve high accuracy.

A data-driven approach, studying two different feature sets based on the lagged power data and their descriptive statistics, and testing Neural Networks (NN), Support Vector Machines (SVR), Random Forests (RF), Long Short Term Memory (LSTM) and Convolutional Neural Networks (CNN) has been proposed to forecast PV power production at aggregated regional level (farm level) [11].

A Deep Extreme Learning Machine (DELM), combined with Enhanced Colliding Bodies Optimisation (ECBO) and Variational Mode Decomposition (VMD) to forecast the PV production of up to 4 hours, using the historical PV production data and forecasted solar irradiance (by using numerical weather prediction) has also been proposed [12].

Integration of the smart persistence prediction algorithm, irradiance and historical production data using RFs, has been proposed to predict PV power, by including a set of features that combines historical data, predictions, averages and variances for training and validation of the algorithm, which improve the accuracy of short-term forecasts [13].

For forecasting the short-term power generated by PV panels, a deep Recurrent Neural Network (RNN) has also been proposed, utilising on-site weather IOT dataset (solar irradiance, module temperature, ambient temperature, humidity and wind speed) and electrical data (DC current and voltage) [14]. Weather data (ambient temperature, atmospheric pressure, solar irradiance, wind speed and relative humidity) with historical production measurements has also been used in a supervised sequence to sequence deep learning model with attention, to forecast PV power production [15].

An ensemble model of two LSTMs with Attention Mechanism on the temperature and power time series to forecast the short-term production using the historical temperature and production data has also been proposed [16]. Discriminative deep models including autoencoders, LSTM networks, Rectified Linear Units, and CNNs for spatiotemporal pattern recognition has been examined to estimate PV energy production, given historical measurements [17]. In this approach, the PV production prediction combined with dictionary learning, is also used for behind-the-meter energy disaggregation in residential and commercial customers.

The performance of several algorithms (LSTM, MLR, SVR, XGBoost, BNN and RT) has also been examined with several weather variables and the historical PV to forecast the next hours’ [18] and the day ahead [19] PV production.

Weather data and the historical production data have also been used for advanced applications for PVs; predicting the output power of PV and determining degradations/anomalies in using a LSTM model [20], scheduling the maintenance operations of PV installations using Multiple Linear Regression (MLR) [21], anomaly detection in PV power production data based on a variational recurrent autoencoder with a variational Bi-LSTM [22], fault/status prediction based on self-organising maps (SOM) and key performance indicators [23], and a macro-level PV forecasting model methodology, to forecast short-term inverter-level PV production power, based on Feed Forward Neural Networks (FFNN), LSTM and Gated Recurrent Units (GRU) [24].

Many solutions and applications use complex algorithms and features engineering, as well as excessive data inputs to predict future performance. These approaches can lead to highly accurate results, but they could be difficult to be applied in the case of energy communities, where the data availability would pose several constraints in the level of analysis needed and could also require significant technological and operational costs to the community. The scope of this paper is to compare the performance and accuracy of different algorithms in short term forecasting of PV production, based on the data availability which an energy community can have. The methodology consists of three different cases, which are separated depending on the data availability. Data inputs are gathered from an energy community’s PV plan and meteorological stations and are consisted of the historical PV production and weather data (solar irradiance, temperature, humidity, wind speed and cloud cover). Five algorithms are proposed; a simple MLR, two ML algorithms (SVR and XGBoost) and two NNs used for DL (LSTM and CNN in a CNN-LSTM combination), and trained in an operational PV plant, forecasting the PV’s production of the next hour. For each case, the algorithms performance is analysed and compared in terms of accuracy. To this end, the ability of producing accurate results in different types of operational conditions of energy communities based on data availability is demonstrated and the most suitable ML/DL algorithms are applied. Then, the highest performing algorithm is used in order to make forecasts in different time horizons and test the accuracy.

Apart from the introduction, the paper is structured in four sections. The second section provides the methodology description and the models architecture. The results from the application of the selected ML/DL algorithms are summarised in the third section and the discussion of results are on the fourth section. Finally, the last section is presenting the conclusions and the next steps.

2. Methodology

2.1 Methodological framework

The methodological framework followed, is based on a six-step approach and starts with the data gathering process. An energy’s cooperative historical PV production data are gathered from online monitoring system. The weather data were retrieved from a website application with data from a local meteorological station [25] (temperature, humidity, cloud cover and wind speed), and Copernicus Atmosphere Data Store [26] (solar irradiance, i.e., global and diffuse radiation).

The second step follows with the separation of the three cases based on the availability of data inputs. In Case 1, only the output of the PV’s production (PAC), is considered as input to the algorithms and only the signal itself is used as input. In this case the goal is to test the accuracy of the algorithms, when there are no weather data available. Case 2 takes into account the historical PV production data with weather variables. The goal of this is to test the accuracy of the algorithms, when the most statistically significant weather features affecting the most the PV’s performance, are available. Case 3 considers using as input the historical PV production data with extended weather variable. In this case it is assumed that an energy community has a fully operational sensor-based monitoring system and can utilise a wealth of information to develop the ML/DL applications, which can measure several types of variables. The scope is to test the accuracy of the forecasting models when the weather variables can be used as input and perform advanced analytics.

The third step of the methodological framework is applying the algorithms to the specific cases. Lagged inputs as a time-series technique are used in order forecast the production in future horizons. The autocorrelation of the production signal is calculated to determine the lag that is required to make the forecasts.

Figure 1.

Methodology overview.

Five different algorithms are considered to perform the forecasts, Long Short-Term Memory (LSTM) [27], Convolutional Neural Network (CNN) [28] in a CNN-LSTM combination, Support Vector Regression (SVR) [29], Multiple Linear Regression (MLR) [30] and XGBoost [31]. For Case 3, where extended variables are considered, a clustering of weather conditions is performed, based on the solar irradiance variable in order to help achieve highest accuracy.

Step 4 continues with the results of the algorithm’s when forecasting the next hour’s PV production. Step 5 follows with the comparison of all algorithms’ performance in each case for every lagged input. This will demonstrate the highest accuracy algorithm in each scenario. Finally, for the highest performing algorithm further forecast horizons are considered to test its accuracy. The forecast horizons are set to 3, 6 and 24 hours following the same process with the lagged inputs. An iterative process is also tested to create the day ahead forecast in order to compare the two approaches. Figure 1 presents the methodological framework.

2.2 Data inputs and pre-processing

The available datasets consisted of a period spanning over 30 months (August 2018–January 2021) of production data from a specific PV plant, with an one-hour interval between observations. The PV plant has nominal capacity of 23.52 kWp and peak capacity of 20 kWn. Data quality of the production data is considered to be very high, as they are deriving from a monitoring platform through the sensor-based system, and the observations of the production are the direct output from 4 DC/AC inverters installed. Weather data are gathered from a meteorological station near the solar plant through the website Copernicus consisting of several features. More specifically, from the local meteorological station the temperature, humidity, wind speed is collected in one hour interval observations. One hour interval observations are available from Copernicus database, where the global solar radiation and the diffuse solar radiation variables are collected. The data quality of the weather information is also considered to be high, as the main driver of PV production, solar irradiance, is extracted from the Copernicus database [32, 33]. The weather features are used as inputs to the models, according to the methodology, depending on the case.

The pre-processing process starts with the autocorrelation of the production signal is calculated (Fig. 2), to assess the impact of different time lags and how they contribute to the PV output forecast. The seasonality of the signal is observed, as autocorrelation shows the degree of similarity between a given time series and a lagged version of itself over successive time intervals.

Figure 2.

Autocorrelation plot of PV production signal.

The autocorrelation of the signal is high in sequent time-steps and shows that the previous values of the production signal affect the value of the current time step. This can be explained by the fact that the intervals are of one hour and the weather conditions that affects the PVs performance do not change significantly hourly. Also, a day-by-day seasonality is observed, which can also be explained by the weather conditions and the state of the PVs, as the performance of the PV modules behave the same, given similar weather conditions during the day. Based on the autocorrelation and the time-series forecasting approach used for the three cases, the lags that are chosen are 1 hour, 5 hours and 10 hours before.

In Case 1, the production data from the energy community’s PV plant are solely used as the methodology suggests. The input consists of the production signal with time lags of 1, 5 and 10 hours.

For Case 2, weather data are used as well, along with the production data. In order to assess which of the available weather data to be used as inputs, a correlation analysis has been conducted. The most statistically relevant parameters are the solar irradiance, temperature, and humidity, which will be used as inputs from the weather datasets. Table 1 presents the Pearson and Spearman correlation coefficients of the above variables to the production of the PV, to demonstrate both the linear and non-linear correlations.

Table 1

Pearson and Spearman correlations between PV production and solar irradiance, temperature and humidity

Features	Pearson coefficient	Spearman coefficient
Production to solar irradiance	0.974	0.977
Production to temperature	0.555	0.511
Production to humidity	$-$ 0.676	$-$ 0.650

For Case 3, all weather-related variables from the website and Copernicus database are considered. In this way, a fully operational sensor-based system of an energy community is demonstrated, which has deployed extensive infrastructure, and assess the algorithms’ accuracy. This approach provides the possibility to test more advanced ML and DL models and possibly to be the base for more demanding energy related applications (e.g., maintenance actions, energy matching, load shifting, etc.). To this end, a clustering of weather conditions is also considered using the K-means algorithm. Since the solar irradiance data is in a time-series format, Dynamic Time Warping was used in order to apply the K-means algorithm to the data. The result is five clusters in which, each data point is assigned based on the value of solar irradiance at the specific timestamp. One-hot encoding of the five clusters of sky-conditions derived from DTW-K-means has been used, in order to insert the clusters in the algorithms.

Table 2

Algorithms and configuration

Algorithm	Short description	Architecture
LSTM	LSTM is an artificial recurrent neural network (RNN) architecture used in the field of deep learning. LSTM has feedback connections and can process entire sequences of data. A common LSTM unit is composed of a cell, an input gate, an output gate and a forget gate. The cell remembers values over arbitrary time intervals and the three gates regulate the flow of information into and out of the cell.	24/48/96/192/284 layersLearning rate: 0.001Adam optimiserEarly stopping patience $=$ 5Epochs: 100Validation_split: 0.2Batch_size: 128
CNN	CNN is a deep neural network which contains two basic operations, namely convolution and pooling. The convolution operation using multiple filters is able to extract features (feature map) from the data set, through which their corresponding spatial information can be preserved.	Conv1D (filters $=$ 128, kernel_size $=$ 5, strides $=$ 1, padding $=$ “causal activation” $=$ “relu”)MAxPooling1D (pool_size $=$ 2, strides $=$ 1, padding $=$ “valid”)
SVR	Support-vector machines are supervised learning models with associated learning algorithms that analyse data for classification and regression analysis.	C: 10Epsilon: 0.01
MLR	Linear regression is a linear approach to modelling the relationship between a scalar response and one or more explanatory variables. The case of one explanatory variable is called simple linear regression; for more than one, the process is called multiple linear regression.	Simple MLR
XGBoost	Gradient boosting is a machine learning technique for regression and classification problems, which produces a prediction model in the form of an ensemble of weak prediction models, typically decision trees. When a decision tree is the weak learner, the resulting algorithm is called gradient boosted trees.	N estimators: 50Max Depth: 3Learning Rate: 0.1

The datasets are split in an 80/20 way, meaning that 80% of the data are used to train the models and 20% are used as unseen data to evaluate and consequently validate the algorithms’ performance. Since the data spans over 30 months starting from month August, the testing set consists of 6 months data, covering one month of summer (August), full autumn season and two months of winter (December, January). Min/max scaling and split sequences for forecasting (lag, horizon) has been also used.

2.3 Algorithms, models configuration and forecasting horizons

Five different algorithms were tested to perform the forecasts as the methodology suggests, LSTM, CNN-LSTM combination, SVR, MLR and XGBoost. Table 2 includes a short description of each algorithm and the values of the parameters/hyperparameters used. The values of the parameters/hyperparameters were chosen in order to optimise the learning process after fine tuning and use the most efficient configuration.

The forecasting horizon is set to the next interval, i.e., the next hour’s PV production. The interval was chosen as only historical data are used (as lagged inputs) to the models and the interval of the observations is of one hour. The models’ output is the forecasted hourly PAC production in kWh.

For the highest performing algorithm, further forecast horizons are calculated following the same approach with the lagged inputs, but instead of producing only the next hour’s PV production forecast (one step ahead), it produces the 3, 6 and 24 hours respectively as a multi-step model. Following this multi-step model, an iterative process has also been created for the day-ahead PV output of the highest performing algorithm. In this iterative process each next time step forecast is used as input to the model, in order to predict the next time step, thus creating the 24-hour forecast.

This iterative process, in a real time operational situation, would require the day ahead weather forecast to be used as input. In this study, where historical data are used as training and testing data for the models and the measurements are known, the error of the weather forecast is not incorporated into the models’ accuracy.

2.4 Algorithms’ evaluation

To assess the algorithms’ forecasting performance, three statistical metrics are calculated, the $R^{2}$ value, the Root Mean Square Error (RMSE) and the Mean Absolute Error (MAE) are calculated, which are defined as followed:

(1)
$R^{2}=1-\frac{SS_{\textit{res}}}{SS_{\textit{tot}}}$ , where SSres is the sum of squared residuals from some estimation procedure, and SStot is the sum of squared deviations of the dependent variable from its sample mean.
(2)
RMSE $=\sqrt{\frac{1}{n}\mathop{\sum}\limits_{i=1}^{n}(x_{\textit{predicted}}[i]-x_{% \textit{actual}}[i])^{2}}$
(3)
MAE $=\frac{1}{n}\mathop{\sum}\limits_{i=1}^{n}|x_{\textit{predicted}}[i]-x_{% \textit{actual}}[i]|$

Table 3
Results, algorithms’ performance

lag t-1 lag t-5 lag t-10

Algorithms R2 RMSE MAE R2 RMSE MAE R2 RMSE MAE

Case 1 LSTM 85.91 1.924 1.109 94.75 1.174 0.571 94.83 1.166 0.605

CNN/LSTM 84.62 2.056 1.357 94.22 1.321 0.613 94.55 1.205 0.642

SVR 83.68 2.070 1.642 87.47 1.814 1.551 87.81 1.791 1.521

MLR 85.53 1.948 1.225 91.43 1.500 1.033 91.54 1.492 1.018

XGBoost 85.91 1.923 1.160 94.23 1.230 0.664 94.18 1.237 0.677

Case 2 LSTM 85.96 1.920 1.164 95.58 1.114 0.540 95.15 1.128 0.771

CNN/LSTM 85.43 1.981 1.182 93.4 1.197 0.793 92.44 1.181 0.789

SVR 84.55 2.014 1.479 91.41 1.601 1.244 91.77 1.471 1.185

MLR 85.69 1.938 1.214 93.74 1.282 0.825 93.81 1.275 0.835

XGBoost 86.66 1.871 1.131 95.65 1.083 0.590 95.56 1.081 0.588

Case 3 LSTM 87.05 1.814 1.112 95.77 1.05 0.55 94.53 1.19 0.56

CNN/LSTM 86.59 1.901 1.127 95.55 1.08 0.62 92.71 1.38 0.72

SVR 84.56 1.997 1.354 93.5 1.30 0.99 93.68 1.28 0.97

MLR 85.87 1.956 1.198 94.56 1.19 0.78 94.67 1.18 0.78

XGBoost 87.12 1.802 1.096 96.55 0.95 0.47 96.54 0.95 0.47

$R^{2}$ is a non-dimensional value and represents the proportion of the variation of the dependent variable from the independent variables. RMSE and MAE both have the same value as the predicting variable, in this case measured in kW. MAE estimates the average significance of the errors in a dataset of forecasts by averaging the differences between actual observations and predicted results of the entire test sample, giving all individual discrepancies equal weight. Similarly, RMSE estimates the mean value of the error utilising the square root of the average of squared differences between forecasted values and actual observations. RMSE is particularly useful, when dealing with large deviations and can detect outliers [34]. For the highest performing algorithm in the RMSE, the Mean Absolute Percentage Error (MAPE) is also calculated.

(4)
MAPE $=\frac{1}{n}\mathop{\sum}\limits_{i=1}^{n}\frac{|x_{\textit{predicted}}[i]-x_{% \textit{actual}}[i]|}{x_{\textit{actual}}[i]}x100\%$

MAPE, is a standard prediction technique that measures the accuracy of forecasting and justifies the prediction diversity for real datasets. MAPE as a percentage error, cannot use zero values because there would be a division by zero. In the case of PV modules, it can provide a useful metric as the error that it calculates, can disregard the night hours, where the PV production is zero, and focus on the production hours.
3. Results

		lag t-1	lag t-5	lag t-10
	Algorithms	R2	RMSE	MAE	R2	RMSE	MAE	R2	RMSE	MAE
Case 1	LSTM	85.91	1.924	1.109	94.75	1.174	0.571	94.83	1.166	0.605
	CNN/LSTM	84.62	2.056	1.357	94.22	1.321	0.613	94.55	1.205	0.642
	SVR	83.68	2.070	1.642	87.47	1.814	1.551	87.81	1.791	1.521
	MLR	85.53	1.948	1.225	91.43	1.500	1.033	91.54	1.492	1.018
	XGBoost	85.91	1.923	1.160	94.23	1.230	0.664	94.18	1.237	0.677
Case 2	LSTM	85.96	1.920	1.164	95.58	1.114	0.540	95.15	1.128	0.771
	CNN/LSTM	85.43	1.981	1.182	93.4	1.197	0.793	92.44	1.181	0.789
	SVR	84.55	2.014	1.479	91.41	1.601	1.244	91.77	1.471	1.185
	MLR	85.69	1.938	1.214	93.74	1.282	0.825	93.81	1.275	0.835
	XGBoost	86.66	1.871	1.131	95.65	1.083	0.590	95.56	1.081	0.588
Case 3	LSTM	87.05	1.814	1.112	95.77	1.05	0.55	94.53	1.19	0.56
	CNN/LSTM	86.59	1.901	1.127	95.55	1.08	0.62	92.71	1.38	0.72
	SVR	84.56	1.997	1.354	93.5	1.30	0.99	93.68	1.28	0.97
	MLR	85.87	1.956	1.198	94.56	1.19	0.78	94.67	1.18	0.78
	XGBoost	87.12	1.802	1.096	96.55	0.95	0.47	96.54	0.95	0.47

Table 3 presents the performance of each algorithm for the next hour’s PV production in each of the three cases.

For Case 1, where only historical production data are considered, LSTM has the highest performance. As expected, the more lags used as inputs the higher the accuracy, with $R^{2}$ value of 91.25 and 91.69, RMSE value of 1.174 kW and 1.166 kW, and MAE value of 0.571 kW and 0.605 kW, for 5-hours and 10-hours lag. The similarity in the performance between the 5-hour and the 10-hour inputs, can be explained by the fact that only the production signal was used as input and the autocorrelation value of it. As no weather information is considered in this approach, the only factor used to forecast production is the signal itself. The algorithm can estimate the future production, but more variables as inputs are needed in order to achieve high accuracy, as PV performance relies heavily on weather information. This is more evident when calculating the MAPE, where for the 5-hours lagged input is 165.60% and for the 10-hours lagged input is 150.53%. The value of MAPE is high as expected, since the production signal itself cannot be the factor of future performance.

Figure 3 presents the actual and the predicted results of LSTM algorithm for lagged input of 10 hours.

Figure 3.

Predicted and actual values (LSTM, Case 1, lag t-10, horizon t+1).

Figure 4.

Predicted and actual values (XGBoost, Case 2, lag t-5, horizon t+1).

Table 4

XGBoost horizons t+1, t+3, t+6, t+24 metrics’ evaluation

	lag t-1			lag t-5			lag t-10
	R2	RMSE	MAE	R2	RMSE	MAE	R2	RMSE	MAE
t $+$ 1	86.66	1.871	1.131	95.65	1.083	0.590	95.56	1.081	0.588
t $+$ 3	38.95	4.004	2.973	81.69	2.192	1.475	82.51	2.143	1.351
t $+$ 6	33.21	5.101	4.392	46.69	3.742	2.613	58.67	3.286	2.160
t $+$ 24	21.56	7.124	5.988	32.57	5.141	5.020	44.67	4.974	4.743

Figure 5.

Predicted and actual values (XGBoost, Case 3, lag t-10, horizon t+1).

In Case 2, where weather data are considered, all algorithms are performing significantly better. The best algorithm in each scenario is XGBoost, with an $R^{2}$ value of 86.66 and RMSE value of 1.871 for inputs with 1-hour lag. A considerably better performance for lagged inputs of 5-hours and 10-hours is observed, with values 95.65/95.56 of $R^{2}$ and 1.083/1.081 kW of RMSE respectively. LSTM is also highly accurate, achieving almost similar performance as the XGBoost with values 95.58/95.15 of $R^{2}$ and 1.114/1.128 kW of RMSE respectively. XGBoost has a MAE of 1.131/0.590/0.588 kW and LSTM of 1.164/0.540/0.771 kW for the 1-hour, 5-hours and 10-hours lags respectively. It can be noticed that the performance has been improved comparing to the Case 1, which is expected since weather data have been included as inputs. MAPE value of XGboost is 92.57% for the 1-hour lag, 71.38% for the 5-hour lags and 59.34% for the 10-hours lag, significantly better than Case 1.

In Fig. 4, the plot of the predicted and the actual values are shown for XGBoost algorithm, in the case of 5 hour lagged input and forecasting horizon of 1 hour.

For Case 3, where extended availability of weather information and several techniques were used in order to achieve better accuracy, the algorithm with the best performance is XGBoost, with a $R^{2}$ value of 96.55, RMSE 0.95 kW and MAE 0.47 kW, for 5 hours lagged input. Similar performance of the algorithm for the 10 hours lagged input is observed, with 96.54 value for $R^{2}$ , 0.95 kW for RMSE and 0.47 kW for MAE.

All algorithms performed with high accuracy in this approach, demonstrating their ability to predict the next hour’s PV output when there is high data availability. The configurations made to Case 3 for the data inputs, and the extended weather dataset, resulted in better and more accurate results. LSTM and CNN-LSTM follows in performance, performing slightly worse than XGBoost, with $R^{2}$ value of 95.77/94.53 and 95.55/92.71 for 5-hours and 10-hours lag, with a MAE of 0.55/0.56 kW and 0.62/0.72 kW respectively. The CNN layer added to the CNN-LSTM approach has not provided a significant value, as it performs slightly worse than the LSTM in all the metrics. However less training time is observed. The MAPE of the best performing algorithm (XGBoost) is also better, with a value of 30.12% for 5-hours and 29.62% for 10-hours lagged input.

Figure 5 shows the actual and the predicted values for XGBoost for lagged inputs of 10 hours.

After identifying the highest performing algorithm (XGBoost), its behaviour for a prediction horizon of 3 and 6 hours was tested. Since the same approach for the multi-step forecasting is used as the one-step, XGBoost is still the more accurate than the proposed algorithms. To this end, the highest performing algorithm from all cases and all lagged inputs scenarios, is tested for the 3 hours, 6 hours and 24 hours forecast as well. Table 4 presents the results.

Figure 6.

Predicted and actual values (XGBoost, lag t-24, horizon t+3).

Figure 7.

Predicted and actual values (XGBoost, lag t-24, horizon t+6).

Figure 8.

Predicted and actual values (XGBoost, day ahead forecast, 6 months data).

Figure 9.

Predicted and actual values (XGBoost, day ahead forecast, autumn season).

Figure 10.

Predicted and actual values (XGBoost, day ahead forecast, summer season).

As only historical inputs are considered, the accuracy is lower as further time-steps are forecasted. A 24-hours lagged input is calculated in order to test if the algorithm can achieve better performance in the multi-step forecast. The results show that the forecast of the next time steps is better with 24-hours lagged inputs, with an $R^{2}$ value of 89.65, RMSE of 1.640 kW and MAE of 1.742 kW for the 3 hours forecast horizon. As expected, the 3-hour horizon is better in the respective metrics of the 6-hour one. However, the accuracy of the model that forecast production 6 hours ahead, is also remarkably good, with an $R^{2}$ value of 87.06, RMSE value of 1.834 kW and MAE of 2.023 kW on the test set. Figures 6 and 7 presents the plots with the prediction and actual values respectively for the 3 hours horizon and the 6 hours horizon respectively.

The day ahead forecast of XGBoost is not very accurate because the PV production is depended on the weather measurements and in this approach only historical data are used. To this end, the iterative process has been created for the XGBoost, in order to test a different approach of using the highly accurate next hour’s forecasting results and predicting more long-term production.

The $R^{2}$ of the test set is 94.30, RMSE’s 1.21 kW and the MAE 0.57 kW. Furthermore, the MAPE for daytime hours is calculated for the different seasons in the dataset (winter, autumn and summer). Figure 8 shows the actual and the predicted results for the overall testing dataset, Fig. 9 for autumn and Fig. 10 for summer season.

The overall MAPE is 14.58% for winter season, 19.036% for autumn season 14.30% and for summer season 5.94%. These results demonstrates that with the iterative process the day ahead forecast (and subsequently all forecasts up to 24 hours) can be accurately forecasted, especially when the weather conditions are stable.

4. Discussion

Based on the results presented in the previous section, it is observed that high accuracy forecast of future PV performance is possible in short-term forecasting horizons.

Using as input only the historical production data are used, the best algorithm (LSTM) performs reasonably well. When weather information is considered as input, the highest performance algorithm was found to be XGBoost. The approach followed in this study, requires very few previous observations in order to make extremely accurate forecast for the next hour. In that way, this approach efficiency is demonstrated in energy community environment where the availability of data is often limited. With extended weather data available and the techniques used, all algorithms have high accuracy. XGBoost is the best algorithm in Case 3 too, but it was observed that the suggested DL algorithms are performing with similar accuracy as well. This can be explained by the fact that the DL algorithms need high amount of data in order to be efficient. If more data become available (e.g., the time intervals are shorter than an hour) then the accuracy of the DL algorithms would be improved.

For further forecasting horizons, the future PV production can also be calculated with good accuracy. However, as for the three cases, only historical data are considered, the iterative process compared to the simple multi-step forecast performs better. The iterative process that was considered to predict the day ahead PV production, performs highly accurate result in a daily basis, but in order to make the day ahead forecasting in an operational PV plant, future weather data are needed.

On a more general remark, the methodology followed, demonstrates how an energy community can use their available data in data-driven applications and produce accurate results of the future PV production up to 24 hours.

Predictions using only production historical data can be used for monitoring purposes of an energy community, as it can provide a quick estimation of the future performance without any infrastructure. More accurate forecasts, by incorporating the weather information can be used in energy trading. The hourly forecasts can be utilised, as energy communities could optimally choose when to sell the excessive energy they produce. Grid operations can also benefit from the proposed algorithms.

Predicting the energy supply from PV systems for the next 3 to 6 hours, can help operators to satisfy the electricity demand and peak shavings. Flexibility applications, such as positive and negative balancing energy provision for grid stability, can also make use of the few hours ahead forecasts. On the consumer side of the energy community, the short-term forecasts of PV production can help on increasing the self-consumption and optimising the scheduling of energy flows inside the community. Finally, maintenance actions of PVs can also be implemented using the day ahead forecast. The forecast can be used as indicator of malfunction of the PV plants of energy communities, if it is considered that the energy sources could be distributed, and they are not easily accessible to identify possible errors.

5. Conclusions

It is evident that energy communities will play a pivotal role in energy transition. In order to fully utilise their capabilities and addressing their possible limitation when deploying data-driven approaches, different considerations should be made depending on their available infrastructure. AI applications that can enhance PV production and energy efficiency, require high quality of data to produce meaningful results and provide added value. Thus, the operational status of the energy community should be examined, in order to provide services that can offer reliable results.

In this study, the energy production of PV plants is forecasted, based only on previous performance and previous weather data. Three different cases have been studied depending on the data availability. Case 1 takes into consideration only the historical production data, Case 2 incorporates the most statistically significant weather variables that affects PV performance, and Case 3 includes several weather variables. Five ML/DL algorithms are used for forecasting purposes, LSTM, CNN-LSTM combination, SVR, MLR and XGBoost. It is observed that all the algorithms perform well, in different combinations of lagged inputs for all three cases using only historical data. More specifically the next hour’s production can be forecasted with high accuracy, while the forecasted values of up to 24 hours also achieve high accuracy. The performance of the algorithms can be expected to be higher, given measurements would become available from on-site sensors, as opposed on this approach, where weather data were retrieved from meteorological stations.

It is observed that depending on the available infrastructure and the suitable algorithms, the power production of PVs can be forecasted in several short-term time horizons. Thus, PV maintenance energy matching, grid operations, demand management, scheduling supply of the energy communities can be achieved efficiently and set the base of more complex applications that require accurate short-term predictions, such as predictive maintenance or energy trading.

Next steps on expanding the results, would include the testing of the algorithms’ performance on datasets with smaller interval times and more data. Smaller intervals could make the DL algorithms perform better, as the NNs would be able to capture the connections better. Also, the use of measurements coming from on-site sensors should be considered, as it will allow more accurate weather information. A further next step would be to test the algorithms to other PV plants in different locations, as well. In that way, the replicability of the algorithms’ performance could be tested. Transfer learning techniques can also be used, based on the models proposed in this paper, when data availability is limited.

Finally, the main issue of forecasting PV production is the dependency of the weather forecast. Very near short-term forecasting can overcome this issue as the weather could be considered as stable, but for more long-term horizons the PV production will rely heavily from the weather forecast. A possible next step could be examining an interval forecast, where the forecasted power could be in range instead of an exact value.

Footnotes

Acknowledgments

The work presented is based on research conducted within the framework of the project “Modular Big Data Applications for Holistic Energy Services in Buildings (MATRYCS)”, of the European Union’s Horizon 2020 research and innovation programme under grant agreement no. 1010000158 (https://matrycs.eu/). The content of the paper is the sole responsibility of its authors and does not necessary reflect the views of the EC.

References

Caramizaru

Uihlein

. Energy communities: An overview of energy and social innovation. EUR 30083 EN, Publications Office of the European Union [Internet], Luxembourg. 2020 [cited 2021 May 14]; 59. Available from: doi: 10.2760/180576, JRC119433.

European Commission [Internet]. Brussels: Directorate-General for Energy; c1990-2000. Internal market in electricity (from 2021) Directive (EU) 2019/944 on Common Rules for the Internal Market for Electricity and Amending Directive 2012/27/EU & Directive (EU) 2018/2001 on the Promotion of the Use of Energy from Renewable Sources. 2019 [cited 2021 April 14]. Available from: https://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:32019L0944.

Soeiro

Dias

. Energy cooperatives in southern European countries: Are they relevant for sustainability targets? Energy Reports [Internet]. 2020 [cited 2021 April 14]; 6(1): 448-453. Available from: https://www-sciencedirect-com-443.web.bisu.edu.cn/science/article/pii/S2352484719305256.

Schmid

Meister

Klagge

Seidl

. Energy cooperatives and municipalities in local energy governance arrangements in Switzerland and Germany. The Journal of Environment & Development [Internet]. 2020 [cited 2021 April 14]; 29(1): 123-146. Available from: doi: 10.1177/1070496519886013.

Tarhan

. Renewable energy cooperatives: A review of demonstrated impacts and limitations. Journal of Entrepreneurial and Organizational Diversity [Internet]. 2015 [cited 2021 April 14]; 4(1): 104-1200. Available from: https://ssrn.com/abstract=2605796.

European Commission [Internet]. Brussels:European Commission; c1990-2000. Solar; 2020 April 28 [updated 2021 July 8; cited 2021 April 14]. Available from: https://ec.europa.eu/energy/topics/renewable-energy/solar-power_en.

Marinakis

Flamos

Stamtsis

Georgizas

Maniatis

Doukas

. The efforts towards and challenges of greece’s post-lignite era: The case of megalopolis. Sustainability [Internet]. 2020 [cited 2021 April 14]; 12(24): 10575. Available from: doi: 10.3390/su122410575.

Marinakis

Doukas

Tsapelas

Mouzakitis

Sicilia

Madrazo

Sgouridis

. From big data to smart energy services: An application for intelligent energy management. Future Generation Computer Systems [Internet]. 2020 [cited May 5]; 110: 572-586. Available from: https://www-sciencedirect-com-443.web.bisu.edu.cn/science/article/pii/S0167739X17318769.

Marinakis

Doukas

. An advanced IoT-based system for intelligent energy management in buildings. Sensors [Internet]. MDPI AG; 2018 [cited 2021 May 5]; 18(2): 610. Available from: doi: 10.3390/s18020610.

10.

Marinakis

Doukas

Karakosta

Psarras

. An integrated system for buildings’ energy-efficient automation: Application in the tertiary sector. Applied Energy [Internet]. 2013 [cited May 5 2021]; 101: 6-14. Available from: https://www-sciencedirect-com-443.web.bisu.edu.cn/science/article/pii/S0306261912004059.

11.

Rana

Rahman

Jin

. A Data-driven Approach for Forecasting State Level Aggregated Solar Photovoltaic Power Production. 2020 International Joint Conference on Neural Networks [Internet]. 2020 [cited May 5 2021]; 1-8. Available from: https://ieeexplore.ieee.org/document/9207594 doi: 10.1109/IJCNN48605.2020.9207594.

12.

Zhang

Jiao

Wang

. A multi-step ahead photovoltaic power prediction model based on similar day, enhanced colliding bodies optimization, variational mode decomposition, and deep extreme learning machine. Energy. 2021 Jun 1 [cited 2021 August 25]; 224: 120094. Available from: https://www-sciencedirect-com-443.web.bisu.edu.cn/science/article/abs/pii/S0360544221003431 doi: 10.1016/j.energy.2021.120094.

13.

Huertas

Centeno

. Using smart persistence and random forests to predict photovoltaic energy production. Energies. 2018 Dec 29; 12(1): 100. Available from: doi: 10.3390/en12010100.

14.

Ahn

Park

. Deep RNN-based photovoltaic power short-term forecast using power IoT sensors. Energies [Internet]. MDPI AG; 2021 [cited 2021 May 7]; 14(2): 436. Available from: doi: 10.3390/en14020436.

15.

Kharlova

May

Musilek

. Forecasting Photovoltaic Power Production using a Deep Learning Sequence to Sequence Model with Attention. In: Proceedings of the International Joint Conference on Neural Networks. Institute of Electrical and Electronics Engineers Inc.; 2020 [cited 2021 May 14]. Available from: doi: 10.1109/IJCNN48605.2020.9207573.

16.

Zhou

Zhang

Yang

Liu

Yan

. Short-Term photovoltaic power forecasting based on long short-term memory neural network and attention mechanism. IEEE Access [Internet]. 2019 [cited 2021 Aug 25]; 7: 78063-74. Available from: doi: 10.1109/ACCESS.2019.2923006.

17.

Khodayar

Jalali

SMJ

. Deep learning for pattern recognition of photovoltaic energy generation. Electr J. 2021 [cited 2021 May 15]; 34(1): 106882. Available from: https://www-sciencedirect-com-443.web.bisu.edu.cn/science/article/abs/pii/S1040619020301743 doi: 10.1016/j.tej.2020.106882.

18.

Dimitropoulos

Sofias

Kapsalis

Mylona

Marinakis

Primo

Doukas

. Forecasting of short-term PV production in energy communities through Machine Learning and Deep Learning algorithms. IEEE 12th International Conference on Information, Intelligence, Systems and Applications (IISA 2021) [Internet]; 2021 [cited 2021 Aug 25]. In press.

19.

Theocharides

Theristis

Makrides

Kynigos

Spanias

Georghiou

. Comparative analysis of machine learning models for day-ahead photovoltaic power production forecasting. Energies [Internet]. MDPI AG; 2021 [cited 2021 Aug 25]; 14(4): 1081. Available from: doi: 10.3390/en14041081.

20.

Vicente-Gabriel

Gil-González

A-B

Luis-Reboredo

Chamoso

Corchado

. LSTM networks for overcoming the challenges associated with photovoltaic module maintenance in smart cities. Electronics [Internet]. MDPI AG; 2021 Jan 4; 10(1): 78. Available from: doi: 10.3390/electronics10010078.

21.

Marinakis

. Big data for energy management and energy-efficient buildings. Energies [Internet]. MDPI AG; 2020 [cited 2021 April 16]; 13(7): 1555. Available from: doi: 10.3390/en13071555.

22.

Pereira

Silveira

. Unsupervised Anomaly Detection in Energy Time Series Data Using Variational Recurrent Autoencoders with Attention. In: Proceedings – 17th IEEE International Conference on Machine Learning and Applications, ICMLA 2018. Institute of Electrical and Electronics Engineers Inc.; 2019 [cited 2021 May 14]; pp. 1275-82. Available from: doi: 10.1109/ICMLA.2018.00207.

23.

Betti

Tucci

Crisostomi

Piazzi

Barmada

Thomopulos

. Fault prediction and early-detection in large PV power plants based on self-organizing maps. Sensors [Internet]. MDPI AG; 2021 [cited 2021 May 14]; 21(5): 1687. Available from: doi: 10.3390/s21051687.

24.

Plessis

Strauss

Rix

. Short-term solar power forecasting: Investigating the ability of deep learning models to capture low-level utility-scale Photovoltaic system behaviour. Appl Energy. 2021 [cited 2021 May 14]; 285: 116395. Available from: doi: 10.1016/J.APENERGY.2020.116395.

25.

TWC Product and Technology LLC. Local Weather Forecast, News and Conditions | Weather Underground [Internet]. 2021 [accessed 2021 Apr 14]. Available from: https://www.wunderground.com/.

26.

COPERNICUS, COPERNICUS Atmosphere Data Store [Internet]. [Accessed 2021 Jun 5]. Available from: https://ads.atmosphere.copernicus.eu/cdsapp#!/home.

27.

Hochreiter

Schmidhuber

. Long short-term memory. Neural Computation [Internet]. 1997 [cited 2021 August 26]; 9(8): 1735-1780. Available from: https://direct.mit.edu/neco/article/9/8/1735/6109/Long-Short-Term-Memory doi: 10.1162/neco.1997.9.8.1735.

28.

LeCun

Bengio

Hinton

. Deep learning. Nature [Internet]. 2015 [cited 2021 August 26]; 521: 436-444. Available from: https://www.nature.com/articles/nature14539 doi: 10.1038/nature14539.

29.

Awad

Khanna

. Support Vector Regression. Efficient Learning Machines [Internet]. Berkeley, CA; Apress. 2015 [cited 2021 August 26]; 67-80. Available from: doi: 10.1007/978-1-4302-5990-9_4.

30.

Jobson

. Multiple Linear Regression. Applied Multivariate Data Analysis. [Internet]. New York, NY: Springer, 1991 [cited 2021 Aug 26]; 219-238. Available from: doi: 10.1007/978-1-4612-0955-3_4.

31.

Chen

Guestrin

. XGBoost: A Scalable Tree Boosting System. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining [Internet]. New York, NY, USA: ACM; 2016 [cited 2021 Aug 26]; pp. 785-94. Available from: doi: 10.1145/2939672.2939785.

32.

CAMS. CAMS solar radiation time-series [Internet]. CAMS solar radiation time-series. 2021 [cited 2021 Jun 5]. Available from: https://ads.atmosphere.copernicus.eu/cdsapp#!/dataset/cams-solar-radiation-timeseries?tab=overview.

33.

COPERNICUS. COPERNICUS Regular Validation Report [Internet]. Issue 32, S-O-N. 2021 [cited 2021 Jun 5]. Available from: https://atmosphere.copernicus.eu/sites/default/files/custom-uploads/EQC-solar/CAMS72_2018SC2_D72.1.3.1-2021Q2_RAD_validation_report_SON2020_v1.pdf.

34.

Ahmed

Sreeram

Mishra

Arif

. A review and evaluation of the state-of-the-art in PV solar power forecasting: Techniques and optimization [Internet]. Vol. 124, Renewable and Sustainable Energy Reviews. Elsevier; 2020 [cited 2021 Aug 16]; 109792. Available from: https://research-repository.uwa.edu.au/en/publications/a-review-and-evaluation-of-the-state-of-the-art-in-pv-solar-power.