Hybrid Fourier Regression – Multilayer Perceptrons Neural Network for forecasting

Abstract

Forecasting methods are advantageous tools to predict the future, especially for agricultural commodities production. This study aims to compare the forecasting method between Fourier Regression, Multilayer Perceptrons Neural Networks (MPNN), and introducing a new forecasting method hybrid Fourier Regression – Multilayer Perceptrons Neural Networks Model proposed by the author. These methods are applied to forecast the production of big chili commodities since it is one of the essential vegetable commodities with a high household and industrial consumption in Indonesia. The big chili production data used is monthly from January 2010 to June 2017 (in quintal units) sourced from Statistics Indonesia. The results show hybrid Fourier Regression – Multilayer Perceptrons Neural Networks Model is more accurate to forecast big chili production than Fourier Regression and Multilayer Perceptrons (MPNN). The MAPE produced by Fourier Regression-MPNN is the lowest compared to the other methods, which is 4.45. In summary, the use of the hybrid Fourier Regression-MPNN method in forecasting big chili production can help the government to find out the potential production of big chili in the next few quarters. Furthermore, the results are useful for considering some government policies about big chili needs such as making a decision to export or import big chili commodities.

Keywords

Hybrid forecasting fourier neural network perceptron

1. Introduction

Forecasting is the art and science of predicting events using historical data and projecting them into the future using some form of mathematical model. There are two approaches to forecasting, namely the qualitative approach and the quantitative approach. Qualitative forecasting methods are used when historical data are not available. The qualitative method of forecasting is a subjective (intuitive) method. This method is based on qualitative information. This information base can predict events in the future. The accuracy of this method is highly subjective. Quantitative forecasting methods are divided into two types, causal and time series. The causal forecasting method includes factors related to the predicted variables such as Regression analysis. Time series forecasting is a quantitative method for analyzing past data that is collected regularly using appropriate techniques. The results used to be a reference for forecasting future values [6].

Previously, forecasting methods such as exponential smoothing, moving average, ARIMA, ARFIMA, and Fourier Regression (Harmonic Regression/Spectral Regression) only used one forecasting method. However, due to advances in computational technology, forecasting methods have become more complex and accurate. Implementing data mining techniques in the forecasting process and even combining them with classical methods into a hybrid forecasting method such as Ref. [1, 3, 4, 7, 8]. results in a much more accurate forecasting value than only one forecasting method (non-hybrid).

Implementation of forecasting method is good enough to predict future agricultural production, such as big chili production. Big chili is an essential vegetable commodity. The need for big chili is high every year. Here is the household consumption participation rate of big chili base on the National Socio-Economic Survey (SUSENAS).

The average household consumption participation rate of chili in 2015–2019 is 59.41%. Not only the household but there is also a high demand for industrial consumption. So the forecasting of big chili is a helpful indicator for the government to control domestic stock. The big chili production is influenced by seasonal factors such as climate.

In this paper, the authors propose a hybrid method between Fourier Regression (Harmonic Regression /Spectral Regression) and Multilayer Perceptrons Neural Network (MPNN) for forecasting big chili production data. The basic idea of this method is data containing strong seasonality (stationary-data) is assumed to be a combination of sinusoidal and noise components. Sinusoidal components are accomplished by Fourier Regression while noise (irregular) components are by Multilayer Perceptrons.

Figure 1.

Household consumption participation rate of big chili in 2015–2019.

2. Methodology

2.1 Fourier regression

The stationary ${x}_{t}$ time series can be represented as a sinusoidal model or a Fourier/Spectral/Harmonic as follows [2, 7]:

$\displaystyle{x}_{t}={A}_{0}+\sum^{q}_{j=1}{({U}_{j}\cos({\omega}_{j}t)+{V}_{j% }\sin({\omega}_{j}t)})$ (1)

where $t=1,\cdots,T,{\omega}_{j}=2\pi{f}_{j}={2\pi j}/{T},j=1,$ $\cdots,q$ . ${f}_{j}$ is called the Fourier frequency or fundamental frequency. If the number of observations is odd, then $T=2q+1$ , if the observed data are even, then $T=2q$ . However, in the model Eq. (1) a linear trend element can be added, so it becomes:

$\displaystyle{x}_{t}={A}_{0}+bt+\sum^{q}_{j=1}{({U}_{j}\cos({\omega}_{j}t)+{V}% _{j}\sin({\omega}_{j}t)})$ (2)

To estimate the parameters in Eq. (2), the least square method or by removing the trend element $(bt)$ in the data can be used, then estimating with the least square based on Eq. (1). In Eq. (1) ${x}_{t}$ is composed of q Spectral components, but to find out which Spectral component is significant, it can be investigated from the periodogram.

2.2 Multilayer perceptrons

The most common form of NN used for forecasting is the multilayer perceptrons feedforward. The multilayer perceptrons architecture is presented in Fig. 2. Figure 2 explains that the output produced in forecasting is only one value. The forecasting uses two hidden layers (hidden layers can be more than one hidden layer) consists of four nodes in the first hidden layer and three nodes in the second hidden layer. Meanwhile, the input layer in the one-step-ahead forecasting, ${\hat{x}}_{t+1}$ , is calculated using the input from the lag $k$ variable observed $({x}_{t-k})$ or other exogenous variables. The neural network input ${p}_{m}$ is $M$ , then the forecasting time series with multilayer perceptrons [5]:

$\displaystyle{\hat{x}}_{t+1}={\beta}_{0}+\sum^{H}_{h=1}{{\beta}_{h}g\left({% \gamma}_{0i}+\sum^{M}_{m=1}{{\gamma}_{hm}{p}_{m}}\right)}$ (3)

Figure 2.

Multilayer perceptrons architecture.

In Eq. (3), $w=(\beta,\gamma)$ is a weigher for each output and hidden layers where $\beta=[{\beta}_{0},\cdots,{\beta}_{0}]$ , $\gamma=[{\gamma}_{11},$ $\cdots,$ ${\gamma}_{\text{HM}}]$ . H is the number of hidden nodes in the network and $g(\cdot)$ is the activation function used to limit the output generated by neurons. Some examples of activation functions often used are binary sigmoid function, bipolar sigmoid, and hyperbolic tangent. Determination of hidden nodes in MPNN using 5-fold-cross-validation.

2.3 Proposed method: Hybrid fourier regression and multilayer perceptrons methods

The hybrid Fourier Regression – MPNN Model combines the Fourier Regression model and the multilayer perceptrons neural network applied to the time series, which assumes that the time series is composed of sinusoidal components ${S}_{t}$ and irregular components ${I}_{t}$ :

$\displaystyle{x}_{t}={S}_{t}+{I}_{t}$ (4)

The sinusoidal components are estimated by the Harmonic Regression model, while the residuals obtained from the Fourier Regression:

$\displaystyle{e}_{t}={x}_{t}-{\hat{S}}_{t}$ (5)

${\hat{S}}_{t}$ is the forecast result from the Fourier Regression, so the residuals ${e}_{t}$ is a representation of the ${I}_{t}$ component. Then the residuals obtained by the Fourier Regression Eqs (1) or (2) is analyzed by multilayer perceptrons in Eq. (3), to produce the ${I}_{t}$ forecast, namely ${\hat{I}}_{t}$ . So the ${x}_{t}$ time series forecasting is:

$\displaystyle{\hat{x}}_{t}={\hat{S}}_{t}+{\hat{I}}_{t}$ (6)

As a consequence of the method that the author proposes on the paper that consists of two stages, the first stage Fourier Regression applied for modeling the sinusoidal component. In the second stage, multilayer perceptrons is applied for modeling the residuals of the Fourier Regression. Because the Fourier Regression cannot explain the irregular component of the data, the residuals of the Fourier Regression contain information about irregularities that are sporadic and random. The hybrid model can improve forecasting accuracy as an implication of the previous explanation.

2.4 Sample spectrum

The sample spectrum is formulated:

$\displaystyle{\hat{s}}_{x}({\omega}_{j})=\frac{1}{2\pi}\sum^{T-1}_{m=-(T-1)}{{% \hat{\gamma}}_{m}{e}^{-i{\omega}_{j}m}}$ $\displaystyle{\hat{s}}_{x}({\omega}_{j})=\frac{1}{2\pi}\left({\hat{\gamma}}_{0% }+2\sum^{T-1}_{m=1}{{\hat{\gamma}}_{m}(\cos({\omega}_{j}m))}\right)$ $\displaystyle{\hat{s}}_{x}({\omega}_{j})=\frac{T}{8\pi}({\hat{U}}^{2}_{j}+{% \hat{V}}^{2}_{j})$ (7)

where: ${\omega}_{j}=2\pi{f}_{j}={2\pi j}/{T},j=1,\cdots,q$ , ${f}_{j}$ is called the Fourier frequency or fundamental frequency. If the number of observations is odd, then $T=2q+1$ , but if the observed data are even, then $T=2q$ .

$\displaystyle{\hat{\gamma}}_{m}=\begin{cases}\displaystyle\frac{1}{T}\sum^{\rm T% }_{t=m+1}{({x}_{t}-\overline{x})({x}_{t-m}-\overline{x})},\\ \phantom{{\hat{\gamma}}_{-m},∼{}}m=0,1,2,\cdots,T-1\\ {\hat{\gamma}}_{-m},∼{}∼{}m=-1,-2,-3,\cdots,-(T-1)\end{cases}$ $\displaystyle{\hat{U}}_{j}=\frac{2}{T}\sum^{\rm T}_{t=1}{{x}_{t}\cos({\omega}_% {j}t)}$ $\displaystyle{\hat{V}}_{j}=\frac{2}{T}\sum^{\rm T}_{t=1}{{x}_{t}\sin({\omega}_% {j}t)}$ $\displaystyle{A}_{j}=\sqrt{{U}^{2}_{j}+{V}^{2}_{j}},∼{}j\text{-th amplitude}$ (8)

The periodogram is one of the analytical tools for detecting periodicity in time series data. In some cases, periodicity cannot be detected but using a smoothed spectrum plot. The periodogram is formulated as follows:

$\displaystyle{\hat{s}}_{x}({\omega}_{j})=\frac{1}{2}I({\omega}_{j})$ (9)

Spectrum plot of sample $({\hat{s}}_{x}({\omega}_{j}))$ is equivalent to plot of periodogram $(I({\omega}_{j}))$ according to frequency. So, the sample spectrum plot has the same function as the periodogram.

3. Results

3.1 Checking seasonal

The hybrid Fourier Regression – MPNN method proposed by the author was applied to forecast big chili production. Big chili production data (in quintal units) is monthly data from January 2010 to June 2017 sourced from Statistics Indonesia. For comparison, the forecasting data was estimated using Fourier Regression, MPNN, and hybrid Fourier Regression – MPNN. The first 84 data points are used as in samples for modeling, then 6 data points as testing samples are used to compare the three methods which are determined based on the MAPE (Mean Absolute Percentage Error) formulated as follows:

$\displaystyle\text{MAPE}=\frac{1}{v-T}\sum^{v}_{t=T+1}\left|\frac{{F}_{t}-{A}_% {t}}{{A}_{t}}\times 100\%\right|$ (10)

where $F_{t}$ is the $t$ -th forecasting result value, and $A t$ is the $t$ -th actual value. MAPE characteristics: (1) If the MAPE $<$ 10%, then the Hybrid Fourier Regression – MPNN method performance is very good, (2) if the MAPE value is in the range of 10%–20%, then the Hybrid Fourier Regression – MPNN method forecasting performance is good, (3) if the MAPE in the range of 20%–50%, then the Hybrid Fourier Regression – MPNN method forecasting performance is adequate, and (4) if the MAPE $>$ 50%, then the Hybrid Fourier Regression – MPNN method forecasting performance is bad.

Figure 3.

Development of big chili production (quintal) in Indonesia, 2010–2016 (dash line: trend).

Figure 3 shows that there is a strong additive trend and seasonality element which indicates the magnitude of the seasonal pattern does not depend on the data value, whether the data value is increasing or decreasing. A periodogram is needed to see if the sinusoidal components have a strong effect by removing the trend element first. Here is the periodogram for big chili production (detrended) in Fig. 4. Based on the periodogram presented in Fig. 4, it shows the significant sinusoidal component in magnitude is the 7th sinusoidal component marked by point A and the 14th component marked by point B. These components are implemented in Eq. (2).

Figure 4.

Periodogram of big chili production.

Figure 5.

Forecasting out samples with Fourier Regression (black smooth line: forecasted of big chili, black dots: actual production data of big chili, gray shaded area: confident interval of forecasted).

3.2 Fourier regression

Based on Eq. (2), the Fourier Regression model estimation obtained as follows:

$\displaystyle{\hat{x}}_{t}=671725.389+3034.147t-7467.600\sin(0.898t)+10772.988% \cos(0.898t)-17138.0485\sin(0.449t)+915.418\cos(0.449t)$ (11)

The ${R}^{2}$ of Fourier Regression in Eq. (11) is 26.82 percent. It means the model can explain the variation in big chili production by 26.82 percent, while the remaining 73.18 percent is explained by other factors outside the model. Based on Fig. 5.3, Fourier Regression is not able to follow the movement pattern of big chili production data which is indicated by a low ${R}^{2}$ value. These are the forecast results for in-sample (1 ${}^{\text{st}}$ data to 84 ${}^{\text{th}}$ data) and out-sample data (85 ${}^{\text{th}}$ data to 90 ${}^{\text{th}}$ data) which are symbolized by black smooth lines presented in Fig. 5.

Forecasting big chili production using Fourier Regression produces a MAPE of 7.44. These are the results of the forecast for the out-sample as follows in Table 1.

3.3 Multilayer perceptrons (MPNN)

Based on the results of processing using in-sample data with 30,000 repetitions, four types of MPNN architectures are obtained, namely, 1 - 2 - 1, 2 - 2 - 1, 6 - 2 - 1, and 12 - 2 - 1, where each input layers are 1, 2, 6, and 12 nodes that use input lagged variables $({x}_{t-1},{x}_{t-2},{x}_{t-6},{x}_{t-12})$ , each architecture has only one hidden layer consisting of 2 nodes, and each produces one output. The MPNN forecasting result is the average of the output of each MPNN architecture. The forecasting results for the out-samples are in Table 2. Forecasting big chili production using MPNN produces a MAPE of 9.55. Visually, the out-sample forecasting is presented in Fig. 6.

Table 1
Forecasting results of big chili peppers using fourier regression

Period Actual Forecasted MAPE

Jan 2017 883,787 923,895 4.54

Feb 2017 1,065,121 910,156 14.55

Mar 2017 970,687 906,245 6.64

Apr 2017 978,902 915,352 6.49

May 2017 986,521 932,678 5.46

Jun 2017 1,020,216 949,093 6.97

Average 7.44

Period	Actual	Forecasted	MAPE
Jan 2017	883,787	923,895	4.54
Feb 2017	1,065,121	910,156	14.55
Mar 2017	970,687	906,245	6.64
Apr 2017	978,902	915,352	6.49
May 2017	986,521	932,678	5.46
Jun 2017	1,020,216	949,093	6.97
Average			7.44

Table 2

Forecasting results of big chili peppers using MPNN

Period	Actual	Forecasted	MAPE
Jan 2017	883,787	760,896	13.91
Feb 2017	1,065,121	907,725	14.78
Mar 2017	970,687	1,050,097	8.18
Apr 2017	978,902	1,034,009	5.63
May 2017	986,521	923,588	6.38
Jun 2017	1,020,216	933,950	8.46
Average			9.55

Table 3

Results of out sample forecasting of big chili production with fourier regression – MPNN

Period	Actual	Forecasted	MAPE
Jan 2017	883,787	936,485	5.96
Feb 2017	1,065,121	972,476	8.70
Mar 2017	970,687	919,096	5.31
Apr 2017	978,902	978,407	0.05
May 2017	986,521	947,387	3.97
Jun 2017	1,020,216	992,702	2.70
Average			4.45

Figure 6.

Forecasting out sample using MPNN (black line: actual productin of big chili, blue line: forecasted production of big chili, shaded area: confident interval of forecasted data).

Figure 7.

Forecasting out sample with hybrid fourier regression – MPNN [black line: forecasted of big chili production.

3.4 Proposes method: Hybrid fourier regression – MPNN

The Hybrid Fourier Regression – MPNN method uses the estimation results of Fourier Regression Eq. (11) with big chili production data input to obtain component estimation ${S}_{t}$ $({\hat{S}}_{t})$ , while the components ${I}_{t}$ are estimated using MPNN with residuals data input from Eq. (11). The results of processing with 30,000 repetitions are four types of MPNN architectures to estimate the components. The four types of MPNN architectures, namely 1 - 1 - 1, 3 - 1 - 1, 9 - 1 - 1, and 11 - 1 - 1, where each input layers are 1, 3, 9, and 11 nodes that use input lagged variables $({e}_{t-1},{e}_{t-3},{e}_{t-9},{e}_{t-11})$ , each architecture only has one hidden layer consisting of one node, and each produces one output. The MPNN forecasting result is the average of the output of each MPNN architecture. The hybrid Fourier Regression – MPNN forecasting for the out-sample uses Eq. (6) in Table 3. Forecasting big chili production using Fourier Regression – MPNN produces a MAPE of 4.45. The MAPE generated by the hybrid Fourier Regression – MPNN is minimum compared to the MAPE value generated by the Fourier Regression and MPNN. Visually, the out-sample forecasting is presented in Fig. 7.

4. Conclusion and recommendations

Based on the results of the previous discussion, forecasting using the Hybrid Fourier Regression – multilayer perceptrons (MPNN) method is more accurate than the Fourier Regression and multilayer perceptrons (MPNN). Since the MAPE produced by the hybrid Fourier Regression – multilayer perceptrons (MPNN) method is the lowest compared to the other methods.

The use of the hybrid Fourier Regression-MPNN method in forecasting big chili production can help the government to find out the potential production of big chili in the next few quarters. Forecasting results of big chili production can help in considering government policies, for example, the decision to export and import big chili commodity. If the forecasting results show the decreasing production of the big chili, the government should prepare for importing big chili. Thus, the price and stock balance of big chili stay controlled. Big chili production data has a significant seasonal effect. The limitation of using the Hybrid Fourier Regression-MPNN method is it can only be applied if the data used has a significant seasonal effect.

For further research, shocks such as the policy effects, sporadic droughts, etc., need to be investigated for their effects on modeling and forecasting results. Thus, strategies and mitigation can be carried out by modifying existing methods.

Footnotes

Acknowledgments

We like to thank Mr. Pieter Everaers from Editor in Chief SJIAOS – IAOS for his kind mentorship in finalizing this paper; Gemma Van Halderen from the Statistics Division in the United Nations Economic and Social Commission for Asia and the Pacific (ESCAP), Our institutions Badan Pusat Statistik-Statistics Indonesia for supporting.

References

Aladag

Egrioglu

and Kadilar

. Improvement in forecasting accuracy using the hybrid model of ARFIMA and feed forward neural network. American Journal of Intelligent Systems. 2012; 2(2): 12-17.

Cryer

and Chan

. Time series analysis: With application in R, second edition. USA: Spinger Science and Businiess Media, LLC; 2008.

Fajar

. Peramalan produksi cabai rawit dengan neural network; 2017. doi: 10.13140/RG.2.2.29279.15524.

Fajar

. Perbandingan kinerja peramalan pertumbuhan ekonomi indonesia antara ARMA. FFNN dan Hybrid ARMA-FFNN; 2016. doi: 10.13140/RG.2.2.34924.36483.

Kourentzes

Barrow

and Crone

. Neural network ensemble operators for time series forecasting. Expert Systems with Applications. 2014; 41: 4235-4244.

Makridakis

and MacGee . Metode dan aplikasi peramalan. Jakarta: Binarupa Aksara; 1999.

Wei

WWS

. Time series analysis: Univariate and multivariate methods. California: Pearson Education, Inc; 2006.

Zhang

. Time series forecasting using a hybrid ARIMA and neural networks model. Neurocomputing. 2003; 50: 159-175.