Autoencoder-based deep belief regression network for air particulate matter concentration forecasting

Abstract

Particulate matter (PM) is one of the most significant air pollutants in recent decades that has tremendous negative effects on the ambient air quality and the public health. Accurate PM forecasting provides a possibility for establishing an early warning system. In this paper, a deep feature learning architecture, i.e., autoencoder-based deep belief regression network (AE-based DBRN), is introduced and utilized to forecast the daily PM concentrations (PM_2.5 and PM₁₀). Prior to establishing this model, Pearson correlation analysis is applied to look for the possible input-output mapping, where the input candidate variables contain seven meteorological parameters and PM concentrations within one-day ahead, and the output variables are the local PM forecasts. The addressed model was evaluated by the dataset in the period of 28/10/2013 to 31/8/2016 in Chongqing municipality of China. Moreover, two shallow models, feed forward neural network and least squares support vector regression, were employed for the comparison. The results indicate that the AE-based DBRN model has remarkable better performances among the comparison models in terms of mean absolute percentage error (PM_2.5 21.092%, PM₁₀ 19.474%), root mean square error (PM_2.5 8.600μg/m³, PM₁₀ 11.239μg/m³) and correlation coefficient criteria (PM_2.5 0.840, PM₁₀ 0.826).

Keywords

Deep belief regression network autoencoder particulate matter meteorological data forecasting

1 Introduction

Particulate matter (PM), one of the major air pollutants, has been of a particular concern due to its negative effects on the ambient air quality and the public health. Moreover, the literature [1] points that the harmful degree for the human health usually depends on the duration of exposure as well as the category and concentration of the PM [2, 3]. To protect public health and the environment, various organizations and agencies established a large number of monitoring stations and different emission standards for regulating permissible concentrations of the PM in the atmosphere. However, the monitoring data are only the representations of the current air quality without reflecting the air quality changes in the future. Thus, it is necessary to implement an accurate forecasting model for the PM concentration that can provide early warning for guiding the works of air pollution control and public health protection.

According to the modeling theory, the PM concentration forecasting methods can be divided into two categories: physics-based models and data-driven models [4]. The formers [5, 6] aim to build and simulate the relationships between the PM concentration and the atmospheric physical and chemical components in the emission, transport and transformation processes. Nonetheless, the successful applications of these models based on the knowledge-driven require a strong background in atmospheric science. In addition, there are still many uncertainties in the evolution mechanism and the simulating model for the PM concentrations. Therefore, compared to the physics-based models, the data-driven model, which is based on the information of the dataset without considering the physical and chemical processes, has been attracted much more attention. Many approaches are developed and improved to forecast the PM concentrations [7 –13].

Among these data-driven models, artificial neural network (ANN), which has good abilities of nonlinear mapping, self-adaption, and robustness, has been regarded as an outstanding model and successfully applied in the PM concentration forecasting with various structures, e.g., feed forward neural network [14], radial basis function neural network [15], and wavelet neural network [16]. These studies indicated that the forecasting performances of the ANN closely depend upon the extracted features from the input variables (meteorology and pollutants). In other words, the excellent feature representation can ensure forecasting accuracy. However, affected by the complicated meteorology condition in practice, the PM concentrations exhibit nonlinear and time-varying indetermination. It will lead to the data-based feature learning difficulties and the calculation complexities of the network structures [17]. Therefore, it is imperative to develop a new method for feature learning in the ANN for the PM concentrations forecasting.

Deep learning [18], including many hidden layers, has a strengthening capability for the complex features’ representation from the inputs, achieving a better generalization. Through representations by the hierarchical levels from the lower layers to the higher layers, the “deeper” feature hidden in the input variables of the PM can be extracted sufficiently, and transferred into the regression networks for the PM concentration forecast. In this regard, Hinton et al. [18] proposed a deep learning architecture (stack network), i.e., deep belief network (DBN), which has been applied successfully in feature extraction [19], classification [20], and regression [21, 22]. Recently, the deep learning technique has been also caught attention in the air pollutants forecasting [23, 24]. All of these studies have shown that the deep learning is a promising technique for better learning sophisticated features compared with traditional “shallow” learning approach (one or none hidden layer in networks).

Owing to the superiority in feature learning and representation, a deep regression network of AE-based DBRN, combining autoencoder-based DBN for the deep feature learning and the NN for the feature regression, is proposed for the PM concentration (PM_2.5 and PM₁₀) forecasting in this paper. In addition, the meteorological conditions are also considered into the deep regression modeling. To estimate the forecasting performance, a comparison study is designed including two “shallow” learning approaches, i.e., feed forward neural network (FFNN) and least square support regression (LSSVR).

The rest of this paper is outlined as follows. Methodologies are introduced in Section 2. A case study is presented in Section 3. Section 4 illustrates the results and discussion. Conclusions are given in Section 5.

2 Methodologies

2.1 AE for feature extraction

An autoencoder aims to reconstruct its input vectors h^l - 1 (hidden layer l = 1, h⁰ = x, x denotes the original data) at the outputs (reconstruction results r shown in Fig. 1) through unsupervised learning [25]. As shown in Fig. 1, it attempts to build a network $r = f (h^{l - 1} | ξ) \approx h^{l - 1},$ (1) where ξ = (w, b) means a parameter set, and w and b represent the weights and the bias values between neighboring layers, respectively.

Fig.1

An AE network structure.

For an AE, there exist two processes, one is feature extraction of encoder f_e(.) (Equation (2)), and the other is feature reconstruction of decoder f_d(.) (Equation (3)). $h^{l} = f_{e} (h^{l - 1}) = Sigm ({wh}^{l - 1} + b),$ (2) $r = f_{d} (h^{l}) = Sigm (w^{T} h^{l} + b^{T}),$ (3)

After achieving the reconstruction values r, the reconstruction errors can be computed as

$\begin{matrix} L (h^{l - 1}, r) \\ = - \sum_{m = 1}^{M} [h_{m}^{l - 1} log (r_{m}) + (1 - h_{m}^{l - 1}) log (1 - r_{m})] . \end{matrix}$ (4)

To obtain optimal parameter sets of ξ = (w, b) of the AE, a loss function (J (ξ) = ∑ L (h^l-1, r)) should be minimization.

2.2 AE– based DBN for feature deep learning

The DBN shown in Fig. 2, a classical and widely-used deep structure, is a stack of simple (AE in this paper) and unsupervised networks [26].

Fig.2

AE-based DBRN model structures.

According to the graphical representation (Fig. 2), it can find that the AE-based DBN for the feature learning is modeled via one AE shown in Fig. 1 extracted and transferred the specific information learned by “layer-wise” in l hidden layers. Specifically, each layer of the AE is trained individually and only the encoder process is kept, and several AEs are stacked together with an output of one AE process to the input of next layer. This is an unsupervised learning process. In addition, in the top layer (h ^l ), supervised learning (e.g., back propagation algorithm) is applied to fine-tuning the weights from top to lower layers on the entire AE-based DBN [27].

2.3 AE-based DBRN for feature forecasting

An AE-based DBRN modeling can be concluded in Fig. 2. Its procedures are given as follows: (1) Perform generative unsupervised learning layer-wise on the AE from lower to higher layers. (2) Fine-tuning using back propagation algorithm (supervised learning) on the entire DBN to tweak the weights from top to lower layers. (3) Active the regression model using the results of the top layer w ^l of the AE-based DBN. In this paper, the FFNN, a typical and common-used forecasting model, is selected as the regression model. The w ^l is utilized to initialize the weights w of the FFNN. It is noted that the FFNN can be replaced with the other regression models.

2.4 Overview of the AE-based DBRN model

Given the composed techniques, the AE-based DBRN modeling procedure, which is summarized in Fig. 3, can be implemented as follows:

Fig.3

The framework of AE-based DBRN for forecasting the PM concentrations.

Step 1. Collect the historical data containing the PM_2.5 and PM₁₀ concentrations and the meteorological data in the corresponding periods.

Step 2. Normalize all the historical data into [0.1, 0.9] respectively.

Step 3. Divided the historical data into two categories, training set (80%) and testing set (20%) respectively.

Step 4. Train the AE-based DBRN model using the training set by the experiments, including various model designs with different input nodes, the hidden layers, and the hidden nodes.

Step 5. Forecast the next time concentrations of the PM with the trained AE-based DBRN.

Step 6. Output the forecasting result. End.

3 Case study

3.1 Data set

In this paper, two kinds of PM are investigated, i.e., PM_2.5 and PM₁₀ (μg/m³). The daily data (24-h mean value) during 28/10/2013 to 31/08/2016 in Chongqing city were collected from the datacenter of the Ministry of Environmental Protection of China. Moreover, seven types of the daily meteorological data, i.e. maximum temperature (°C), minimum temperature (C), mean atmospheric pressure (kpa), mean relative humidity (%), precipitation (mm), mean wind speed (km/h), and visibility (km), were collected in the corresponding periods (http://www.tianqihoubao.com). The detail data (total 1040×9 samples) are shown in Fig. 4.

Fig.4

Air particulate matter concentrations and meteorological conditions during the period from 28/10/2013 to 31/8/2016.

According to “Ambient Air Quality Standards (GB 3095-2012)” published by the Ministry of Environmental Protection and the General Administration of Quality Supervision, Inspection and Quarantine of China, the daily legal concentration limits of the PM are set as PM_2.5 ≤ 75μg/m³ and PM₁₀ ≤ 150μg/m³, respectively, which are referred as the horizontal dashed line shown in Fig. 4. Therefore, compared to the standard (GB 3095-2012), it can find that the PM concentrations were often above the permissible limits, and the number of days (PM_2.5 218 days and PM₁₀ 110 days) with exceeding values (maximum over 1.81 times for PM_2.5 and 0.95 times for PM₁₀) is particularly striking. Considering the meteorological conditions shown in Fig. 4, it can find that the atmosphere situations corresponding to the PM concentration levels out of the standard are lower temperature, higher atmospheric pressure, and less precipitation. There have little significant relationships between the PM concentrations and the rest meteorological data from the qualitative aspect.

3.2 Model design

As mentioned in Section 2, the proposed AE-based DBRN model has three key structure variables, i.e., the number of the input nodes, the number of the hidden layers, and the number of the hidden nodes.

As analyzed above (Fig. 4), the PM and the meteorological conditions are associated with each other in a complex relationship, making the modeling of the PM as a more complex process. To reveal their relationships, multiple inputs and one output structure are developed in this study, i.e., the meteorological parameters and the PM concentrations of one-day in advance as the input variables, and the PM forecasts of the next day as the outputs. Therefore, to improve the fitting degree and reduce the computational burden, Pearson correlation analysis method [28] is utilized for analyzing their relationships between the inputs and outputs, thereby determining the number of the input nodes.

For the hidden layers and hidden nodes in the DBN, there has no a theoretical method to determine their deep levels. Therefore, an experimental pattern (trial and error) is applied to define these variables. The experimental designing is shown as follows: four levels of the hidden layers from 1 to 4 are considered, and ten levels of the hidden nodes in each hidden layer from 5 to 50 (interval 5) are investigated.

In addition, the other computation parameters of the proposed model are set as follows: learning rate 1, size of batch training 100, and maximum iteration of training 100. The computation software is Matlab 2014 with the computation environment Intel Core i5-2450M CPU @2.50 GHz, and Memory 4.00GB. In the following study, the original time series of the PM shown in Fig. 4 will be divided into the training set (800×9 samples, 28/10/2013–04/01/2016) and the testing set (240×9 samples, 05/01/2016–31/08/2016) for modeling and validating the AE-based DBRN performance.

3.3 Performance criteria

4 Results and discussion

4.1 Pearson correlation analysis for input nodes specification

The relationships between the current PM and the meteorological factors in one-day in advance are quantified by the Pearson correlation coefficient as shown in Table 2. Label * in Table 2 represents 0.01 level of significance. From Table 2, it can find that the variables of T_max and T_min have the highest Pearson correlation coefficient (0.938), and AP and T_min have the secondary higher mutual correlation (– 0.846). Moreover, the variables T_min and PMC have a higher correlation than that of T_max and MPC, as well as AP and MPC respectively. Therefore, to reduce the modeling burden of the multiple variables with higher correlations, the variable T_max and AP can be deleted. In addition, the dataset that contains only variables strongly correlated with a coefficient greater than 0.1 can be considered as the input nodes as well. Thus four meteorological variables (T_min, P, WS and V) and MPC in one-day ahead are chosen as the input nodes for the PM_2.5 modeling. Five variables (T_min, RH, P, WS and V) and MPC in one-day ahead are selected as the inputs for the PM₁₀ modeling.

Table 1
Description of the evaluation criteria

Performance criteria Equation

Mean absolute percentage error – MAPE (%) $MAPE = \frac{100}{N} \sum_{t = 1}^{N} | \frac{C_{o} (t) - C_{p} (t)}{C_{o} (t)} |$

Root mean square error – RMSE (μg/m³) $RMSE = \sqrt{\sum_{t = 1}^{N} (C_{o} (t) - C_{p} (t))^{2} / - N}$

Correlation coefficient – CC $CC = \frac{\sum_{t = 1}^{N} (C_{o} (t) - {\bar{C}}_{o}) (C_{p} (t) - {\bar{C}}_{p})}{\sqrt{\sum_{t = 1}^{N} {(C_{o} (t) - {\bar{C}}_{o})}^{2} \sum_{t = 1}^{N} {(C_{p} (t) - {\bar{C}}_{p})}^{2}}}$

Performance criteria	Equation
Mean absolute percentage error – MAPE (%)	$MAPE = \frac{100}{N} \sum_{t = 1}^{N} \| \frac{C_{o} (t) - C_{p} (t)}{C_{o} (t)} \|$
Root mean square error – RMSE (μg/m³)	$RMSE = \sqrt{\sum_{t = 1}^{N} (C_{o} (t) - C_{p} (t))^{2} / - N}$
Correlation coefficient – CC	$CC = \frac{\sum_{t = 1}^{N} (C_{o} (t) - {\bar{C}}_{o}) (C_{p} (t) - {\bar{C}}_{p})}{\sqrt{\sum_{t = 1}^{N} {(C_{o} (t) - {\bar{C}}_{o})}^{2} \sum_{t = 1}^{N} {(C_{p} (t) - {\bar{C}}_{p})}^{2}}}$

Note: N denotes the length of the investigation periods, C_o and C_p represent the values of the recorded and the forecasted values respectively, and ${\bar{C}}_{o}$ and ${\bar{C}}_{p}$ are the mean values of the recorded and the forecasted values respectively.

Table 2

Results of the Pearson correlation analysis

Factors	T_max	T_min	AP	RH	P	WS	V	PMC
T_max	1	0.938^*	– 0.835^*	– 0.480^*	0.146^*	0.123^*	0.558^*	– 0.464^/– 0.367^
T_min		1	– 0.846^*	– 0.303^*	0.210^*	0.122^*	0.517^*	– 0.546^/– 0.470^
AP			1	0.315^*	– 0.210^*	– 0.202^*	– 0.451^*	0.467^/0.405^
RH				1	0.250^*	– 0.177^*	– 0.625^*	– 0.009/– 0.108^*
P					1	0.138^*	– 0.041	– 0.241^/– 0.263^
WS						1	0.439^*	– 0.278^/– 0.278^
V							1	– 0.487^/– 0.418^
PMC								1

Note: T_max denotes maximum temperature (C), T_min denotes minimum temperature (C), AP denotes mean atmospheric pressure (kpa), RH denotes mean relative humidity (%), P denotes precipitation (mm), WS denotes mean wind speed (km/h), V denotes visibility (km), and PMC denotes PM_2.5 and PM₁₀ concentrations (μg/m³) respectively. The Pearson correlation coefficients between meteorological variables and the PMC are separated by the slash (/).

4.2 AE-based DBRN model for PM forecasting

According to the analysis for Table 2, five inputs are set for the PM_2.5 forecasting, and six inputs for the PM₁₀. The rest parameters, the hidden layers and hidden nodes, are then determined by the experiments. In this study, taking PM_2.5 as a representation to introduce the experimental process as described in Section 3. Note that all the experimental results are the mean values under the maximum iteration times.

In the first place, an AE-based DBRN is initialized using a structure with five input nodes, one hidden layer with ten level hidden nodes, and one output node. Table 3 shows the learning results of the AE-based DBRN with one hidden layer for the testing dataset. The results reveal that the best performance appears at 45 hidden nodes (MAPE = 21.771%, RMSE = 9.023μg/m³, and CC = 0.825).

Table 3
Results of an AE-based DBRN by one hidden layer with different hidden nodes for PM_2.5 forecasting

Hidden nodes MAPE RMSE CC Hidden nodes MAPE RMSE CC

5 27.341 9.534 0.809 30 25.207 8.999 0.824

10 25.607 9.053 0.819 35 26.318 9.047 0.825

15 27.593 9.037 0.821 40 25.703 9.084 0.822

20 26.794 9.149 0.817 45 21.771 9.023 0.825

25 26.589 9.151 0.819 50 25.281 9.127 0.822

Hidden nodes	MAPE	RMSE	CC	Hidden nodes	MAPE	RMSE	CC
5	27.341	9.534	0.809	30	25.207	8.999	0.824
10	25.607	9.053	0.819	35	26.318	9.047	0.825
15	27.593	9.037	0.821	40	25.703	9.084	0.822
20	26.794	9.149	0.817	45	21.771	9.023	0.825
25	26.589	9.151	0.819	50	25.281	9.127	0.822

After confirming the number of hidden nodes in the first hidden layer (45 hidden nodes), the second AE-based DBRN is initialized as five input nodes, two hidden layers with ten level hidden nodes in the second hidden layer, and one output node. The forecasting performances of the testing dataset are shown in Table 4. The result indicates that the best evaluation group, i.e., MAPE (21.329%), RMSE (8.460μg/m³) and CC (0.840), also appears at 45 hidden nodes. In addition, the performances with two hidden layers are better than that with a single hidden layer, which show MAPE 0.448%, RMSE 0.563μg/m³, and CC 0.015 increases respectively. Therefore, the stack networks can be regarded as an effective framework to enhance the feature representation.

Table 4

Results of an AE-based DBRN by two hidden layers with different hidden nodes for PM_2.5 forecasting

Hidden nodes	MAPE	RMSE	CC	Hidden nodes	MAPE	RMSE	CC
5	22.799	8.634	0.837	30	22.156	8.773	0.840
10	23.377	8.590	0.836	35	23.324	8.470	0.840
15	25.004	8.460	0.836	40	22.442	8.681	0.839
20	23.429	8.484	0.842	45	21.329	8.460	0.840
25	24.387	8.498	0.839	50	22.266	8.727	0.839

The selecting experiment for the hidden layer is then continued by the same way. As shown in Table 5, the experimental results demonstrate that the structure with 45 hidden nodes in the third hidden layer has the best regression performances. However, the improvements are limited, that is, the MAPE has 0.237% increases and the RMSE has 0.015μg/m³ decreases.

Table 5

Results of an AE-based DBRN by three hidden layers with different hidden nodes for PM_2.5 forecasting

Hidden nodes	MAPE	RMSE	CC	Hidden nodes	MAPE	RMSE	CC
5	22.848	8.477	0.842	30	24.304	8.600	0.842
10	24.819	8.501	0.843	35	25.805	8.552	0.838
15	24.662	8.375	0.841	40	26.751	8.633	0.839
20	24.589	8.462	0.841	45	21.092	8.475	0.840
25	25.446	8.479	0.841	50	29.651	9.136	0.838

The experiment is repeated for the AE-based DBRN with four hidden layers. The forecasting performances of the testing dataset are listed in Table 6. It shows that the evaluation values with four hidden layers are firstly worse than that with three hidden layers. Due to this phenomenon, a three-hidden-layer AE-based DBRN is selected as the optimal architecture for the PM_2.5 concentration forecasting, i.e., 5-45-45-45-1.

Table 6

Results of an AE-based DBRN by four hidden layers with different hidden nodes for PM_2.5 forecasting

Hidden nodes	MAPE	RMSE	CC	Hidden nodes	MAPE	RMSE	CC
5	29.587	10.062	0.796	30	29.599	10.630	0.797
10	29.610	9.945	0.797	35	28.991	9.926	0.798
15	27.007	9.924	0.795	40	29.087	9.945	0.795
20	31.756	10.363	0.793	45	23.794	10.063	0.793
25	28.140	10.000	0.797	50	27.119	9.903	0.798

Following the experimental procedures aforementioned, an AE-based DBRN structures for the PM₁₀ forecasting are trained with six input nodes, and the performances for the testing data are concluded in Table 7. Results show that the AE-based DBRN with 45 hidden nodes in the first hidden layer has the optimal performance in terms of MAPE = 21.056%, RMSE = 11.837μg/m³, and CC = 0.800. The model with 30 hidden nodes in the second hidden layer achieves the best performance according to the three criteria, and the model with 25 hidden nodes in the third hidden layer obtains the best outcomes. Therefore, a three-hidden-layer AE-based DBRN is chosen as the optimal architecture for the PM₁₀ concentration forecasting, i.e., 6-45-30-25-1. This experiment also indicates that the learning performances of the multiple hidden layers are better than that of one hidden layer. Nevertheless, its performances are not related to the number of the hidden layers. In other words, the deep architecture with too many hidden layers may lead to the over-fitting issue.

Table 7

Results of an AE-based DBRN with different hidden structures for PM₁₀ forecasting

Hidden level	Hidden nodes	MAPE	RMSE	CC	Hidden nodes	MAPE	RMSE	CC
1	5	24.653	12.606	0.783	30	24.516	11.862	0.801
	10	23.178	11.918	0.792	35	22.415	11.917	0.799
	15	23.724	11.665	0.802	40	24.394	11.887	0.799
	20	23.505	12.383	0.790	45	21.056	11.837	0.800
	25	23.721	12.257	0.793	50	24.190	12.034	0.794
2	5	20.963	11.358	0.812	30	20.244	11.444	0.817
	10	20.837	11.447	0.812	35	21.234	11.486	0.818
	15	22.998	11.234	0.813	40	23.086	11.280	0.815
	20	21.980	11.237	0.817	45	20.464	11.283	0.815
	25	22.760	11.445	0.812	50	22.700	11.288	0.815
3	5	19.705	11.202	0.824	30	24.773	11.417	0.823
	10	23.631	11.163	0.825	35	27.722	11.839	0.822
	15	22.479	10.923	0.825	40	25.108	11.354	0.824
	20	24.084	11.143	0.823	45	20.001	11.394	0.823
	25	19.474	11.239	0.826	50	23.136	11.351	0.823
4	5	22.743	11.923	0.798	30	26.424	12.090	0.802
	10	29.547	12.674	0.796	35	22.508	11.689	0.804
	15	21.582	12.060	0.798	40	22.173	11.965	0.797
	20	25.323	12.019	0.802	45	19.256	12.299	0.800
	25	27.666	12.606	0.796	50	21.845	11.959	0.799

Figure 5 plots the forecasting results using the optimal network structures. Figure 5(a) shows the PM_2.5 concentration forecasting results with 5-45-45-45-1, and Fig. 5(b) displays the PM₁₀ concentration forecasting results with 6-45-30-25-1. To investigate the agreement between the recorded and forecasted value, the scatter plots are also given in Fig. 5.

Fig.5

Forecasting results and scatter plots with AE-based DBRN. (a) PM_2.5, (b) PM₁₀.

From Fig. 5, it can find that the forecasting results can track the fluctuations successfully, except for the peak values during the late spring and early summer. Affected by the seasonal alternate, atmospheric conditions change frequently, unsatisfactory performance appears. In addition, it can be found from the scatter plots that there have fewer angles between the ideal fit (solid line) and the real fit (dash line). It illustrate that there exist a certain degree of convergence between the forecasted values and the recorded ones. According to Table 1, three statistics criteria are computed: MAPE = 21.092% (PM_2.5), 19.474% (PM₁₀), RMSE = 8.475μg/m³ (PM_2.5), 11.239μg/m³ (PM₁₀), and CC = 0.840 (PM_2.5), 0.826 (PM₁₀), respectively. The qualitative and quantitative results reveal that the proposed AE-based DBRN model has a successful application in the PM forecasting.

4.3 Comparison study

To evaluate the superiority of the proposed approach, two “shallow learning” approaches, i.e., the FFNN and the LSSVR is employed to forecast the daily PM concentrations using the same data. The input-output structure of the comparison models is the same as the AE-based DBRN.

For FFNN, the optimal network structure is trained as 5-10-1 (input nodes 5, hidden nodes 10, and output nodes 1) for the PM_2.5 forecasting, and 6-10-1 (input nodes 6, hidden nodes 10, and output nodes 1) for the PM₁₀ forecasting. For LSSVR [29], the parameters are trained by ten-fold cross validation. The forecasting results and the scatter plots of the testing data using FFNN and LSSVR models trained are shown in Fig. 6.

Fig.6

Forecasting results and scatter plots using different models. (a) FFNN for PM_2.5, (b) FFNN for PM₁₀, (c) LSSVR for PM_2.5, and (d) LSSVR for PM₁₀.

As shown in Fig. 6, both FFNN and LSSVR models have acceptable forecasting results. However, the same phenomena those significant deviations during the late spring and early summer also appeared. Compared with the AE-based DBRN model, these deviations are greater, and the discrete degree between the forecasted and the recoded values is lager. The qualitative results demonstrate that the methods based on the shallow learning cannot extract features sufficiently, leading to the unfortunate forecasting results.

Besides, a quantitative estimation is listed in Table 8. The three criteria also indicate that the proposed model has the best forecasting performance among the comparison approaches.

Table 8

Comparison of the forecasting performances using different models

Model	PM	Performance criteria
		MAPE (%)	RMSE (μg/m³)	CC
AE-based DBRN	PM_2.5	21.092	8.475	0.840
	PM₁₀	19.474	11.239	0.826
FFNN	PM_2.5	26.686	8.532	0.820
	PM₁₀	23.719	11.274	0.808
LSSVR	PM_2.5	24.667	9.306	0.822
	PM₁₀	23.071	12.277	0.794

In addition, individual air quality index (IAQI), a number to define the pollution level, is utilized to evaluate the forecasting accuracy rank of the models. The bigger the AQI is, the severer the PM pollution is [30]. Based on the air quality standards of China, the IAQI of PM are classified into six levels, i.e., I 0–50 (excellent), II 51–100 (good), III 101–150 (slight pollution), IV 151–200 (moderate pollution), V 201–300 (heavy pollution), and VI more than 300 (hazardous). The IAQI values of each day are calculated using Equation (5). $IAQI = \frac{I_{high} - I_{low}}{C_{high} - C_{low}} (C - C_{low}) + I_{low},$ (5) where C denotes the PM concentrations (the records or the forecasts), C_low and C_high represent the limited value of the PM concentrations, and I_low and I_high are the limited value of the PM index. C_low, C_high, I_low and I_high are the constants, defined by “Technical Regulation on Ambient Air Quality Index (on trial) (HJ-633-2012)” published by the Ministry of Environmental Protection of China.

After getting the IAQI of the testing periods for both the recodes and forecasts, the numbers of the relevant day are matched to estimate the rank forecasting accuracy. Table 9 gives the confusion matrix of the IAQI forecasted with the AE-based DBRN (Table 9a), FFNN (Table 9b), and LSSVR models (Table 9c), respectively. In the confusion matrix, each row shows the number of the relevant days of the recorded rank, and each column denotes the number of the relevant days of the forecasted rank. The results shown in Table 9 for the PM_2.5 and the PM₁₀ are separated by the slash (/).

Table 9a

Confusion matrix of the IAQI evaluation using the AE-based DBRN model

Forecasted\Recorded	I	II	III	IV	V	VI	Accuracy (%)
I	59/46	20/21	0/0	0/0	0/0	0/0	74.68/68.66
II	0/0	131/167	1/1	0/0	0/0	0/0	99.24/99.40
III	0/0	0/1	26/4	0/0	0/0	0/0	100/80
IV	0/0	0/0	0/0	0/0	0/0	0/0	100/100
V	0/0	0/0	0/0	1/0	2/0	0/0	66.67/100
VI	0/0	0/0	0/0	0/0	0/0	0/0	100/100
Accuracy (%)	100/100	86.75/88.36	96.30/80	0/100	100/100	100/100	90.83/90.42

Table 9b

Confusion matrix of the IAQI evaluation using the FFNN model

Forecasted\Recorded	I	II	III	IV	V	VI	Accuracy (%)
I	57/37	22/30	0/0	0/0	0/0	0/0	72.15/55.22
II	0/0	130/168	2/0	0/0	0/0	0/0	98.48/100
III	0/0	2/2	24/3	0/0	0/0	0/0	92.31/60
IV	0/0	0/0	0/0	0/0	0/0	0/0	100/100
V	0/0	0/0	0/0	1/0	2/0	0/0	66.67/100
VI	0/0	0/0	0/0	0/0	0/0	0/0	100/100
Accuracy (%)	100/100	84.42/88.42	100/100	0/100	100/100	100/100	88.75/86.67

Table 9c

Confusion matrix of the IAQI evaluation using the LSSVR model

Forecasted\Recorded	I	II	III	IV	V	VI	Accuracy (%)
I	47/35	32/32	0/0	0/0	0/0	0/0	59.49/52.24
II	0/0	129/168	3/0	0/0	0/0	0/0	97.73/100
III	0/0	0/1	26/4	0/0	0/0	0/0	100/80
IV	0/0	0/0	0/0	0/0	0/0	0/0	100/100
V	0/0	0/0	0/0	2/0	1/0	0/0	33.33/100
VI	0/0	0/0	0/0	0/0	0/0	0/0	100/100
Accuracy (%)	100/100	86.75/88.42	96.30/100	0/100	100/100	100/100	84.58/86.25

As shown in Table 9, it is clearly that the overall forecasting accuracy rank of the level (IAQI) is the AE-based DBRN (90.83% for PM2.5, 90.42% for PM10) >FFNN (88.75% for PM2.5, 86.67% for PM10) >LSSVR (84.58% for PM2.5, 86.25% for PM10). The results also prove that the AE-based DBRN model has the best forecasting ability due to its deep feature representation.

To sum up, it is apparent that the forecasting performance of the testing data is satisfactory when applying the deep learning framework, because that the deep learning with good feature representation capability can capture the characteristics of the time series by the “layer-wise” learning from the multiple hidden layers.

5 Conclusions

In this study, an autoencoder-based deep belief regression network (AE-based DBRN) is built for the PM forecasting. Three steps of the proposed model are implemented as follows: (a) Perform generative unsupervised learning layer-wise on the AE from lower to higher layers. (b) Fine-tuning using supervised learning method on the entire DBN to tweak the weights from top to lower layers. (c) Active the regression model using the results of the top layer of the AE-based DBN. The proposed approach is trained and tested using the real data of the PM_2.5 and PM₁₀ in Chongqing, China. Moreover, the meteorological data in the corresponding periods are also employed for this task. To investigate the forecasting performance, the AE-based DBRN is compared with two classical “shallow learning” models. As shown in the experiment and the comparison, the proposed model exhibits a good feature capture capacity, and overwhelms the competitors in accordance with the three statistics criteria and the forecasting accuracy rank of the IAQI level. In addition, the AE-based DBRN model performs the same forecasting abilities, demonstrating its generalizability in the air pollution forecasting.

Footnotes

Acknowledgments

This work is supported in part by the National Natural Science Foundation of China (51775112), National Key Research & Development Program of China (2016YFE0132200), the Postdoctoral Science Foundation of China (2016M602459), and the Research Program of Higher Education of Guangdong (2016KQNCX165). We also thank the anonymous referees for their constructive comments that have helped improve this paper.

References

European Environment Agency, Air quality in Europe-Report, (2014).

Fajersztajn

, Veras

, Barrozo

L.V.

, Saldiva

, Airpollution: A potentially modifiable risk factor for lung Cancer, Nature Reviews Cancer13 (9) (2013), 674–678.

Mannucci

P.M.

, Harari

, Martinelli

, Franchini

, Effects onhealth of air pollution: A narrative review, Internal &Emergency Medicine10 (6) (2015), 657–662.

Cobourn

W.G.

, An enhanced PM_2.5 air quality forecast model based on nonlinear regression and back-trajectory concentrations, Atmospheric Environment44 (25) (2010), 3015–3023.

Saide

P.E.

, Carmichael

G.R.

, Spak

S.N.

, Gallardo

, Osses

A.E.

, Mena-Carrasco

M.A.

and Pagowski

, Forecasting urban PM₁₀ and PM_2.5 pollution episodes in verystable nocturnal conditions and complex terrain using WRF-Chem COtracer model, Atmospheric Environment45 (16) (2011), 2769–2780.

Djalalova

, Monache

L.D.

, Wilczak

, PM_2.5 analogforecast and Kalman filter post-processing for the CommunityMultiscale Air Quality (CMAQ) model, Atmospheric Environment108 (2015), 76–87.

Jian

, Zhao

, Zhu

Y.P.

, Zhang

, Bertolatti

, Anapplication of ARIMA model to predict submicron particleconcentrations from meteorological factors at a busy roadside inHangzhou, China, Science of the Total Environment426 (2) (2012), 336–345.

Domańska

, Woktylak

, Application of fuzzy time seriesmodels for forecasting pollution concentrations, Expert Systemswith Applications39 (9) (2012), 7673–7679.

Pai

T.Y.

, Hanaki

, Chiou

R.J.

, Forecasting hourly roadsideparticulate matter in Taipei county of Taiwan based on first-orderand one-variable grey model, Clean Soil Air Water41 (8) (2013), 737–742.

10.

Russo

, Lind

, Raischel

, Trigo

, Mendes

, Neuralnetwork forecast of daily pollution concentration using optimalmeteorological data at synoptic and local scales, Atmospheric Pollution Research6 (3) (2015), 540–549.

11.

Reid

C.E.

, Jerrett

, Petersen

M.L.

, Pfister

, Morefield

, Tager

, Raffuse

and Balmes

, Spatiotemporal prediction of fine particulate matter during the Northern California wildfires using machine learning, Environmental Science & Technology49 (6) (2015), 3887–3896.

12.

Bai

, Li

, Wang

X.X.

, Xie

J.J.

, Li

, Air pollutantsconcentrations forecasting using back propagation neural networkbased on wavelet decomposition with meteorological conditions, Atmospheric Pollution Research7 (3) (2016), 557–566.

13.

Niu

, Gan

, Sun

, Li

, Application ofdecomposition-ensemble learning paradigm with phase spacereconstruction for day-ahead PM_2.5 concentration forecasting, Journal of Environmental Management196 (2017), 110–118.

14.

, Wang

, Le

, Khorram

, Prediction of particularmatter concentrations by developed feed-forward neural network withrolling mechanism and gray model, Neural Computing &Applications26 (8) (2015), 1789–1797.

15.

Zou

, Wang

, Wan

, Wilson

J.G.

, Fang

, Tang

, Spatialmodeling of PM_2.5 concentrations with a multifactoralradial basis function neural network, Environmental Science &Pollution Research22 (14) (2015), 10395–10404.

16.

Dunea

, Pohoata

, Iordache

, Using wavelet-feedforwardneural networks to improve air pollution forecasting in urbanenvironments, Environmental Monitoring & Assessment187 (7) (2015), 477–492.

17.

, Oliveira

J.V.D.

, Sanchez

, Cerrada

, Fuzzydetermination of informative frequency band for bearing fault detection, Journal of Intelligent & Fuzzy Systems30 (2016), 3513–3525.

18.

Hinton

G.E.

, Osindero

, Teh

Y.W.

, A fast learning algorithm fordeep belief nets, Neural Computation18 (7) (2006), 1527–1554.

19.

, Cabrera

, Oliveira

J.V.D.

, Sanchez

R.V.

, Cerrada

and Zurita

, Extracting repetitive transients for rotating machinery diagnosis using multiscale clustered grey infogram, MechanicalSystems & Signal Processing76-77 (2016), 157–173.

20.

Rahhal

M.M.A.

, Bazi

, Alhichri

, Alajlan

, Melgani

and Yager

R.R.

, Deep learning approach for active classification of electrocardiogram signals, Information Sciences345 (1) (2016), 340–354.

21.

Bai

, Chen

Z.Q.

, Xie

J.J.

, Li

, Daily reservoir inflowforecasting using multiscale deep feature learning with hybridmodels, Journal of Hydrology532 (2016), 193–206.

22.

Bai

, Sun

Z.Z.

, Zeng

, Deng

, Li

, A multi-pattern deepfusion model for short-term bus passenger flow forecasting, Applied Soft Computing58 (2017), 669–680.

23.

Ong

B.T.

, Sugiura

, Zettsu

, Dynamically pre-trained deeprecurrent neural networks using environmental monitoring data forpredicting PM_2.5, Neural Computing andApplications27 (6) (2016), 1553–1566.

24.

, Peng

, Hu

, Shao

, Chi

, Deep learningarchitecture for air quality predictions, Environmental Scienceand Pollution Research23 (22) (2016), 22408–22417.

25.

Vincent

, Larochelle

, Bengio

, Manzagol

P.A.

, Extractingand composing robust features with denoising autoencoders, International Conference on Machine Learnin, New York, USA, 2008, 1096–1103.

26.

Bengio

, Courville

, Vincent

, Unsupervised feature learning and deep learning: A review and new perspectives, Cornell University Library; 2014, arXiv: 1206.5538v3.

27.

Längkvist

, Karlsson

, Loutfi

, A review of unsupervised feature learning and deep learning for time-series modeling, Pattern Recognition Letters42 (1) (2014), 11–24.

28.

Martellini

, Giannoni

, Lepri

, Katsoyiannis

and Cincinelli

, One year intensive PM_2.5 bound polycyclicaromatic hydrocarbons monitoring in the area of Tuscany, Italy.Concentrations, source understanding and implications, Environmental Pollution164 (1) (2012), 252–258.

29.

Bai

, Wang

, Li

, Xie

J.J.

, Wang

, Dynamic forecast ofdaily urban water consumption using variable-structure supportvector regression model, Journal of Water Resources Planning and Management141 (3) (2015), 04014058.

30.

Plaia

, Salvo

F.D.

, Ruggieri

, Agró

, Amultisite-multipollutant air quality index, Atmospheric Environment70 (2) (2013), 387–391.