Implementing Spectral Decomposition of Time Series Data in Artificial Neural Networks to Predict Air Pollutant Concentrations

Input and output data

Two criteria pollutants, NO_x, and PM₁₀, were studied to examine the ability of models in predicting both gaseous and particulate pollutants. These components are among the key pollutants of large cities, which have also frequently exceeded the allowable thresholds in Tehran during the recent years. Both of these pollutants are affected by complex primary and secondary formations, and physiochemical and photochemical parameters (Seinfeld and Pandis, 2006). PM₁₀ and NO_x concentrations were obtained from one of the central Tehran Air Quality Control Company's (AQCC) stations, Fatemi station. This station is located in the central part of Tehran (45°45’N, 30°50’ E) surrounded by main streets, with usually congested traffic, and affected by common urban sources. NO_x concentrations were measured with the Chemiluminescence NO_x analyzer (Model AC31M, Environment SA), and PM₁₀ mass concentrations were measured by the beta attenuation mass monitor (BAM, Model 1020; Met One Instruments, Inc.), operated at standard conditions.

Potentially influential meteorological variables on air pollutants' concentrations were used as predictors for modeling hourly pollutants' concentrations. Meteorological variables used in this study were obtained from Mehrabad station, the nearest synoptic station to the studied air quality station, located at 35°41’N, 51°19’E and 1,190 m above the mean sea level. Among all meteorological variables measured at the Mehrabad station, four parameters were chosen as ANN's meteorological input variables based on the criteria presented in our previous study (Arhami et al., 2013) for optimized model's performance. These selected variables are: Wind Speed (WS) in m/s, Wind Direction (WD) in degree, Air Temperature (Temp) in °C, and Relative Humidity (RH) in percent. Two other variables, hour of the day (HOD) and month of the year (MOY), were also used as input parameters representing traffic pattern and variations of atmospheric conditions through the year and day. Moreover, for all the analysis in this study, seven separated ANNs were trained and used for each day of the week, representing different traffic pattern during the week. The significance of using HOD and MOY and separating the weekdays were completely explained in Arhami et al. (2013). Three input parameters of HOD, MOY, and WD, which have periodical rotation, were incorporated using Equations (1)–(3), respectively. \documentclass{aastex}\usepackage{amsbsy}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{bm}\usepackage{mathrsfs}\usepackage{pifont}\usepackage{stmaryrd}\usepackage{textcomp}\usepackage{portland, xspace}\usepackage{amsmath, amsxtra}\pagestyle{empty}\DeclareMathSizes{10}{9}{7}{6}\begin{document} \begin{align*} { \rm HOD } = \Bigg ( \cos \left( { \frac { 2 \pi \rm h } { 24 } } \right) , \sin \left( { \frac { 2 \pi \rm h } { 24 } } \right) \Bigg ) \tag { 1 } \end{align*} \end{document} \documentclass{aastex}\usepackage{amsbsy}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{bm}\usepackage{mathrsfs}\usepackage{pifont}\usepackage{stmaryrd}\usepackage{textcomp}\usepackage{portland, xspace}\usepackage{amsmath, amsxtra}\pagestyle{empty}\DeclareMathSizes{10}{9}{7}{6}\begin{document} \begin{align*} { \rm MOY } = \Bigg ( \cos \left( { \frac { 2 \pi \rm m } { 12 } } \right) , \sin \left( { \frac { 2 \pi \rm m } { 12 } } \right) \Bigg ) \tag { 2 } \end{align*} \end{document} \documentclass{aastex}\usepackage{amsbsy}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{bm}\usepackage{mathrsfs}\usepackage{pifont}\usepackage{stmaryrd}\usepackage{textcomp}\usepackage{portland, xspace}\usepackage{amsmath, amsxtra}\pagestyle{empty}\DeclareMathSizes{10}{9}{7}{6}\begin{document} \begin{align*} { \rm WD } = \Bigg ( \cos \left( { \frac { 2 \pi \rm d } { 360 } } \right) , \sin \left( { \frac { 2 \pi \rm d } { 360 } } \right) \Bigg ) \tag { 3 } \end{align*} \end{document}

Spectral decomposition of time series data

The Kolmogorov–Zurbenko (KZ) filter was used in this study which is a low-pass filter removing high-frequency variation from the data time series. This filter was used to decompose time series into the temporal components of long term, seasonal, and short term. The trends of each of these components reflect different temporal variation in pollutants' sources and conditions. The long-term trend generally reflects the changes in the overall emission pattern, pollutant transport, climate condition, implemented policies, and economic situations; seasonal components mainly relates to solar cycle; and short-term component varies with sub-daily variations in weather condition and pollutants' sources activities, such as traffic volume (Wise and Comrie, 2005). The long-term, seasonal, and short-term components were computed by varying the length of the window and the number of iterations of KZ filter, as explained in previous studies (Rao and Zurbenko, 1994; Eskridge et al., 1997; Hogrefe et al., 2003; Wise and Comrie, 2005). Short-term variations contained cycles with period less than days and long-term trend reflected cycles longer than a year (Eskridge et al., 1997; Wise and Comrie, 2005), and seasonal window was in between. The baseline components of pollutants which are mainly their urban background concentrations were obtained as summation of the long and seasonal components (Hogrefe et al., 2003; Wise and Comrie, 2005). The KZ filters method is able to deal with the time series containing some missed data, which fits the case of pollutants of concern measured in Tehran (Eskridge et al., 1997; Wise and Comrie, 2005).

Artificial neural networks

Multilayer Perceptrons (MLPs) neural network were selected to model pollutants' levels and their components. The MLP neural networks were shown as suitable ANN types for modeling air pollutants' concentration in several previous studies (Hornik et al., 1989; Gardner and Dorling, 1999; Kolehmainen et al., 2001; Ibarra-Berastegi et al., 2008; Hoi et al., 2009; Hrust et al., 2009; Yazdanpanah et al., 2009; Arhami et al., 2013). Backpropagation learning algorithm was used to train these networks and the activation function used to convert a neuron's weighted input to its output activation was sigmoid. The MLP networks are made of an input, an output, and hidden layers, and each layer is comprised of several neurons. The optimized neural network's structure was selected by varying the numbers of hidden layers and neurons on each layer, through a trial and error procedure. The number of hidden layers of 3–5 and number of neurons on each layer of 14–24 were used for networks constructed for this study.

A key factor in optimizing ANN performance is the input data set volume and as it was described and examined in our previous study (Arhami et al., 2013), the best performances were obtained by using the most recent 1 year of input variables for short-term predictions. The data set were initially divided into two subsets, one for training and testing (validation) and one for prediction period. Subsequently, the data set for training and testing were randomly divided into two subsets, 85% of the data were dedicated for training and 15% for testing procedures. The networks were trained using the training subset, and then their performances were evaluated by testing the subset. Subsequently, the performances of the optimized networks to predict the pollutants' level were examined during the prediction periods, which were not used in networks' construction, including training and testing procedures. The whole data set used in this study were within the time period of January 2002 to January 2008. For seasonal and long-term components, 5 years of data set (January 2002 to December 2006) were used to train and test networks and one whole year (year 2007) was used for prediction period. For short-term component, whole year 2007 was used for training and testing, and 1 month of January 2008 was used for prediction period.

To evaluate the networks' performances, three statistical parameters of Mean Absolute Error (MAE), Root Mean Square Error (RMSE), and Index of Agreement (IA), in addition to correlation coefficient (R²) and slope of trend line (S), were calculated by comparing the observed and predicted concentrations.

Developing the model

A model by using both KZ filter and ANN was developed to model the air pollutants' level. As described, KZ filter was used to separate three components of NO_x and PM₁₀ concentrations and selected meteorological variables (WD, WS, Temp, and RH) time series, and ANN to model pollutants' level and/or their components. Three different approaches, Method 1 to 3, for combining components of dependent and independent data with ANN to develop the model were examined, as shown in Table 1. The performance of these three methods in predicting pollutants' levels were evaluated.

In the first method, the nonfiltered time series of the independent (input) variables were used through three different networks to model three components of pollutants' concentrations separated by KZ filter. These three modeled components were summed up to obtain the hourly pollutant concentrations. In the second method, all three components of the meteorological variables were used as input data set for modeling the nonfiltered pollutant concentration. In other words, instead of using hourly values of the meteorological variables, their components were used to model hourly pollutant concentrations. This method helps to understand the importance and impact of separating time series to the networks performance. In the third method, three different components of filtered input data sets were used to train and predict three different corresponding components of filtered pollutants' time series. In this approach, short, seasonal, and long terms of meteorological predictors were used to model short, seasonal, and long pollutant concentrations, respectively. In all methods, additional variables, such as HOD and MOY, besides meteorological variables, were used as model inputs to model pollutant levels or their filtered components.

Implementing method 1 and method 3, three different ANN's for each time series components (short term, seasonal, and long term), and seven different networks for each day of a week were constructed, resulted in 21 different networks for each pollutant. Whereas by implementing method 2 only seven networks reflecting 7 days of weeks were constructed. These networks for each pollutant, were trained, validated, and used to predict different components of concentration time series as described in the methods section. Since each day of the week was trained and predicted separately, seven different networks were prepared for each pollutant. The results of each time series components modeled by seven networks for different days of a week were combined for consecutive days to obtain a continuing time series of a pollutant during the prediction period and be comparable with the measured values. By means of this comparison the corresponding statistical parameters were calculated and presented in the results section.

Finally the developed model was used to predict the baseline concentration. Baseline concentration is the summation of long and seasonal components, contains resistant variation of a signal while removing peak values from the signal. Baseline concentration modeling is beneficial in long-term planning, and observing or predicting the effect of control strategies on pollutants' level. Hence, baseline concentration data sets were modeled for each of the pollutants, using the same predictors as explained. In other words, baseline components of four meteorological variables (WS, WD, Temp, and RH) and other two inputs (HOD and MOY) are used as input variables (predictors) and baseline components of pollutants' concentrations as targets in training and prediction steps.

Time series components and selecting best model

Temporal components of long term, seasonal, and short term for NO_x, PM₁₀, and meteorological variables were obtained by KZ filter. NO_x time series from 2002 to 2007 and their separated components are presented in Fig. 1 as examples. As expected, the main fluctuations of the pollutants' time series are attributed to short-term component of this pollutant as it varied largely between ∼ −150 and +500 ppb. It was shown that hourly and daily variations reflect the noise behavior in the pollutants' time series. A harmonic trend for seasonal variation was found during the studied period; however, the amplitude of the seasonal term wave varied in different years. The fluctuations of seasonal term are much less than the short term and ranged from ∼ −45 to +75 ppb. The long-term component varied rather smoothly, whereas the noise and periodic variation of time series was omitted by short and seasonal term, respectively. The short-term, long-term, and seasonal components of other variables behaved rather similarly to NO_x, with short and seasonal term reflecting noise and periodic variations, respectively, and smoothed long-term variations. The short-term component is accountable for more than 50% of total NO_x and PM₁₀ variations. This relative contribution of each component to the total concentration variations (not magnitude) was calculated by dividing the variance of temporal components to the variance of original time series of air pollution data.

OPEN IN VIEWER

FIG. 1.

(a) Original time series of NOx data and separated temporal component, including (b) short term, (c) seasonal and (d) long term.

Performance of different methods for combining filtered data with ANNs to build the models (called method 1 to 3 as described in Table 1) were compared, to find the best method for predicting the air pollutants' data. In this regard, comparisons between predicted NO_x concentrations through different approaches, with observed levels, are presented in Fig. 2. Same input and target periods were used to train and verify the networks for all the three methods. For method 1 and 3, results of modeling three time series components of NO_x were summed up to generate the total hourly concentrations. Hence all the plotted results in Fig. 2 represent the values for hourly pollutants' concentration (not their components). Among three different methods, method 1 showed the worst, and method 3 showed the best performance. Method 2 performed rather similar to method 3, but less accurately. Method 3 was also the best method regarding prediction of the peak levels of the time series data. Besides superior modeling accuracy, this approach would provide the opportunity of modeling the time series components separately, which is not achievable while implementing method 2. Hence, this approach was selected for building the model to predict the pollutants' levels.

OPEN IN VIEWER

FIG. 2.

Predictions made by different combining methods, (a) method 1, (b) method 2, and (c) method 3.

Selection of best input parameters for ANN modeling was assessed in our previous study (Arhami et. al, 2013). To verify the suitability of the selected data set for modeling each of the time series components (long term, seasonal, and short term), and examine the models' sensitivities to each set of input variables, extra networks were trained with eliminating each of the input parameters, one at a time. Comparison of these models' performances to the model constructed by all the selected input variables, as described in the methods section, represented the impact of eliminated input variables. Data obtained for predictions' performance of these networks for short term were listed in Table 2. The performance of ANNs was highly influenced by HOD and MOY which shows the models' high sensitivity to eliminating these two predictors. As it was mentioned earlier, these two predictors represent the variations of traffic pattern (main pollutants' sources) and atmospheric conditions through the day and year. In general, short components of pollutants' time series are highly dependent on each input parameters and all of them are essential for modeling and making predictions. This analysis was also done for long and seasonal time series components, but eliminating input variables (one at a time) did not impose significant impact on the performances of models, this shows that in long and seasonal components' modeling, other five input variables covered the lack of the eliminated one in each step. This indicated that the seasonal and long term could be modeled without including all of the input parameters. MOY, WS, and Temp are suggested variables at this time according to proper performances of long and seasonal networks trained only with MOY, WS, and Temp as input parameters.

Modeling results

Since method 3 (from different methods explained in Table 1) was selected, three different ANNs were constructed for each of three different time series components of short term, seasonal, and long term. The results of these networks were added to obtain the time series of pollutants' concentrations. Prediction of hourly short-term component of NO_x and PM₁₀ concentrations were compared with observed values in Fig. 3. Also the time series of hourly NO_x and PM₁₀ concentrations (summation of short-term, seasonal, and long-term values) were plotted in Fig. 4. For short-term components of concentrations, predictions resulted in R² of 0.89 for NO_x and 0.91 for PM₁₀, which were close to their values for the total concentrations. This similarity indicated the uncertainties of the model's results in predicting that total concentrations are mainly attributed to the uncertainties in modeling the short-term component, influenced by sub-daily fluctuations in emission sources (affected mainly by traffic conditions) and meteorological variables. The performances in modeling short-term time series, due to sudden and sizable fluctuations in values, declined in comparison with long and seasonal components. In fact, the developed model performed accurately in modeling long and seasonal components, this could be attributed to slight variations and smooth changes of these components during the short prediction period of 1 month.

OPEN IN VIEWER

FIG. 3.

Comparison between observed values and predictions of developed model for short-term components of (a) NO_x and (b) PM_10.

OPEN IN VIEWER

FIG. 4.

Performance of hourly predictions of (a) NO_x and (b) PM₁₀ by optimized model.

Predictions of the developed model, using preprocessed data by KZ filter (separating time series components), were compared with predictions of ordinary ANN models trained with same input variables, without separating the components, in Fig. 5. The predictions by models were better than or similar to those of the ordinary ANN model. Although the overall performances of ordinary ANNs and this new approach were rather close, but due to separated time series and trained networks for each of them, ANNs with filtered data showed better ability in dealing with highly elevated peak concentrations. To illustrate this advantage of ANN with filtered data, the predicted period presented in Fig. 5 was selected in a way to include high pollutants' levels episodes for NO_x in Tehran. As it could be seen, the ANN approach performed better, especially in predicting peak points, where NO_x concentrations reach values of over 400–500 ppb. It is crucial to be able to have more accurate prediction of extreme pollutants' levels. These situations could reflect critical and unhealthy pollutant levels, and their predictions would help in taking proper urgent actions. For instance, in Tehran, episodes of elevated pollution levels frequently occurred in recent years, caused increased rate of hospital admission, morbidity, and mortality (Saadat et al., 2009, 2010; Nabavi et al., 2012). During these episodes local authorities were forced to take urgent actions such as asking the children and elderly to stay indoors, enforcing traffic limitations, and shutting down some organizations and schools. More reliable forecasting of highly elevated levels could be beneficial to the officials for better management of such crisis and protect public health as effective as possible. The developed models could be helpful at this point as they showed to be more reliable in predicting peak levels. The other advantage of using the model is providing separated predictions of each of the time series components and could predict their trend of variations. An important aspect of this advantage is presented and discussed in the following section.

OPEN IN VIEWER

FIG. 5.

Comparison of predictions' performance between ordinary optimized Artificial Neural Network (ANN) and developed hybrid model of ANN with filtered data for the period of containing the episode of elevated pollution levels in Tehran.

Predicting baseline levels

It is beneficial for policy makers to access predictions for different components of time series, such as baseline concentration component. In fact, variation in this component reflects the variation in time series concentrations excluding the sub-daily variations and it reflects the ambient urban background concentration which residences are exposed to. Hence, baseline concentration data sets were modeled for each of the pollutants, using the same predictors as explained earlier while implementing method 3. The separate trained networks were used to predict 1 year (2007) of baseline concentrations (seasonal and long-term components) using 5 years of baseline data which were not inserted in the training step (2002 to 2006), with R²=0.88 for NO_x and 0.90 for PM₁₀ and IA=0.96 for NO_x and 0.97 for PM₁₀. The whole year of hourly prediction (weekly moving average) for NO_x and PM₁₀ were plotted in Fig. 6. Modeling/predicting baseline concentration could be potentially beneficial in long-term planning, and observing or predicting the effect of control strategies on the pollutants' levels. This long period prediction for baseline concentration would illustrate a rather clear view of the pollutants' level in the coming years in the presence of meteorological predictors. Such models could be beneficial for the strategist who makes long-term mitigation planning or observe the effect of previously implemented actions.

OPEN IN VIEWER

FIG. 6.

Long-term prediction (∼1 year ahead) of baseline concentration for (a) NO_x and (b) PM_10.