Particular matter prediction using synergy of multiple source urban big data in smart cities

Abstract

At present, the issue of air quality in populated urban areas is recognized as an environmental crisis. Air pollution affects the sustainability of the city. In controlling air pollution and protecting its hazards from humans, air quality data are very important. However, the costs of constructing and maintaining air quality registration infrastructure are very expensive and high, and air quality data recording at one point will not be generalizable to even a few kilometers. Some of the gains come from the integration of multiple data sources, which can never be achieved through independent single-source processing. Urban organizations in each city independently produce and record data relevant to the organization’s goals and objectives. These issues create separate data silos associated with an urban system. These data are varied in model and structure, and the integration of such data provides an appropriate opportunity to discover knowledge that can be useful in urban planning and decision making. This paper aims to show the generality of our previous research, which proposed a novel model to predict Particulate Matter (PM) as the main factor of air quality in the regions of the cities where air quality sensors are not available through urban big data resources integration, by extending the model and experiments with various configuration for different settings in smart cities. This work extends the evaluation scenarios of the model with the extended dataset of city of Aarhus, in Denmark, and compare the model performance against various specified baselines. Details of removing the heterogeneity of multiple data sources in the Multiple Data Set Aggregator & Heterogeneity Remover (MDA&HR) and improving the operation of Train Data Splitter (TDS) part of the model by focusing on the finding more similar pattern of air quality also are presented in this paper. The acceptable accuracy of the results shows the generality of the model.

Keywords

Air quality big data sustainable smart cities multi-source data urban data analytics

1. Introduction

Today, 64% of the world’s population lives in urban areas, and by 2050, it is expected to grow to two-thirds of the world’s population. In other words, more than six billion people might live in cities by 2050 [1]. Thus, cities will be the forerunners of the shift towards competitive and efficient economies with reduced air pollution [2]. Certainly, cities are considered among the major players in addressing the challenges of society and the economy, including developments with low air pollution, energy efficiency, renewable energy innovation, economic growth, and so on.

The rapid growth of urbanization has led to the creation of large cities and thus the modernization of people’s lives. This has also created new challenges such as air quality and pollution, high energy consumption, and high traffic. Today, due to the enormous computing technologies and sensors and infrastructure, huge data is being generated in urban areas, such as data of transportation, air quality, traffic patterns, and geographic data. Big data holds knowledge about cities and can overcome the challenges mentioned if used correctly. For example, it is possible to identify city road problems by analyzing the mobility data of people at the city level. This discovery could be very useful in future urban planning [3].

Air quality affects human life and breathing. As the weather changes day by day and even hour by hour, the quality of the air can also vary. Managing air quality monitoring and monitoring in large cities converts air quality data into

air quality indexes and provides the public with the information they need. Therefore, the air quality index is a key tool for understanding air quality, how air pollution affects health, and air pollution protection practices.

At present, the issue of air quality in populated urban areas is recognized as an environmental crisis. Air pollution affects the sustainability of the city. Therefore, measuring air pollution and exploiting the results is very important for predicting and discovering relationships between various urban issues. Despite the development of technology, data generation, data recording, maintenance, and processing are costly and complex in some city resources. Therefore, data recording, productivity, and data analysis are crucial in these resources. For example, in controlling air pollution and protecting its hazards from humans, air quality data are very important. However, the costs of constructing and maintaining air quality registration infrastructure are very expensive and high, and air quality data recording at one point will not be generalizable to even a few kilometers.

Some of the gains come from the integration of multiple data sources, which could never be achieved through independent single-source processing [4]. Urban organizations in each city independently produce and record data relevant to the organization’s goals and objectives. These issues create separate data silos associated with an urban system. These data are varied in model and structure, and the integration of such data provides an appropriate opportunity to discover knowledge that can be useful in urban planning and decision making.

Today, a large amount of data is available on the characteristics of cities. Due to the development and advancement of ICT in smart cities, the sources of such data are expanding. However, production and maintenance of some of these resources, such as air quality sensors, are costly. Therefore, this study seeks to replace air quality recording sensors with other available data sources, with acceptable accuracy to predict one of the most important elements of air quality, particulate matter. Based on this objective, the development of an air pollution particle parameter prediction system using the integration of a variety of large and diverse data sources in the smart city and the use of synergies designed to predict the use of air pollution sensors without the use of data source Installation and maintenance of such sensors is the focus of this research. It requires the integration of diverse data sources in smart cities to access hidden knowledge that cannot be extracted from standalone data sources.

By leveraging huge data opportunities and capabilities and integrating diverse heterogeneous data sources into smart cities, one can find the right solutions for the sustainable development of cities. In this paper, by combining different big data sources in the smart city, particulate matter is predicted as one of the main sources of air pollution without the use of costly air pollution recording sensors. In other words, the feasibility of predicting air quality levels, especially particulate matter, is examined using the economic resources of data collection in smart cities. This paper is an extended version of our previous research [5] with extended experiments to show the generality of the model.

Section 2 investigates previous works that tried to use alternative data sources in urban computing applications and research that relates to air quality predictions. Section 3 introduces the dataset we used in experiments. Section 4 details the proposed framework and model, Section 5 illustrates the experiments that were conducted as well as an evaluation of their results, to verify the generality of the model, and finally, Section 6 provides conclusions.

2. Related work

This section first explores previous works that tried to use alternative data sources, instead of the main dataset in the domain of urban computing applications. Then, it investigates recent works that tackle issues which relate to air pollution.

2.1 Alternative dataset in urban computing

A number of investigations have shown that health-promoting centers (e.g., fitness and dance centers) can be more easily accessed in rich urban neighborhoods, whereas major health-damaging centers (e.g., fast-food outlets) are mainly located in poorer districts. The official land-use information was collected for these investigations. Nevertheless, some researchers used social media as an alternative data source to explore the connection between resources and neighborhood deprivation [6, 4, 7].

The likelihood of extracting free, recent, and reliable land-use information from Foursquare’s users, to an acceptable extent, can assist the discussion about the usability of “organic data” from geospatially-referenced social network data, in contrast to curated spatial data (i.e., the official land-use information) [7]. Some authors also declared that social media (such as Foursquare) can be used to observe the physical changes in a neighborhood at reasonable temporal resolutions, over what is possible with the official land-use data [8].

Ruiyun et al. [9] suggested a method to predict the Air Quality Index (AQI) of the whole regions in Shenyang (China), in accordance with the AQI which was reported by the air quality monitoring stations, the meteorological data of the weather stations, the road information and the real-time traffic data collected from Baidu Map and Google Maps, and the POI distributions provided by Baidu Map and Google Maps. Moreover, the random forest algorithm was applied to predict all of the uncovered regions in the downtown area.

Some differences exist between the abovementioned research and this work. This study employed a model based on the concept of transfer learning to predict the PM ${}_{10}$ in the city regions, which were divided according to the main urban roads, whereas the abovementioned research adopted random forest to classify the AQI in equal grids in the city. The purpose of this research was to study the feasibility of replacing air quality measurement sensors without using air pollution sensors, through integrating other data sources. Hence, different combinations of data sources were considered for the evaluation of the system. Instead, the purpose of Ruiyun et al.’s study was to predict AQI, using data sources which were similar to this research as well as air pollution sensors. Although the nature of the data sources is similar, the distinction is in terms of the features and in the use of air pollution sensors in their research. For instance, in their work, the number of all POIs in one grid was employed as a feature for model training; in this research, instead, the total of each POI category in the region was carefully taken into account. Another example is about road traffic, for which in this research the number of cars and their average speed were the extracted features, while in Ruiyun et al.’s research traffic load using Baidu Maps was the extracted feature.

The city-wide air quality, by using a small number of the AQI stations and in accordance with Granger causality analysis, was predicted in [10]. An extended spatial-temporal (S-T) Granger causality model was presented; it investigates all the causalities among urban dynamics and air pollution in a consistent approach. The inappropriate groups of urban dynamics were avoided by executing the non-causality test. In order to cope with time efficiency, a method was suggested to recognize the Region of Influence (ROI) and next, separate “big data” into “small data”. The city-wide air quality map can be obtained and visualized by integrating the training of the urban dynamics with the identification of the ROI. The results demonstrated that the causality-based method outpaced other interpolation or training techniques because it considered all the S-T correlations and processed only highly effectual urban data.

In [11] the global PM ${}_{2.5}$ quality was predicted by using aerosol optical thickness (AOT) retrieval from the moderate resolution imaging spectroradiometer (MODIS), on board of NASA’s Terra and Aqua satellites, alongside ground measurements. Even though such methods, which are generally known as ancillary sensor networks, allow to reliably predict PM ${}_{2.5}$ without using expensive air pollution sensors, they intrinsically require expensive infrastructures and equipment.

2.2 Prediction of air pollution

Li et al. [12] developed and constructed PiMi air box, which is an economical and portable sensor and is capable of predicting PM ${}_{2.5}$ mass concentration. However, the principal problem is the sensing inaccuracy, because the existing portable sensors cannot reach a comparable precision for the monitoring stations. Additionally, the device requires more than one hour for data collection and averaging to sense PM ${}_{2.5}$ .

In [13], with the aid of air quality data for PM ${}_{2.5}$ concentration, which were gathered hourly from an environmental monitoring station in Beijing (China), from December 10, 2015 to January 10, 2016, the Elman-chaos prediction model was applied, based on the chaos theory, to predict PM ${}_{2.5}$ time series. The presented method provided enhanced predictability, resulting from Elman’s rich dynamic behavior and the addition of chaotic characteristics

Honarvar et al. [14, 15] apply spark platform for extracting sequences patterns from appliances’ power usage in smart homes. Findings show the importance of extracting sequence pattern from power usage data to various applications such as decreasing CO2 and greenhouse gas emission.

Dong et al. [16] proposed a model to extract real-time PM ${}_{2.5}$ , according to the support vector machine (SVM), by adding the POI features as inputs and using LDA to reduce dimension and wavelet decomposition to increase accuracy. Consequently, the numerical results disclosed that LDA is considerably appropriate over PCA for the reduction of dimension in the stage of data pre-processing.

Xia et al. [17] devised a technique to predict PM ${}_{10}$ at an acceptable cost and with high accuracy, by means of the artificial neural network method as well as other pollutants and meteorological aspects. An entropy maximization procedure was suggested to prevent the over-fitting issue, accompanied with the distribution of the data of the pollutant. Moreover, the selection of input features was hypothesized into an optimization problem.

Wang et al. [18] suggested a new method for assessing the environmental quality of an area using Chinese social media. They performed sentiment analysis and classification using support vector machine algorithm, which had an 85.64% accuracy rate. They defined a term called the Environmental Quality Index (EQI) and developed an environmental assessment model to assess and predict the environmental quality of the local area. The result of comparing environmental quality between different provinces of China is also very close to the air pollution rating data released by the Ministry of Environmental Protection (MEP) of the People’s Republic of China.

In [19], a machine learning framework is presented that includes multi-source heterogeneous data processing and real-time data processing. The fine-grained air quality estimates were based on data from five real-world sources, including image data provided by smartphone users. Data based on temporal and spatial distribution are divided into three categories. The results show that rational data classification can effectively ameliorate the accuracy of the assessment. On the other hand, the processing of any subclass can be more flexible. Aggregator design can effectively decrease the impact of scattered data on evaluation.

A method of estimating fine-grained PM2.5 based on random forest with data reported by meteorological departments and collected from smartphone users without any PM2.5 measuring device was presented in [20]. The researchers designed and implemented a context for real-world data collection, including user-presented imagery. By merging online learning and offline learning, the random forest-based approach performs well in terms of complexity and temporal accuracy.

Li et al. [21] proposed a new Bayesian-based kernel method to achieve fine-grained PM2.5 concentration. This model leverages heterogeneous data, which share image information, camera lens information, GPS information, and magnetic sensor information. To investigate the relationship between PM2.5 concentration and image information, a crowdsourcing system has been created and collected pictures for 16 consecutive months. The results show that compared to tripods, the proposed algorithm can reduce the forecast error by up to 35% on average.

3. DATA

Today, a large amount of data is available on the characteristics of cities. Due to the spread of Information and Communication Technology (ICT) in smart cities, the sources of such data are expanding. However, production and maintenance of some of these resources, such as air quality sensors, are costly. Therefore, this study seeks to replace air quality recording sensors (such as ozone, carbon monoxide, sulfur dioxide and nitrogen dioxide) with other available data sources to predict with precision acceptable accuracy, one of the most important elements of air quality which is particulate matter.

The data sources used are meteorology, traffic, points of interest and the structure of the street network. This data is known as metropolitan data and reflects the dynamics of the city. Data sources in the city are divided into two static and dynamic categories. Static data sources are data related to land use, waterways, buildings, roads, facilities and points of interest. Dynamic data sources represent meteorological, traffic, air quality, and parking data for the time period in question.

Aarhus city in Denmark, covers a 91-kilometer area with a population of approximately 270,000. Like any other city, it faces the issue of particulate matter in the area of air pollution. Figure 1 illustrates the study area. The points marked on the map depict the location of air quality stations.

Table 1
Data source features

Metrological and Weather Sensor Data (WEA)

Temp-high, Temp-avg, Temp-low, DewPoint-high, DewPoint-avg, DewPoint-low, Humidity-high, Humidity-avg, Humidity-low, Sea Level-high, Sea Level_avg, Sea Level_low, Visibility_high, Visibility_avg, Visibility_low, Wind_low, Wind_avg, sum_Precip

Air Pollution Sensor Data (POU)

ozon, carbon_monoxide, sulfure_dioxide, nitrogen_dioxide, particullate_matter

Point of Interest Data (POI)

accounting, airport, amusement_park, aquarium, art_gallery, atm, bakery, bank, bar, beauty_salon, bicycle_store, book_store, bowling_alley, bus_station, cafe, campground, car_dealer, car_rental, car_repair, car_wash, casino, cemetery, church, city_hall, clothing_store, convenience_store, courthouse, dentist, department_store, doctor, electrician, electronics_store, embassy, establishment, finance, fire_station, florist, food, funeral_home, furniture_store, gas_station, general_contractor, grocery_or_supermarket, gym, hair_care, hardware_store, health, hindu_temple, home_goods_store, hospital, insurance_agency, jewelry store, laundry, lawyer, library, liquor store, local_government_office, locksmith, lodging, meal_delivery, meal_takeaway, mosque, movie rental, movie_theater, moving_company, museum, night_club, painter, park, parking, pet_store, pharmacy, physiotherapist, place_of_worship, plumber, police, post_office, real_estate_agency, roofing_contractor, rv_park, school, shoe_store, shopping_mall, spa, stadium, storage, store, subway_station, synagogue, taxi_stand, train_station, transit_station, travel_agency, university, veterinary_care, zoo

Traffic Sensor Data

avgSpeed, vehicleCount

Figure 1.

The location of air quality record stations.

The study period is from August 1, 2014 until September 30, 2014. Five types of data were studied and used for this study. The data used is an expanded version of the City Pulse Project Data [22], which has been added to other sources collected from online sources. Meteorological data is extracted from the Wunderground website [23], air quality information as well as street traffic from the City Pulse project [22], points of interest data from Google Map [24], and road network structure from Openstretmap site [25]. Table 1 shows the desired characteristics of these data sources. The meteorological data as well as the traffic data were recorded 12 times per hour over a five-minute interval and produced a total of 14124555 data samples over the desired time period.

Part of the prepared dataset which is published in [26] are used for experiments of this paper. The prepared data are organized in three viewpoints of spatial granularity (grid based, main road, and circular area around the sensors). Details of this dataset can be found in our previous published paper [26].

4. Framework and prediction model

4.1 Framework

Figure 2 presents the framework of the predictive model which is introduced in previous research [5], consisting of five components: Multiple Data Set Aggregator & Heterogeneity Remover (MDA&HR), Train Data Splitter (TDS), Split Data Predictor (SDP), Test Data Cluster Finder (TDCF), and Split Data Prediction Aggregator (SDPA).

Figure 2.

Framework of the proposed predictive model.

The MDA&HR collects, integrates, and removes the heterogeneity of multiple-source data, including metrology, traffic, road network, POI, and pollution data. The heterogeneity of this dataset originates from temporal, spatial, and application domain. Temporal heterogeneity occurs as a result of various sensors in the data sources capturing data in different periods. For example, the metrological dataset is collected hourly, but traffic data are collected over shorter periods, such as 5- to 10-minute intervals. The data which are collected from sensors in the domains cover various portions of the city with different granularity. For example, weather data show the weather conditions of a region, but each traffic sensor presents the traffic conditions of a street. This causes the spatial heterogeneity of multiple data sources in the cities. The domain heterogeneity is evident because each domain shows the condition of the city from its lens, with different formats. Spatial heterogeneity is removed by dividing the city into regions and grids of various size, depending on the purpose of the study, and by mapping the features of the datasets in these regions or grids. Temporal heterogeneity is removed by exploiting aggregative functions, such as min, max, average, and sum, on the measured data features in the intended intervals for the study. For example, the average of vehicle count in one-hour period has to be considered to map and study all the datasets in 1-hour intervals. Domain heterogeneity cannot be neglected due to the conceptual and technical issues which arise in capturing the status of the city from the domain perspective. Indeed, when weather conditions are considered, humidity, wind direction, perceptions, and further elements have to be registered, but when traffic data are required, the number of vehicles and the speed of vehicles need to be captured. Domain heterogeneity cannot be omitted, although this heterogeneity is beneficial because it allows for synergy. Details of the temporal and spatial heterogeneity removing of the model described in the following section which is described in details how our models remove these heterogeneity.

The output of the MDA&HR is an Integrated Dataset (IDS) from multiple data sources. The IDS is divided into two parts: Train Integrated Dataset (Train-IDS) and Test Integrated Dataset (Test-IDS). The train and test dataset of an IDS are created based on the purpose of the study. In this paper, the test and train data are created based on the regions that are identified considering the main roads of the city. For example, if the city is divided into 11 regions, the data of 10 regions are used for the train section and the data of one region are used to test the proposed model. This process is repeated to evaluate the data of all regions as a test set. The integrated features of all data sources are included in both Train-IDS and Test-IDS implicitly for the road network or explicitly for other urban data sources. In relation to the purpose of the paper, which is to predict particulate matter using urban data excluding air quality sensors, air quality features in Test-IDS only will be used for the evaluation purpose and will not be used as the input to the proposed model. The IDS is temporally and spatially homogeneous. The TDS aims to segment the Train-IDS into multiple clusters. This causes the most similar records of the Train-IDS to be grouped in the same clusters. The rationale behind this segmentation is the fact that urban air quality is affected by multiple complex factors, such as traffic flow, metrology, and land use [27, 28], and these factors follow similar patterns of air quality for the regions of the city in each segment. Each cluster contains records of the Train-IDS that have similar patterns for urban air quality, according to the time interval parameter of the MDA&HR. These clusters are denoted by Train-IDS-Seg ${}_{i}$ . In the proposed model, the Kohenen clustering algorithm is applied in the TDS to segment the Train-IDS. In order to compute the proximity, the Euclidean distance method is used and the average of proximity of each test record to all records of the Train-IDS-Seg ${}_{i}$ is considered in the comparison. Details of temporal and spatial similarity computation described in next section.

The SDP is devised to create multiple predictive models for each segmented dataset (Train-IDS-Seg ${}_{i}$ ). The predictive model for each Train-IDS-Seg ${}_{i}$ is denoted by SDP ${}_{i}$ . Neural network and regression are used as the core of the predictive models for SDP ${}_{i}$ in the authors’ experiments, because they achieved good performance for these algorithms in the proposed model. The comparative analysis of the proposed model is also based on these conventional algorithms.

The TDCF is devised to find the proximity of the test records to all Train-IDS-Seg ${}_{i}$ clusters and to return the most similar ones. None of the air quality features are considered when the proximity of a test record is computed against the clusters of data (Train-IDS-Seg ${}_{i}$ ). The rationale behind this process is to find the SDP ${}_{i}$ which are built using the train data with more similar features of weather, traffic, POI, and road network to the test data. This causes to predict the particulate matter of the test records by those SDP ${}_{i}$ s that can achieve the maximum accuracy. The SDPA receives the aggregative function and top ${}_{k}$ as parameters of input and runs the aggregative function on the predicted values of top ${}_{k}$ SDP ${}_{i}$ models which are selected by the TDCF for each test record. For example, if the aggregative function of SDPA is average and the value of top ${}_{k}$ is 3, then the predicted particulate matter by top 3 SDP ${}_{i}$ s will be averaged and returned by the SDPA as the predicted value of the proposed model for the intended test record.

4.2 Temporal and spatial homogenization of heterogeneous data

In the MDA&HR section, time homogenization is based on Eq. (4.2). The input parameter date specifies the desired date, and part specifies the timespan for time homogenization. Depending on the scope and problem, part values can include minutes, days, hours, multi-hour intervals, etc. The variable $ds_{i}^{\textit{time\_spatial}}$ represents input data that has already been homogenized according to the spatial criteria and grouped according to the desired timeframe to be grouped by the Agrfun function on the desired properties. For example, if the desired time interval for daily homogenization is 6 hours, then the EP would be 23, SP would be 0, 6, 12 and 18, and the $I$ parameter would be 6. The data is from 3 different dates in 3 categories with values of Time $=$ 0, Time $=$ 1, Time $=$ 3, along with other attributes. This data is sent to the homogenizer function as its input.

After spatial and temporal homogenization, each data source is merged independently for the use and productivity of multiple independent data sources based on spatial and temporal homogeneity parameters.

Equation (4.2) shows this integration.

$\displaystyle ds_{i}^{\textit{time\_agr}}=G_{\textit{date},\textit{part},% \textit{time}}^{\textit{Agrfun}(\textit{features})}({ds_{i}^{\textit{time\_% spatial}}})$ $\displaystyle ds_{i}^{\textit{time}_{\textit{spatial}}}=\mathop{\sum}\limits_{% sp=I\ast i,i=0,1,\ldots,\frac{EP}{I}}^{n}\pi_{\textit{features},\textit{time}}% ({\sigma_{\textit{Date}=\textit{date}\&SP\leqslant\textit{Time}<SP+I}({ds_{i}^% {\textit{spatial\_agr}}})})$ $\displaystyle\qquad\text{Equation 1. {Temporal homogenization of heterogeneous% data}.}$ $\displaystyle\textit{IDS}=\pi_{\textit{features}_{ds1}}({ds_{1}^{\textit{% spatial}_{\textit{time}}}})\mathop{\cup}\nolimits\pi_{\textit{features}_{ds2}}% ({ds_{2}^{\textit{spatial}_{\textit{time}}}})\mathop{\cup}\nolimits\ldots% \mathop{\cup}\nolimits\pi_{\textit{features}_{\textit{dsn}}}({ds_{n}^{\textit{% spatial}_{\textit{time}}}})$ $\displaystyle\qquad\text{Equation 2. Homogeneous data source integration.}$

After preparing the IDS, the data is divided into training and test data sets. This segmentation can be done in different ways based on the type of criteria used for modeling and testing. For example, 30% of the data can be selected as a test and the rest as train data or using cross-validation with different $k$ values. IDS is homogeneous in time and space. The purpose of the IDS is to divide Train IDS into several clusters. This causes most similar training data records to be divided into different clusters. The rationale behind this segmentation is that air quality is influenced by several complex factors, such as traffic flow, meteorology, and land use, and these factors follow the same patterns of air quality for urban areas in each cluster and sector. Each cluster contains data records of train_IDS that have similar air quality patterns based on MDA&HR time parameters. The outputs of this section are shown as train_IDS_segi. Different algorithms can be used for clustering based on data type and application. Depending on the data structure, different algorithms are used to calculate the similarity of data such as Euclidean, Manhattan, cosine, and so on.

4.3 Similarity computation based on temporal and spatial properties

Given the rationale that air quality data segmentation is influenced by complex factors of traffic, meteorology, land use, transportation, and the quality of air. The following Eq. (4.3) illustrates how similarity is calculated according to the proposed criteria. Similarities are computed according to Eqs (4.3)–(4.3). Values $\textit{similarity}_{\textit{road}}$ , $\textit{similarity}_{\textit{weather}}$ , $\textit{similarity}_{\textit{transporation}}$ , and $\textit{similarity}_{\textit{landuse}}$ are represent the similarity based on road network, weather, transportation, and land use, respectively. In Eq. (4.2), $W$ represents weight and is considered for each part of the similarity computation. In $\textit{similarity}_{\textit{landuse}}$ , $\textit{Region}_{x}$ is a vector in which POIs and their numbers of each type are specified; $|{\textit{Region}_{x}}|$ shows the size of this vector. In Eq. (4.3), vector elements are number and types of roads, and $|{\textit{Region}_{x}}|$ shows the size of this vector.

$\displaystyle\textit{similarity}({\textit{Region}_{x},\textit{Region}_{y}})=[w% _{r}*\textit{similarity}_{\textit{road}}({\textit{Region}_{x},\textit{Region}_% {y}})$ $\displaystyle{}+w_{w}*\textit{similarity}_{\textit{weather}}({\textit{Region}_% {x},\textit{Region}_{y}})+w_{t}*\textit{similarity}_{\textit{transporation}}({% \textit{Region}_{x},\textit{Region}_{y}})$ $\displaystyle{}+w_{l}*\textit{similarity}_{\textit{landuse}}({\textit{Region}_% {x},\textit{Region}_{y}})]/[w_{r}+w_{w}+w_{t}+w_{l}]$ $\displaystyle\qquad\text{Equation 3. Similarity computation of regions.}$ $\displaystyle\textit{similarity}_{\textit{landuse}}({\textit{Region}_{x},% \textit{Region}_{y}})=\frac{\textit{Region}_{x}\ast\textit{Region}_{y}}{|{|{% \textit{Region}_{x}}|\ast|{\textit{Region}_{y}}|}|}$ $\displaystyle\textit{Region}_{x}=({\textit{poi}_{1},\textit{poi}_{2},\textit{% poi}_{3},\ldots,\textit{poi}_{n}})$ $\displaystyle\textit{Region}_{y}=({\textit{poi}_{1},\textit{poi}_{2},\textit{% poi}_{3},\ldots,\textit{poi}_{n}})$ $\displaystyle|{|{\textit{Region}_{x}}|}|=\sqrt{\textit{poi}_{1}^{2}\ast\textit% {poi}_{2}^{2}\ast\textit{poi}_{3}^{2}\ldots\textit{poi}_{n}^{2}}$ $\displaystyle\qquad\text{Equation 4. Similarity computation of land use.}$ $\displaystyle\textit{similarity}_{\textit{road}}({\textit{Region}_{x},\textit{% Region}_{y}})=\frac{\textit{Region}_{x}\ast\textit{Region}_{y}}{|{|{\textit{% Region}_{x}}|\ast|{\textit{Region}_{y}}|}|}$ $\displaystyle\textit{Region}_{x}=({\textit{road\_type}_{1},\textit{road\_type}% _{2},\textit{road\_type}_{3},\ldots,\textit{road\_type}_{n}})$ $\displaystyle\textit{Region}_{y}=({\textit{road\_type}_{1},\textit{road\_type}% _{2},\textit{road\_type}_{3},\ldots,\textit{road\_type}_{n}})$ $\displaystyle|{|{\textit{Region}_{x}}|}|=\sqrt{\textit{road\_type}_{1}^{2}\ast% \textit{road\_type}_{2}^{2}\ast\textit{road\_type}_{3}^{2}\ldots\textit{road\_% type}_{n}^{2}}$ $\displaystyle\qquad\text{Equation 5. Similarity computation for road network.}$ $\displaystyle\textit{similarity}_{\textit{transportaion}}({\textit{Region}_{x}% ,\textit{Region}_{y}})$ $\displaystyle{}=1-\sqrt{(\textit{region}_{\textit{speed}_{x}}-\textit{region}_% {\textit{speed}_{y}})^{2}+(\textit{region}_{\textit{count}_{x}}-\textit{region% }_{\textit{count}_{y}})^{2}}$ $\displaystyle\qquad\text{Equation 6. Similarity computation for transportation.}$

The traffic of a region can be simulated based on speed and count of vehicles. In Eq. (4.3), the $\textit{region}_{\textit{speed}}$ and $\textit{region}_{\textit{count}}$ parameters represent the speed and the number of vehicles, respectively, in the time frame in the study area. Since meteorological data has nominal and numerical variables, the similarities of these two groups of attributes are calculated independently and finally combined according to Eq. (4.3). In this respect, $p$ is the total number of nominal attributes, and $m$ is the number of nominal common attributes. The term $\textit{region}_{xi}$ represents the numeric attribute $i$ of the $x$ region, and $w_{no}$ and $w_{nu}$ are the weight given to the numeric and nominal attributes, respectively. This weight can be calculated based on the number of attributes in each set.

$\displaystyle\textit{similarity}_{\textit{weather}_{\textit{nominal}}}({% \textit{Region}_{x},\textit{Region}_{y}})=1-\frac{p-m}{p}$ $\displaystyle\textit{similarity}_{\textit{weather}_{\textit{numeric}}}({% \textit{Region}_{x},\textit{Region}_{y}})$ $\displaystyle{}=1-\sqrt{(\textit{region}_{x_{1}}-\textit{region}_{y_{1}})^{2}+% (\textit{region}_{x_{2}}-\textit{region}_{y_{2}})^{2}+\ldots+(\textit{region}_% {x_{n}}-\textit{region}_{y_{n}})^{2}}$ $\displaystyle\textit{similarity}_{\textit{weather}}$ $\displaystyle{}=\frac{w_{no}\ast\textit{similarity}_{\textit{weather}_{\textit% {nominal}}}({\textit{Region}_{x},\textit{Region}_{y}})+w_{nu}\ast\textit{% similarity}_{\textit{weatherr}_{\textit{numeric}}}({\textit{Region}_{x},% \textit{Region}_{y}})}{w_{no}+w_{nu}}$ $\displaystyle\qquad\text{Equation 7. Similarity computation of weather.}$

5. Experiments and results

Previously, the proposed model was evaluated based on main roads zones, and the results show the particularity of the approach to predict particulate matter with acceptable accuracy [5]. In addition to previous scenarios, we evaluated the models for various viewpoints of dataset in new scenarios to show the generality of the models. In these new scenarios, one region is considered as training data, and the selected region is considered for testing. In the current evaluation, the zones are considered based on the range and radius of one kilometer of traffic sensing and air quality sensors. These zones are also considered for criteria of eliminating spatial heterogeneity.

Figure 3 shows the location of the sensors and the area around each sensor. In this case, by considering a range of one kilometer within the radius of the location of each sensor, all data sources that have the geographical coordinate attribute that fall within that range will be in a position to eliminate spatial heterogeneity. POI, Traffic, air quality sensors, and road structure sensors are data sources that can be located within these areas using the geographic coordinates feature. The reason for choosing this method for zoning is to evaluate the performance of the model at smaller spatial data scales, at finer points.

In this evaluation method, as discussed earlier, within a one-kilometer range, some data sources such as points of interest will be differentiated from the regional assessment method based on the city’s main road. In this case, the number of points of interest in each area will be calculated with the desired radius and used as a feature of the data source. This is because points of interest are very diverse in terms of group and category, and practically, within a kilometer of each sensor, there is only a limited number of points of interest. Using the type of points of interest as a feature will face a problem of sparseness, negatively impacting model performance. Hence, the number of points of interest is considered as a feature of this source.

The structure of roads and streets in a city has a direct impact on the traffic of an area as well as implicitly reflecting the performance and use of an environment [29, 30]. The number of streets in an area will be extracted as another feature of the data sources of the road network structure and will be considered as input features.

According to the available meteorological data sources, all areas of the one-kilometer margin of the air pollution sensors have same feature values. Therefore, the features of this data source will not be applicable to the spatial homogenization of the different datasets. To evaluate the performance of the model, combinations of data sources that perform more efficiently in the evaluations are used. The set of combinations of data sources investigated are presented in Table 2. POI, RDN, TRA, and WEA presents point of interest, road network structure, traffic, and weather, respectively, in this table.

Table 2
Combining data sources

Number	Data source(s)
1	POI $+$ RDN
2	POI $+$ RDN $+$ TRA
3	WEA $+$ TRA
4	POI $+$ RDN $+$ WEA
5	TRA $+$ WEA
6	POI $+$ RDN $+$ TRA $+$ WEA

Figure 3.

The location of the sensors and the area around each sensor.

Air quality data is only used to evaluate the accuracy of the model; therefore, in the SDP section of the model, the air quality data source attributes will not be used as inputs in the training phase. Model performance evaluation is based on the RMSE criterion. Time intervals for experiments are repeated as in the case of assessment based on the main road. The time parameter as input is used in the MDA&HR section. This parameter specifies when to exclude temporal heterogeneity between samples of different data sources based on which time intervals are merged and aggregated.

In evaluating the model in this case, as in the previous method, the models are compared with a set of basic methods such as regression and neural network that consider a combination of data sources as input feature sets. The results are presented as the avaerage of different experiments in the presented regions. In experiments using cross-validation method with parameter $k=$ 10 at each step, one set of areas was used as test data, with the other set of areas used as training data. This means that the set of zones, which is equal to the total number of air pollution quality sensors, is divided into 10 parts, and at each stage, one set of zones is used as test data, and the other nine zones are used as training data. This operation is performed 10 times for all sections, and the average results are presented.

In the experiments, the $\textit{top}_{k}$ of the SDPA is considered for $\textit{top}_{k}=$ 1 and $\textit{top}_{k}=$ 3 as inputs. Based on the values of these parameters, and since the core of the prediction model is based on neural network and regression methods, the model is tested in four modes: TOP3REG, TOP1REG, TOP3NN, and TOP1NN. Tables 3–6 and Figs 4–7 show the results in the various considered time intervals.

Table 3

Estimation values over a 3-hour radius-based period

	REG	TOP3REG	TOP1REG	NN	TOP3NN	TOP1NN
POI $+$ RDN	28.55	31.617	29.071	24.854	25.226	24.461
POI $+$ RDN $+$ TRA	28.55	29.232	29.194	24.797	25.212	24.565
WEA $+$ TRA	29.476	30.341	29.156	24.691	25.23	24.515
POI $+$ RDN $+$ WEA	28.544	30.422	28.912	25.35	25.306	24.306
TRA $+$ WEA	29.328	30.483	29.34	25.562	25.264	26.817
POI $+$ RDN $+$ TRA $+$ WEA	29.265	30.512	24.72	25.448	25.015	24.331

Table 4

Estimation values over a 6 hour radius-based period

	REG	TOP3REG	TOP1REG	NN	TOP3NN	TOP1NN
POI $+$ RDN	27.89133	31.617	27.07914	27.57062	25.226	24.96162
POI $+$ RDN $+$ TRA	29.47598	29.74219	32.90724	27.83882	27.15983	26.83121
WEA $+$ TRA	28.12763	32.45645	26.93858	28.28267	25.23	24.515
POI $+$ RDN $+$ WEA	27.71988	33.28247	28.912	28.00007	23.02496	22.64851
TRA $+$ WEA	29.328	29.83986	30.0122	28.54111	24.12607	24.51672
POI $+$ RDN $+$ TRA $+$ WEA	31.40137	31.79569	27.42804	26.46515	22.13312	22.03524

Table 5

Evaluation values over a 12 hour radius-based period

	REG	TOP3REG	TOP1REG	NN	TOP3NN	TOP1NN
POI $+$ RDN	26.54296	33.73245	24.86172	31.1623	25.226	24.96162
POI $+$ RDN $+$ TRA	28.65186	32.60266	32.90724	30.48888	27.8788	27.17371
WEA $+$ TRA	28.12763	31.81331	27.61078	31.26178	27.09207	25.21472
POI $+$ RDN $+$ WEA	29.85625	34.56616	31.62004	29.01722	24.14308	24.35274
TRA $+$ WEA	31.64629	30.56615	27.74266	29.08289	25.84032	24.72521
POI $+$ RDN $+$ TRA $+$ WEA	31.40137	30.27966	25.6856	29.54902	25.76438	24.41973

Table 6

Estimation values over a 24-hour radius-based period

	REG	TOP3REG	TOP1REG	NN	TOP3NN	TOP1NN
POI $+$ RDN	25.7066	31.29705	28.71172	29.63888	26.16682	26.5132
POI $+$ RDN $+$ TRA	28.65186	33.73831	32.90724	30.48888	25.09317	26.73463
WEA $+$ TRA	30.49432	28.01513	26.14826	29.30863	25.64918	25.08942
POI $+$ RDN $+$ WEA	27.96605	32.58325	32.24313	27.90629	26.51246	24.65932
TRA $+$ WEA	32.64845	28.91393	28.24071	26.96205	27.91227	26.10963
POI $+$ RDN $+$ TRA $+$ WEA	33.52048	28.5839	27.37718	32.06036	25.50327	24.63112

Figure 4.

Comparison of the evaluation over a 3 hour radius-based period.

Figure 5.

Comparison of the evaluation over a 6-hour radius-based period.

Figure 6.

Comparison of the evaluation over a 12 hour radius-based period.

Figure 7.

Comparison of the evaluation over a 24-hour radius-based period.

As can be seen at lower intervals, the model performs best in TOP1NN mode in combining POI, RDN, TRA, and WEA data sources. The best model performance in combining these data sources is in the TOP1NN data for a period of six hours, with a square error rate of 22.035. However, in this timeframe, the simple regression model performs better than the proposed model with the regression kernel, and the simple neural network model performs better than the ordinary regression method. When the model performs best in TOP1NN, the TOP3REG method performs more poorly than TOP1REG. The weakest model of the TOP1NN is the six-hour timeframe when the combination of POI, RDN, and TRA data sources is used. Considering the larger time intervals, the overall performance of the proposed model is generally weaker than the six-hour time interval.

In Fig. 8, model evaluation results are shown in different combinations of data sources and main road-based zones, while the data of one area as training and data of area is the most similar one as test. In calculating the most similar area, it takes into account the similarity of points of interest, meteorology, type of roads and traffic. Based on these results, the best performance of the TOP3NN method is obtained when combining points of interest, traffic, and meteorological data sources. In Fig. 9, the results are compared with the region that has the most dissimilarity in terms of road structure, traffic, meteorology, and points of interest. It is evident that the performance and accuracy of the model have decreased; this confirms the consideration of the part of the model that tries to use the data with most similar pattern to the intended area for predictions.

Figure 8.

RMSE values for a 24-hour time interval (spatial heterogeneity $=$ regions based on main roads) Comparing the selective region with the most dissimilar region.

Figure 9.

RMSE values for a 24-hour time interval (spatial heterogeneity $=$ regions based on main roads) Comparing the selective region with the most dissimilar region.

Figure 10.

RMSE values for a 6-hour time interval (spatial heterogeneity $=$ regions based on circular region) Comparing the selective region with the most dissimilar and non-adjacent region.

Figure 10 shows the results of the model evaluation in different combinations of data sources and zones based on a one-kilometer radius. The data of one area is considered as training data, with data from the most similar non-neighboring area as test. Based on these results, the best performance of the TOP1NN method is obtained when combining the POI, TRA, WEA, and RDN data sources. In Fig. 10, the results are compared with the non-neighboring area data that has the most dissimilarity in terms of road structure, traffic, meteorology, and points of interest. It is evident that the performance and accuracy of the model are the weakest in TOP3REG mode and that the average accuracy of estimations and predictions has decreased.

6. Conclusion

This paper investigated more evaluation scenarios on the proposed model for measuring air pollution using less-expensive means (e.g., the POI, meteorological, road network, and traffic data) and without using expensive pollution sensors and facilities. It presented details of removing the heterogeneity of multiple data sources in the Multiple Data Set Aggregator & Heterogeneity Remover (MDA&HR) and improving the operation of the Train Data Splitter (TDS) part of the model by focusing on the finding more similar patterns of air quality. We evaluated the models for various viewpoints of dataset in new scenarios to show the generality of the models. In these new scenarios, one region is considered as train data and selected region is considered as the test. In the current evaluations, the zones are considered based on the range and radius of one kilometer of traffic sensing and air quality sensors in addition to previous viewpoints. Results confirm the generality of the model and consideration of the part of the model that tries to use the data with most similar pattern to the intended area for predictions. In future works an extension of the proposed framework can be applied for healthcare system that uses Fifth generation network infrastructures applied to smart cities environment that can enhance the applicability of the proposed framework [30, 31, 32, 33].

References

Nations

World’s population increasingly urban with more than half living in urban areas. 2014, United Nations New York, NY.

Innovation

M.o.t.E.

Available from: https://eu-smartcities.eu/about/european_context.

Zheng

, et al. Urban computing with taxicabs. in Proceedings of the 13th International Conference on Ubiquitous Computing. ACM. 2011.

Puustjärvi

, and Leena

The role of smart data in smart home: health monitoring case. Procedia Computer Science. 2015; 69: 143–151.

Honarvar

A.R.

, and Sami

Towards sustainable smart city by particulate matter prediction using urban big data, excluding expensive air pollution infrastructures. Big data research, 2018.

Fan

, and Bifet

Mining big data: Current status, and forecast to the future. ACM sIGKDD Explorations Newsletter. 2013; 14(2): 1–5.

Battista

, et al. Assessment of the air pollution level in the city of Rome (Italy). Sustainability. 2016; 8(9): 838.

Kumari

, and Jain

M.K.

A Critical Review on Air Quality Index, in Environmental Pollution. 2018, Springer. pp. 87–102.

RAQ – A Random Forest Approach for Predicting Air Quality in Urban Sensing Systems. Sensors. 2016; 16(1).

10.

Granger-Causality-based air quality estimation with spatio-temporal (ST) heterogeneous big data. in Computer Communications Workshops (INFOCOM WKSHPS). 2015.

11.

Satellite remote sensing of particulate matter and air quality assessment over global cities. Atmospheric Environment. 2006; 40(30).

12.

Demonstration abstract: PiMi air box: a cost-effective sensor for participatory indoor quality monitoring. in Proceedings of the 13th International Symposium on Information Processing in Sensor Networks. 2014.

13.

Prediction of PM2.5 based on Elman neural network with chaos theory. in Control Conference (CCC). 2016.

14.

Extracting Usage Patterns from Power Usage Data of Homes’ Appliances in Smart Home using Big Data Platform. International Journal of Information Technology and Web Engineering (IJITWE). 2016; 11(2).

15.

Improve Home Energy Management System by Extracting Usage Patterns from Power Usage Big Data of Homes’ Appliances, in Handbook of Research on Contemporary Perspectives on Web-Based Systems. IGI Global. 2018.

16.

An improved model for PM2.5 inference based on support vector machine. in Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing (SNPD). 2016.

17.

An accurate and low-cost PM2.5 estimation method based on Artificial Neural Network. in Design Automation Conference (ASP-DAC). 2015.

18.

Wang

, et al. Monitoring environmental quality by sniffing social media. Sustainability. 2017; 9(2): 85.

19.

Feng

, et al. Air quality estimation based on multi-source heterogeneous data from wireless sensor networks. in Wireless Communications and Networking Conference (WCNC), 2018 IEEE. IEEE. 2018.

20.

Feng

, et al. MCS-RF: Mobile crowdsensing-based air quality estimation with random forest. International Journal of Distributed Sensor Networks. 2018; 14(10): 1550147718804702.

21.

, et al. Inferring fine-grained PM2.5 with bayesian based kernel method for crowdsourcing system. in GLOBECOM 2017–2017 IEEE Global Communications Conference. IEEE. 2017.

22.

Ali

M.I.

Gao

, and Mileo

Citybench: A configurable benchmark to evaluate rsp engines using smart city datasets. in International Semantic Web Conference. Springer. 2015.

23.

Wunderground. Available from: https://www.wunderground.com.

24.

Map

Available from: https://developer.google.com/maps.

25.

Openstreetmap.

26.

Honarvar

A.R.

, and Sami

Multi-source dataset for urban computing in a Smart City. Data in Brief. 2019; 22: 222–226.

27.

Wang

, and Mahadevan

Heterogeneous domain adaptation using manifold alignment. in Twenty-Second International Joint Conference on Artificial Intelligence. 2011.

28.

Duan

, and Tsang

Learning with augmented features for heterogeneous domain adaptation. arXiv preprint arXiv:1206.4660. 2012.

29.

Yuan

Zheng

, and Xie

Discovering regions of different functions in a city using human mobility and POIs. in Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM. 2012.

30.

Tecer

L.H.

, et al. Effect of meteorological parameters on fine and coarse particulate matter mass concentration in a coal-mining area in Zonguldak, Turkey. Journal of the Air & Waste Management Association. 2008; 58(4): 543–552.

31.

Skondras

Michalas

, and Vergados

D.D.

Mobility management on 5g vehicular cloud computing systems. Vehicular Communications. 2019; 16: 15–44.

32.

Tsoulchas

Tsolis

Zoumi

Skondras

, and Vergados

D.D.

Health Monitoring of People with Diabetes using IoT and 5G Wireless Network Infrastructures. in 2020 11th International Conference on Information, Intelligence, Systems and Applications (IISA). IEEE. 2020, July; pp. 1–6.

33.

Tsoulchas, Vasileios, Nikolaos Tsolis, Eirini Zoumi, Emmanouil Skondras, and Dimitrios D. Vergados. Health Monitoring of People with Diabetes using IoT and 5G Wireless Network Infrastructures. in 2020 11th International Conference on Information, Intelligence, Systems and Applications (IISA). IEEE. 2020; pp. 1–6.

Particular matter prediction using synergy of multiple source urban big data in smart cities

Abstract

Keywords

1. Introduction

2. Related work

2.1 Alternative dataset in urban computing

2.2 Prediction of air pollution

3. DATA

Table 1 Data source features

4.1 Framework

4.3 Similarity computation based on temporal and spatial properties

5. Experiments and results

Table 2 Combining data sources

References

Table 1
Data source features

Table 2
Combining data sources