Abstract
Traffic congestion occurs when the number of the vehicles increases more than the existing space of the road. This deleterious problem is increasing at an alarming rate in the whole world. For any effective Intelligent Transportation System, early detection of traffic congestion is very important to take corrective action. Several techniques have been developed to detect traffic congestion, most of which are infrastructure based. Even though these techniques are widely used, but they have many downsides as well. They require large capital input for installation as well as for maintenance. In this paper, we propose an efficient and cost-effective method using smartphones to determine the traffic state of the road. The acoustic data collected from commuter’s smartphone is segmented into fixed size frames. Various time and frequency based features such as (MFCC, Delta & Delta-Delta, ZCR, STE, and RMS) are extracted from each frame and used for detecting traffic state as ’busy street’ or ’quiet street’. We have compared the accuracy of two classifiers Support Vector Machines and Neural Network by using acoustic data collected from 320 different recording sessions. Experiments have shown that feature set having features MFCC, STE and RMS, results in better classification accuracy of 91.8% with Neural Network and 93% with SVM. Furthermore, various relevant factors affecting the classification accuracy are also tested like frame size, window functions, overlapping size and different combination of features. The frame size of 8192 and hamming window function proved to be more efficient than others.
Keywords
Introduction
Traffic congestion is a problem which is increasing rapidly. As a result of this, the daily life of commuters is getting affected severely. In developing countries, roads are unable to cope up with the increased number of vehicles which leads to traffic jams at poorly planned road networks. These traffic jams ultimately end up in frustration, road rage and considerable monetary losses including fuel consumption and wastage of time. In addition, it has an adverse effect on environment (air, noise pollution). In recent years, instead of building infrastructure to manage traffic, focus has shifted towards better management of existing roads. An automatic traffic detection system can help us to devise policies for effective management of roads so as to reduce traffic congestion and run transportation smoothly and safely.
Several Intelligent Transportation System (ITS) solutions have been proposed for traffic monitoring and mitigation. These systems use different types of fixing techniques like inductive loop detectors, magnetic sensors and video cameras. Even though these technologies are implemented widely in various parts of the world but they still have certain disadvantages, such as, these are very expensive and require high maintenance and installation costs. Other than infrastructure based techniques, many infrastructureless techniques have also been used for monitoring traffic like GPS, GSM and accelerometers.
Sound has got great importance in the life of human beings through which they can sense and understand the world. Thus we believe that sound can also be used significantly in machine automated monitoring in various applications. Moreover, acoustic (sound) sensors like microphones are cost effective as compared to other infrastructured techniques, have low power requirements and the ability to work 24/7.
Acoustic signals can thus prove to be efficient in monitoring the traffic state of the road, since various types of sounds can be acquired from the road like vehicular tire noise, engine noise, honking sounds, etc. The cumulative of these sounds can be used to efficiently monitor traffic state of road. Many researches have been done in which acoustic sensors (microphones) were installed on roadside. As an alternative, we can use smartphones to capture acoustic signals which can further help in crowdsourcing. Since smartphones are used by large number of people in the whole world, they can be easily used for the purpose of traffic monitoring in real time.
The main objective of our research is to classify different traffic states. For this we collect acoustic data using smartphones and extract multiple features from the data collected. In the proposed approach, frame-wise classification is done so that even the small isolated events are detected. Various factors which help in extracting features like different frame sizes, type of window function and overlapping size are analysed and their effect on the system in terms of computational power and classification accuracy is verified. Neural Network is used for classification of the two traffic states of the road (busy and quiet) and its comparison with SVM based model has been done.
Main contributions of this paper are: Feature set analysis of smartphone based acoustic recording for traffic state determination. Effect of frame size and windowing function used for segmenting the acoustic data. Comparison of accuracy of SVM and Neural Network based classifier.
The rest of the paper is organised as follows: in Section 2, related work done in traffic detection using acoustics sensors is discussed. In Section 3, we describe the methodology followed and the features and classifiers used. Section 4 provides a description of the experimental setup and the classification results achieved and in Section 5 conclusion and future work is presented.
Related work
Majority of Intelligent Transportation Systems are designed for optimal use of road capacity with minimum transportation delay. The problem of detecting road traffic density is being solved by various techniques like inductive loop detectors [1], magnetic sensors [2], video cameras [3], microwave radar [4], etc. Most widely used sensors are magnetic sensors which detect traffic congestion by detecting vehicular speed and its length. But they are implemented where lane system is present and assume traffic to be homogeneous [2] and orderly, which makes it inefficient in chaotic traffic conditions. On the other hand vision based sensors like video cameras can provide images of several lanes. Large amount of research has been done using this technique for traffic monitoring and surveillance by installing the cameras at the roadsides [5, 6] and further detecting activities like accidents and reckless driving. But the installation and equipment cost of video cameras is relatively high and they fail to perform efficiently under various weather conditions like rain, storm, etc. Further, processing of videos for classification is computationally intensive and time consuming.
The problems related to infrastructure based techniques has shifted the focus towards probe-based solutions which include the use of sensors which are deployed on vehicles or smartphones carried by commuters. Sensors like GPS, GSM and accelerometer are some of the probe-based sensors. The usage of these sensors present in smartphone were used in developing an energy efficient approach for detecting traffic congestion by Mohan et al. [7]. Another smartphone based sensor, barometer was used by Dimri et al. [8] for detecting traffic congestion. Using barometer, the altitude changes are detected, which are more on the free flowing road as compared to congested road, which further helps in detecting the traffic state of the road.
However, acoustic signals can also be used for efficient monitoring of traffic. Acoustic signals can be acquired by installing microphones at various places on roadside. These microphones are cheap, do not disturb traffic while installation and maintenance and can be easily embedded at any place. Several researches have been done to detect various events related to roads like accidents [9], vehicle detection [10], traffic congestion [11], etc. Chen et al. [12] has proposed a solution in which 12 microphones in the form of 4 arrays were installed on the road side. They further used sound maps for the detection of sounds from vehicles. Although this approach gave high accuracy but it did not perform well in case of highly jammed traffic conditions where vehicles were hardly moving. Two stereo microphones were installed alongside the road in the approach developed by Kato et al. [13]. But they considered only two lane traffic and neglected the sound of idling cars and weather conditions like wind and rain. Further, work done in [7] was augmented by Sen et al. [14], which included an additional aspect of classifying congested and free-flowing roads. Further, a pair of roadside audio sensors were used in this approach, separated by a distance. Congestion was detected based on the speed of the vehicle by using threshold based classification technique. The practical applicability of this work was done in [15] through prototype development. Road-side deployment of the recorders was done to acquire data from 6 different roads and this data was transferred through GPRS to the server.
spaceskip =.18em plus.1em minus.1emAcoustic data can also be used for the surveillance of urban traffic. Ntalampiras [16] detected and identified vehicle sounds and crash sounds by using a time, frequency and wavelet domain feature combination. Universal based HMM classification technique improved the results of class specific HMM. A small frame of 30 ms was considered with shift of 10 ms. Further, Tyagi et al. [11] proposed vehicular traffic density estimation approach using acoustic signal which is cumulative of tire noise, engine noise, etc. of multiple vehicles. With the help of these acoustic signals, the traffic density was classified into one of the following states: jammed (0–10 kmph), medium flow (10–40 kmph) and free flow (40 and above) using GMM-Bayes and SVM classifier. They also analysed early detection time for traffic signals and the results were best when 30 seconds signal span was considered.
In our research we have used smartphones to record acoustics for detecting the traffic state of the road and classify into busy and quiet street. Our objective is to investigate different time-frequency based features and efficacy of different classifiers such as SVM or Neural Network. We propose frame-wise classification for early detection of traffic state. We have tested different classifiers for different feature sets, frame lengths, window function and overlapping size. Section 4 presents detailed description of the experiments defined above.
Methodology
In typical traffic scenes, different acoustic signals are generated by different vehicles. In busy street or congested street emphasis is laid on various sounds like engine idling sound, honking sounds, etc. However, in quiet street emphasis is laid on air turbulence sound and other environmental sounds. These sounds are collected in the acoustic signals and can be represented in the form of features which are further classified into busy street or quiet street. The features collected are divided into training and testing where 2/3 of the data is used for training and 1/3 for testing. The training dataset builds up the model which then classifies the testing data accordingly.
The full overview of the process is shown in Fig. 1. The first step in the system is pre-processing in which the acoustic signals are normalized and divided into frames from which features are extracted using feature extraction techniques. These feature vectors are then used for building a model and then classified into any of the two classes, busy and quiet.
Pre-processing
As the acoustic signals of traffic scenario carry noise, so smoothening of signal is done by using median filter so as to lessen the effect of noise. Further, to avoid amplitude variation during capturing the acoustic signal, the amplitude is normalized. Normalization of raw input audio signal is done by first determining the maximum amplitude value and then dividing the values of the whole signal by it. This helps in making the system robust to loudness variations. Equation used for normalization is described below.
Further, the continuous acoustic signal is divided into small frames, so that each frame is small enough to be considered as stationary and thus can be processed easily. The size of the frame depends upon the type of application it is being used in.
We extracted both time based and frequency based features. For frequency based features we extracted Mel Frequency Cepstral Coefficients (MFCC) and its Delta and Delta-Delta coefficients. In temporal i.e time-based features, we extracted Zero Crossing rate, Root Mean Square and Short Time Energy.
Mel Frequency Cepstral Coefficients (MFCCs) features are widely used in various applications like automatic speech and speaker recognition. However, they have evolved as one of the efficient features for acoustic applications. MFCC represents short-term power spectrum of a sound, which are computed by windowing the signal into short frames and applying Fourier transform (FFT in this case). Further, mel-filter bank is applied which maps the powers of the spectrum onto the mel scale by using 23 mel filter banks. Then the logarithm of these filterbank energies is taken which emphasize the low varying frequency characteristics of the signal, followed by discrete cosine transform of the resulting energies. The equation for this is as described below:
Usually acoustic signals having sound of traffic are of low frequency, so we focus on first 13 coefficients and discard the rest.
For a single frame, MFCC describes only the power spectral envelope but delta and delta-delta are the dynamic features which describe the trajectories of the MFCC features over time. For 13 MFCC features, 13 delta and 13 delta-delta features are calculated.
Further some temporal features are also considered: Root mean square (RMS) – The RMS value is a measure of energy in a signal. The RMS value is however defined to be the square root of the arithmetic mean of a squared signal, as described in the equation.
Zero crossing rate (ZCR) – It is defined as the number of sign-changes in a time-domain signal within a frame. It also illustrates the frequency of signal amplitude sign change.
Short-time energy (STE) – STE is used for detection of loudness and silence of the audio signal as it conveniently represents the amplitude variation over time.
If one of the features has a broad range of values, it will dominate the other features. To avoid such problem we should normalise our data so that each feature value lies in a particular range. We have used Z-score method for normalization. In this method, the mean and standard deviation of the feature vector are computed and then normalized value isevaluated.
The aforementioned features can either be used alone or together in some combination. However, combination of all the features is not useful as the size of feature vector increases, which leads to more computations. This will not be helpful when we are using a smartphone. In Section 4.1.3, we perform experiments to choose a feature set which will give us not highest but acceptable classification accuracy and without consuming much computationpower.
Features extracted from the input signal are subjected to the classifier, which further helps in determining the class to which they belong. The training data is used to create a model on the basis of which classification is done. Five fold cross-validation is performed so that all the recordings are included in the test data atleast once, in order to get unbiased results. No change in results was observed by increasing the cross-validation, thus we keep it to five fold cross-validation. Two classifiers, SVM and neural Network are used and their classification accuracy is compared.
The Support Vector Machines are the supervised learning models that examine the training and testing data. It is a non-probabilistic binary linear classifier as its training algorithm builds a model which assigns one or the other category to the given data. The training example set is given as an input to the classifier, where each frame is initially assigned to the class it belongs, out of the two classes. Different kernels for SVM are experimented and Radial Basis function performed better over others. The values of cost and gamma function is experimented and is set to a value where training error would be minimum. The training model so formed can be represented as the points in space, such that examples of different categories are separated by an evident gap. The testing examples are then predicted to fall on either side of the gap depending upon the class to which they belong, which may be either ’busy street’ or ’quiet street’ in our work.
Neural Networks are distributed models. In Neural Network, each feature vector is designated as input. A network of input, output and hidden layer(s) of neurons is set up and then this network is trained. The purpose of training is to iteratively minimize the error between the desired output and estimated output. The training method used is scaled conjugate gradient backpropagation, which is set by testing other training functions. The overall input-output response is based on connection weights and biases which are assigned to the network. The input layer of the neural network is given the feature vectors and output layer has two neurons corresponding to two output classes i.e. busy street and quiet street. The number of hidden layers and neurons are also tested. The network performed better when one hidden layer with number of neurons equal to number of elements in the feature vector are taken.
In both the classifiers, frame wise classification is done so that every small event is detected and classified up to maximum possible accuracy. This means every frame is classified into one of the classes (busy or quiet).
Experimental setup and results
The feature extraction depends upon the size of the frame and the type of window function used. Moreover, we are performing frame-wise classification i.e. recognizing every frame as busy or quiet. Because of this we need to set the size of the frame and the window function which is applied over that frame optimally. For this, we experimented with various frame-sizes and window functions. As we are using smartphones, so computation power is an important aspect. Therefore, features used should be such that their computation power is low and accuracy achieved is high. Experiments are performed for choosing the appropriate feature set. Comparison of SVM and Neural Network is alsodone.
We have collected about 320 recordings, each of 30 seconds from roadside using microphones present in different smartphones like Nexus, Samsung S3, Samsung S4 and Samsung A3 using an android application. This data was collected over different sessions and also from different roads. The audios collected are all in wav format in order to remove artefacts. Also, they are all mono channel and collected over 16 KHz sampling rate. The labelling of data is done by manual perception. Moreover, the recordings are taken such that they do not contain any speech and music. The collected data is divided into two broad categories, busy street and quiet street. The data is divided equally among the two classes (busy street and quiet street) in both training and testing datasets. Various experiments are performed on the dataset by dividing it into 70% for training and 30% fortesting.
Results
Effect of frame size variation
In our first experiment we have tested various frame sizes that should be used to divide the signal into frames. As acoustic signal is a continuous signal, so it is divided into frames because these frames are short enough to be considered as pseudo-stationary. The amount of information that is represented by features depends on the frame size. To find out the appropriate frame size, different frame sizes (1024, 2048, 4096 and 8192) are tested by extracting MFCC features from the frames. These frames are further classified using a classifier. Figure 2 illustrates the classification accuracy of different frame sizes with MFCC features. It is observed that by using MFCC features, we get higher accuracy (71.8%) at frame size of 8192. This is because larger frame size means that a single frame contains more classifiable feature information. Frame sizes larger than 8192 are not efficient since they may capture multiple events. Thus, half second (8192) frame size is considered asoptimal.
Comparison of window functions
In this experiment, we varied the type of window functions and verified which window function is appropriate for our system. As we have seen that changing the frame size has different effect on the audio signal, similarly different types of window function have different effects. During the process of feature extraction, window encompass the frame that is to be considered. This frame size may vary depending on the type of problem (as shown in Section 4.1.1). Types of window functions include some simpler ones like rectangular, triangular windows and some complex ones like Hamming, Hanning, etc. The window functions that we have experimented are: Hamming, Bartlett, Hanning and Blackman. Figure 3 illustrates the classification accuracy of different window functions. It is observed that Hamming window gives higher accuracy (71.8%) than other window functions. This is because, hamming window function does a better job of cancelling the effect of side lobes of the sinusoidalsignal.
Comparison of different feature sets
The time and frequency based features are experimented in this section so as to verify which combination of features perform better. As described in Section 3.2, each feature extraction technique represents different type of information about the acoustic signal. Various types of features can be used but choosing appropriate features is important. Features should be chosen such that they give high accuracy and consume low computation power. Also, since frame size of 8192 and Hamming window function have given higher accuracy, so all features are extracted by using the frame size of 8192 and Hamming window function. All features (MFCC, ZCR, RMS, STE and Delta and Delta-Delta) are combined into different feature sets and compared. Figure 4 illustrates the accuracy of different feature sets with SVM and Neural Network. It is observed that feature set: MFCC+STE+RMS gives highest accuracy for both SVM and Neural Network.
It can be seen that SVM performs little better that Neural Network. Classification accuracy of SVM is 93% whereas Neural Network gives 91.8% accuracy.
While extracting features, the overlapping size was considered to be 50%. Overlapping ensures that features which occur at the discontinuity are included in the overlapped frame. Larger the overlapping size (smaller shift size), more computation it takes as more number of frames are generated. But overlapping is more important in speech where the lost data is very important. However, unlike speech, the acoustic signals of traffic still gives good accuracy when the overlapping is not considered. This is proved in the following experiment in which the shift sizes are varied (60%, 70%, 80%, 90% and full frame). Figure 5 illustrates the classification accuracy by varying the shift size. It can be seen that larger shift size (smaller overlapping size) does not have much effect on the accuracy of the system. However, by considering smaller overlapping size, the computational expense will be reduced manifold.
Conclusion
We have proposed an efficient technique to classify the traffic state of the road into busy or quiet street using time and frequency based feature set. Frame-wise classification was done to detect smallest event possible with high accuracy. All possible relevant factors like frame size, window function and shift size are considered such that high classification accuracy is achieved. Experiments were conducted to verify various feature sets and results have shown that the accuracy of MFCC, STE and RMS combination is best of all. Also a large frame size (8192) and hamming window function has proved to be better than others. We implemented and compared two frameworks, SVM and Neural Network for classification. It has also been shown that by increasing the shift size, the accuracy remains almost the same, but the computational expense of the system decreases. We envisaged that accuracy can be further enhanced using crowd sourcing of data collected from smartphones of various commuters present on particular road intersection.
Footnotes
Acknowledgments
The work has been supported by Design Innovation Center, Panjab University, Chandigarh and undertaken as a part of the sponsored project “CARTS- Communication Assisted Road Transportation System” supported by ITRA, Media Lab Asia.