Performance analysis of K-nearest neighbor,support vector machine,and artificial neural network classifiers for driver drowsiness detection with different road geometries

Abstract

This article comparatively analyzed the performance of K-nearest neighbor, support vector machine, and artificial neural network classifiers for driver drowsiness detection with different road geometries (straight segments and curve segments) based on a driving simulator. First, vehicle performance measures (speed, acceleration, brake pedal, gas pedal, steering angle, and lateral position) were collected through sensors. These measures were analyzed, and their correlation with drowsiness on different road segments was examined. The analysis was based on data obtained from a study that involved 22 subjects in the driving simulator located in the Traffic Research Center, Beijing University of Technology. Second, six classifiers were constructed for six curve segments, respectively, while only one classifier was constructed for all straight segments because the waveforms by subtracting the road curvature from the steering angle in the curve segments were different from the waveforms of the straight segments. Furthermore, the less the radius of curvature, the more the difference. Third, the performance of K-nearest neighbor, support vector machine, and artificial neural network classifiers were compared and evaluated. The experimental results illustrate that the support vector machine classifier achieved the fastest classification time and the highest accuracy (80.84%). Support vector machine and artificial neural network are effective classification methods for detecting drowsy driving on different road segments.

Keywords

Sensors driver drowsiness detection road geometries pattern classification performance comparison

Introduction

Driver fatigue or drowsiness is one of the major causes of traffic crashes. Fatigue is a condition characterized by a reduced capacity that can be physical or mental. Fatigue can be induced by different external factors, while drowsiness is a natural state with the need to sleep. Accordingly, fatigue detection may be more difficult due to the complexity and variety of its characteristic.¹ Fatigue is usually accompanied by drowsiness. Brown’s² study showed that poor sleep or lack of sleep is one cause of driver fatigue. Fatigue and drowsiness are often used interchangeably.³ The National Highway Traffic Safety Administration (NHTSA) estimated that about 100,000 crashes were caused by fatigue or drowsy driving each year in the United States. The crashes resulted in more than 71,000 injuries and 1500 deaths.⁴ The study of Klauer et al.⁵ showed that drowsy driving increases the crash risk four to six times. The study showed that at least 15%–20% of all crashes are related to fatigue and drowsiness.⁶ The ability of a driver to operate a vehicle degrades substantially under the fatigue or drowsiness condition. Therefore, developing a model, algorithm, or designing a system, which can detect fatigue or drowsy driving, is important for the prevention of road crashes. In the past few years, many approaches have been proposed for fatigue or drowsiness detection. Generally, the approaches are divided into the following three categories:^7–9

Physiological features–based method. The methods are based on physiological signals, such as brain waves, skin electric potential, heart rate, and pulse rate. Lal et al.¹⁰ used electroencephalogram (EEG) to develop a fatigue countermeasure algorithm. Physiological features–based methods are regarded as the most accurate in fatigue detection.¹¹ Zhang et al.¹² proposed a multi-features-based fatigue detection method. The measures included EEG, electromyogram, and electrooculogram signals. Wang et al.¹³ proposed an EEG-based method to detect driving fatigue using the correlation dimension. However, the measures are intrusive. The electrodes are attached to the body of the subjects, which make the driver feel inconvenient and annoying.⁹

Visual features–based method. The visual features come from the driver’s head activity, eyes activity, facial expression, body sagging posture, and so on. Jo et al.¹⁴ proposed a vision-based method that combined appearance features with statistical features. Wang et al.¹⁵ detected the driver fatigue by monitoring his or her mouth movement. Azim et al.¹⁶ proposed a video analysis–based fatigue detection system. Compared to physiological features, these approaches are non-intrusive. The methods generally use image processing to analyze the driver’s state. However, the image can be affected by lighting conditions or other factors, which may decrease the detection accuracy.¹⁷

Driving behavior–based method. The method detects driving fatigue by analyzing vehicle speed, acceleration, steering angle, lateral displacement, brake pedal, and so on. The method is also non-intrusive. Compared to visual features, the feature acquisition of driving behavior is quite easy. And it cannot be affected by the light condition.¹⁸ For safety reasons, these features are generally collected using a driving simulator. Krajewski et al.¹⁹ detected drowsy driving by analysis of steering wheel behavior in the driving simulation lab. Eskandarian and Mortazavi²⁰ collected steering wheel signal in a truck driving simulation environment and used a signal processing scheme on steering wheel signal to detect the drowsiness. Sayed and Eskandarian²¹ detected the drowsiness using vehicle parameter data in a driving simulator. King et al.²² used a test vehicle to collect the data and proposed a simple algorithm based on the steering wheel movement, which demonstrated the potential for fatigue detection by driving behavior.

Each approach requires, first, finding the correlation between fatigue/drowsiness and the corresponding features, regardless of whether they are the vehicle-related or driver-related features obtained through the sensors. The existing researches have shown that the correlations do exist.¹ Second, a classification method should be proposed based on the features. Generally, the classification approaches are divided into the following two categories. The first is to directly input the features into a predefined model to detect the fatigue/drowsiness. The percentage of eyelid closure over the pupil, over time (ePERCLOS) proposed by Wirewille and Ellsworth²³ is a typical example. The second is to use pattern classification technique to detect the fatigue/drowsiness. K-nearest neighbor (KNN), artificial neural networks (ANN), support vector machine (SVM), and Bayesian networks (BN) are very popular techniques for the classification. Yeo et al.²⁴ used SVM to classify EEG into four frequency bands and then detected the drowsiness. Hu and Zheng²⁵ proposed a method of detecting drowsiness using SVM to analyze 11 eyelid-related features extracted from Electro-Oculogram. Shen et al.²⁶ showed that the accuracy of the probabilistic-based multiclass SVM classifier was better through comparing a probabilistic-based and a standard multiclass SVM classifier. JH Yang et al.²⁷ developed drowsy-driver detection systems using the paradigm of BN. YL Liang et al.²⁸ used SVM to propose a real-time method of detecting the distraction.

The above researches show that the techniques, ANN and SVM, are effective in detecting driver fatigue or drowsiness. As we all know, accuracy and real time are two important indicators for fatigue or drowsy driving detection. Therefore, performance analysis of KNN, ANN, and SVM classifiers is very necessary for fatigue or drowsy driving detection. In addition, although there have been numerous studies about detecting driver fatigue or drowsiness, few studies considered road geometry and curvature in drowsiness detection. Sayed and Eskandarian²¹ removed the effect of curvature from steering wheel signal to detect the drowsiness. It is deduced that the effect of curvature on driver drowsiness detection may vary in different road geometries. Accordingly, this article focuses primarily on driver drowsiness detection with different road geometries (straight segments and curve segments) and performance comparison of KNN, SVM, and ANN classifiers. First, vehicle performance measures (speed, acceleration, brake pedal, gas pedal, steering angle, and lateral position) were analyzed and evaluated to examine their correlation with drowsiness on different road segment types. Second, the classifiers were constructed for each curve segment, while only one classifier was constructed for all straight segments because the waveforms by subtracting the road curvature from the steering angle in the curve segments were different from the waveforms of the straight segments. Furthermore, the less the radius of curvature, the more the difference. Third, the performance of KNN, SVM, and ANN classifiers was compared and evaluated.

The rest of the article is organized as follows. Section “Experiment and data collection” presents the experimental design, which includes participants, apparatus, scenario, and experimental process. Section “Data analysis and feature extraction” presents the data analysis and feature extraction. Next, section “Driver drowsiness detection” presents the driver drowsiness detection using KNN, SVM, and ANN classifiers and performance comparison of the classifiers. Section “Conclusion” summarizes the conclusions and proposes future work.

Experiment and data collection

The alert and drowsy driving experiments were conducted at the Car Simulator Laboratory located in the Traffic Research Center, Beijing University of Technology. During the experiments, drivers drove a car simulator under alert and drowsy states. Data were recorded through the sensors.

Participants

A total of 25 subject drivers were recruited to drive the car simulator. Driver age ranged from 20 to 35 years with an average age of 25 years. All participants were licensed drivers. Drugs and alcoholic drinks were banned during the experiments. All of the subjects received no less than 400 RMB as payment at the end of the experiments.

Apparatus

The experiments were based on Autosim simulator system located in the Transportation Research Center, Beijing University of Technology. The body of the simulator is technologically remolded from a Toyota vehicle, and the steering wheel, brake, parking brake, and safety belt are identical to a real vehicle. Sensors were used to collect data. The data sample frequency is 30 Hz. Three software tools were used for the simulation experiment. The first is Evariste, which is used to create experimental scenarios. The second tool, Simword, is used to control the scenarios. The third is Scancer, which is used to record the data. Eye movement data were recorded by a head-mounted eye tracking system.

Scenario

The driving scenario consists of a 10-km urban road, which is composed of straight and curve segments, as shown in Figure 1. The radii of curvature for the three left-turn curves are 200, 500, and 800 m, and the radii of curvature for the three right-turn curves are 200, 500, and 800 m. Let L200 denote the left-turn curve with the radius of curvature of 200 m. Similar definitions also apply to R200, L500, R500, L800, and R800. Let S denote the straight segments.

Figure 1.

The scenario.

Experimental protocol

The protocol was designed for driver drowsiness detection in a simulator. During the experiments, drivers drove the simulator under alert and drowsy states, respectively. Each driver was required to complete two driving tasks, a morning task (8:30–11:30 am) and a night task (1:00–5:00 am). A participant practiced the operation of the simulator before performing the tasks. The experimental tasks were performed when the participant became familiar with the operation. For the morning task, drivers were required to have at least 8 h of sleep before their testing date. Subjects were picked up to the Car Simulator Laboratory 1 h before the start of the task. For the night task, drivers were required to keep awake for about 17 h before the experiments began. They arrived at the lab at 1:00 am. Drivers were required to drive for about 1 h on a circular highway ahead of the actual test to insure they were drowsy. After completing the driving, drivers were required to fill out a questionnaire regarding their evaluations of the simulator and subjective feelings of the experiments. And then, the actual test was immediately conducted. After completing the task, drivers were required to fill out another questionnaire.

In order to assess the effectiveness of drowsiness detection, it is essential that the ground truth is labeled. In this work, the drowsiness of the tested drivers was validated in three ways: the sleep–wake cycle, video recording, and the questionnaire.

Data collection

The vehicle parameter data included speed, acceleration, steering angle, brake pedal, gas pedal, and lateral position. These data were recorded at a frequency of 30 Hz from sensors through simulation software.

Data analysis and feature extraction

A statistical analysis and significance testing of the data collected during the experiment were conducted. Six variables, speed, acceleration, brake pedal, gas pedal, steering angle, and lateral position, were studied in this article. First, each variable was studied by analyzing the mean and standard deviation. The t-test was used to determine whether there was a significant difference between the variables during alert driving and drowsy driving. The results indicate that lateral position and steering angle are correlated with drowsiness, which corresponds with the previous research.^1,29–31

While driving, drivers make two categories of steering adjustments to drive a vehicle:¹

Micro-corrections. These adjustments are for lane-keeping. Thus, the adjustments are small-amplitude oscillations in steering angle.

Macro-corrections. These adjustments are to change lanes or negotiate a curve. Thus, the adjustments are large-amplitude steering adjustments.

The scenario used in the experiments included both straight and curve segments. In the straight segments, the steering angle waveforms consist only of the small-amplitude adjustments for lane-keeping, while in the curve segments, the waveforms contain both road curvature and the component of steering adjustments to keep the vehicle in the center of the lane.²⁹ Eskandarian et al.²⁹ proposed that the effect of road curvature may be removed from the steering signal by subtracting the trend from the original steering wheel angle signal. The steering angle signals collected during the experiments were analyzed and tested using the method proposed by Sayed and Eskandarian. However, the results were unsatisfactory. Steering angle data of subjects 12 and 18 are taken as the examples to illustrate. Steering angle data of subjects 12 and 18 before and after road curvature removal are shown in Figures 2 and 3, respectively. Figures 2(a) and 3(a) show the steering angle data of subjects 12 and 18 who drove the car simulator on the whole road. Figure 2(b)–(d) shows the enlarged partial details of Figure 2(a). Figure 3(b)–(d) shows the enlarged partial details of Figure 3(a). From Figures 2(b) and (d) and 3(b)–(d), we can find that the waveforms by subtracting the road curvature from the steering angle in the curve sections are different from the waveforms of the straight sections. Generally, the less the radius of curvature, the more the difference. Therefore, one classifier is not enough to classify drowsiness for different road geometries. Considering these different waveforms, we constructed different classifiers for different road segments in the next section.

Figure 2.

Steering angle data of subject 12 before and after road curvature removal.

Figure 3.

Steering angle data of subject 18 before and after road curvature removal.

Driver drowsiness detection

As shown in section “Data analysis and feature extraction,” the features, lateral position and steering angle, have been extracted. Classification can now be performed. The goal of drowsiness detection is to classify the driver’s state into two different possible states/classes: alert or drowsy.

Drowsiness detection using three classifiers

Our experiments were conducted on a Windows 7 machine with a 2.50-GHz CPU and 4.0-GB RAM. In total, 22 drivers completed the actual experiment and 44 samples were obtained. Three kinds of classifiers were used: KNN, SVM, and ANN. The 10-fold cross-validation was used to evaluate the performance of the classifiers. Lateral position and steering angle during the first 13 s when the subject entered the different road segments were selected as input feature vector for classification. It means that the number of input feature vector is 780 (two features, the data collected during 13 s and the collection frequency of 30 Hz). The features were normalized and labeled as either “alert” or “drowsy” before proceeding to build a classifier.

Drowsiness detection using KNN classifier

KNN does not have the training process. The block diagram of drowsiness detection using KNN classifiers is given in Figure 4. We used Euclidean distance as distance metric. Since the number k of nearest neighbor can influence the accuracy of the overall classification, we tested with six different k values: 1, 3, 5, 7, 13, and 19. As described in section “Data analysis and feature extraction,” the classifiers were constructed for each curve segment, respectively, while only one classifier was constructed for all straight segments. Namely, there were seven KNN classifiers, among which six classifiers were constructed for six curve segments, respectively, and one classifier was constructed for all straight segments.

Figure 4.

Block diagram of drowsiness detection.

Drowsiness detection using SVM classifier

Unlike KNN, SVM has both training and testing processes. The block diagram of drowsiness detection using SVM classifiers is also given in Figure 4. In these experiments, radial basis function (RBF) kernel was employed. Accordingly, there are two parameters, the RBF kernel parameter σ and the cost parameter C, to be set. In our experiments, we tested with six different σ values: 0.1, 10, 15, 50, 90, and 200 and C was set equal to 1. Similarly, there were seven SVM classifiers, among which six classifiers were constructed for six curve segments, respectively, and one classifier was constructed for all straight segments. The classifiers were tested by the 10-fold cross-validation.

Drowsiness detection using ANN classifier

Like SVM, ANN has both training and testing processes. The block diagram of drowsiness detection using ANN classifiers is given in Figure 4. In this article, a two-layer feedforward neural network was selected. Number of neurons in the input layer is 780. Number of neurons in the output layer is 1. Number of neurons in the hidden layer is not fixed. We tested with five different numbers of neurons in the hidden layer: 10, 20, 30, 50, and 100. The sigmoid was used for the hidden layer, and the linear was used for the output layer. Gradient descent backpropagation method with momentum and adaptive learning rate was used for neural network training.³² The network was trained by gradient descent backpropagation method with momentum and adaptive learning rate, with an initial learning rate of 0.01. The training was terminated when the total sum of squared errors became less than 0.0001. Similarly, there were seven ANN classifiers. The classifiers were also tested by the 10-fold cross-validation.

Classification results

Classifier performance measures

Let us consider a drowsiness detection that seeks to determine whether a driver is drowsy. Detection outcomes are labeled either as drowsy (positive) or alert (negative). Classifier performance was evaluated with three different measures: true positive rate (TPR), false positive rate (FPR), and accuracy (ACC).

Classification result analysis

KNN classification results

TPR, FPR, and ACC of the classifiers were calculated. Table 1 presents the performance of KNN classifier when different values of k are considered. From Table 1, we observe that the best value for the k is 3. It seems that k values between 1 and 5 are suited for drowsiness detection tasks in the experiment. KNN is very simple to understand and easy to implement. However, the classifiers show low accuracy. Table 2 shows the total execution time in seconds.

Table 1.

Performance measures of KNN classifier.

Different road segments	Performance measures	KNN classifierNo. of neighbors considered
		1	3	5	7	13	19
L200	TPR	86.36%	86.36%	90.91%	86.36%	90.91%	100%
	FPR	59.09%	45.45%	54.55%	54.55%	68.18%	86.36%
	ACC	63.64%	70.45%	68.18%	65.91%	61.36%	56.82%
R200	TPR	77.27%	72.73%	77.27%	72.73%	81.82%	81.82%
	FPR	9.09%	22.73%	27.27%	13.64%	22.73%	22.73%
	ACC	84.09%	75.00%	75.00%	79.55%	79.55%	79.55%
L500	TPR	77.27%	86.36%	90.91%	100%	95.45%	100%
	FPR	31.82%	36.36%	59.09%	72.73%	81.82%	81.82%
	ACC	72.73%	75.00%	65.91%	63.64%	56.82%	59.09%
R500	TPR	72.73%	81.82%	90.91%	86.36%	81.82%	77.27%
	FPR	50.00%	50.00%	59.09%	54.55%	68.18%	54.55%
	ACC	61.36%	65.91%	65.91%	65.91%	56.82%	61.36%
L800	TPR	68.18%	86.36%	100%	100%	95.45%	100%
	FPR	50.00%	68.18%	86.36%	90.91%	86.36%	90.91%
	ACC	59.09%	59.09%	56.82%	54.55%	54.55%	54.55%
R800	TPR	81.82%	86.36%	95.45%	90.91%	95.45%	100%
	FPR	31.82%	31.82%	40.91%	40.91%	54.55%	50.00%
	ACC	75.00%	77.27%	77.27%	75.00%	70.45%	75.00%
S	TPR	86.36%	90.91%	90.91%	96.97%	93.94%	95.45%
	FPR	22.73%	25.76%	25.76%	36.36%	40.91%	43.94%
	ACC	81.82%	82.58%	82.58%	80.30%	76.51%	75.76%
Average	TPR	78.57%	84.41%	90.91%	90.48%	90.69%	93.51%
	FPR	36.36%	40.04%	50.43%	51.95%	60.39%	61.47%
	ACC	71.10%	72.19%	70.24%	69.27%	65.15%	66.02%

KNN: K-nearest neighbor; TPR: true positive rate; FPR: false positive rate; ACC: accuracy.

Table 2.

Execution time of KNN classifier in seconds.

Different road segments	KNN classifierNo. of neighbors considered
	1	3	5	7	13	19
L200	0.0064	0.0065	0.0062	0.0067	0.0060	0.0072
R200	0.0069	0.0062	0.0071	0.0069	0.0064	0.0062
L500	0.0061	0.0059	0.0060	0.0064	0.0067	0.0073
R500	0.0065	0.0058	0.0055	0.0062	0.0054	0.0063
L800	0.0066	0.0061	0.0059	0.0065	0.0063	0.0064
R800	0.0058	0.0061	0.0071	0.0072	0.0066	0.0062
S	0.0066	0.0064	0.0066	0.0056	0.0055	0.0056
Average	0.0064	0.0061	0.0063	0.0065	0.0061	0.0065

KNN: K-nearest neighbor.

SVM classification results

TPR, FPR, and ACC of the classifiers were calculated. Table 3 presents the performance of SVM classifier when different parameters of σ are considered. Table 4 shows the training time and classification time in seconds. From Table 3, we observe that the best value for σ is 15. It seems that σ values between 10 and 50 are well suited for drowsiness detection tasks in the experiments.

Table 3.

Performance measures of SVM classifier.

Different road segments	Performance measures	SVM classifierRBF kernel parameter σ
		0.1	10	15	50	90	200
L200	TPR	27.27%	72.73%	77.27%	72.73%	81.82%	81.82%
	FPR	31.82%	18.18%	22.73%	22.72%	36.36%	45.45%
	ACC	47.73%	77.27%	77.27%	75.00 %	72.73%	68.18%
R200	TPR	27.27%	77.27%	72.73%	81.82 %	86.36%	81.82%
	FPR	31.82%	9.09%	9.09 %	18.18 %	22.73%	18.18%
	ACC	47.73%	84.09%	81.82%	81.82 %	81.82%	81.82%
L500	TPR	68.18%	81.82%	86.36%	90.91%	95.45%	81.82%
	FPR	72.73%	13.64%	9.09%	22.73 %	36.36%	40.91%
	ACC	47.73%	84.09%	88.64%	84.09 %	79.55%	70.45%
R500	TPR	59.09%	63.64%	68.18%	77.27%	63.64%	68.18%
	FPR	59.09%	18.18%	27.27%	40.91%	31.82%	36.36%
	ACC	50%	72.73%	70.45%	68.18%	65.91%	65.91%
L800	TPR	45.45%	81.82%	90.91%	90.91%	81.82%	77.27%
	FPR	54.55%	22.73%	22.73%	45.45 %	45.45%	50%
	ACC	45.45%	79.55%	84.09%	72.73%	68.18%	63.64%
R800	TPR	31.82%	63.64%	77.27%	81.82%	81.82%	86.36%
	FPR	31.82%	13.64%	18.18%	27.27%	31.82%	31.82%
	ACC	50%	75%	79.55%	77.27%	75%	77.27%
S	TPR	43.94 %	65.15 %	81.82%	84.85 %	78.79%	75.76 %
	FPR	48.49 %	9.09 %	13.64%	19.70%	22.73 %	19.70 %
	ACC	47.73 %	78.03 %	84.09%	82.58%	78.03%	78.03%
Average	TPR	43.29%	72.30%	79.22%	82.90%	81.39%	79.00%
	FPR	47.19%	14.94%	17.53%	28.14%	32.47%	34.63%
	ACC	48.05%	78.68%	80.84%	77.38%	74.46%	72.19%

SVM: support vector machine; RBF: radial basis function; TPR: true positive rate; FPR: false positive rate; ACC: accuracy.

Table 4.

Execution time of SVM classifier in seconds.

Different road segments	Performance measures	SVM classifierRBF kernel parameter σ
		0.1	10	15	50	90	200
L200	Training time	0.0679	0.0654	0.0773	0.0655	0.0639	0.0640
	Classification time	0.0030	0.0027	0.0029	0.0028	0.0034	0.0042
R200	Training time	0.0677	0.0703	0.0785	0.0663	0.0681	0.0641
	Classification time	0.0034	0.0027	0.0030	0.0031	0.0034	0.0030
L500	Training time	0.0764	0.0740	0.0734	0.0749	0.0655	0.0688
	Classification time	0.0035	0.0036	0.0036	0.0029	0.0028	0.0032
R500	Training time	0.0727	0.0653	0.0640	0.0675	0.0680	0.0642
	Classification time	0.0027	0.0037	0.0031	0.0036	0.0031	0.0043
L800	Training time	0.0794	0.0743	0.0753	0.0868	0.0696	0.0656
	Classification time	0.0028	0.0035	0.0033	0.0026	0.0032	0.0031
R800	Training time	0.0686	0.0705	0.0854	0.0810	0.0716	0.0777
	Classification time	0.0027	0.0044	0.0032	0.0050	0.0031	0.0034
S	Training time	0.0696	0.0642	0.0685	0.0696	0.0648	0.0737
	Classification time	0.0035	0.0032	0.0034	0.0042	0.0028	0.0031
Average	Training time	0.0718	0.0691	0.0746	0.0731	0.0674	0.0683
	Classification time	0.0031	0.0034	0.0032	0.0035	0.0031	0.0035

SVM: support vector machine; RBF: radial basis function.

ANN classification results

As described in section “Data analysis and feature extraction,” the network was trained by gradient descent backpropagation method with momentum and adaptive learning rate, with an initial learning rate of 0.01. The training was terminated when the total sum of squared errors became less than 0.0001. We calculated TPR, FPR, and ACC. The results are shown in Table 5. According to this table, the classifier with 100 neurons in the hidden layer performs the best in terms of accuracy. Table 6 shows the training time and classification time in seconds.

Table 5.

Performance measures of ANN classifier.

Different road segments	Performance measures	ANN classifierNo. of neurons in the hidden layer
		10	20	30	50	100
L200	TPR	86.36%	90.91%	90.91%	95.45%	95.45%
	FPR	22.73%	31.82%	22.73%	31.82%	22.73%
	ACC	81.82%	79.55%	84.09%	81.82%	86.36%
R200	TPR	72.73%	72.73%	72.73%	77.27%	72.73%
	FPR	22.73%	18.18%	18.18%	18.18%	18.18%
	ACC	75%	77.27%	77.27%	79.55%	77.27%
L500	TPR	100%	90.91%	100%	86.36%	90.91%
	FPR	18.18%	13.64%	18.18%	13.64%	22.73%
	ACC	90.91%	88.64%	90.91%	86.36%	84.09%
R500	TPR	81.82%	63.64%	77.27%	68.18%	77.27%
	FPR	45.45%	31.82%	36.36%	36.36%	22.73%
	ACC	68.18%	65.91%	70.45%	65.91%	77.27%
L800	TPR	77.27%	72.73%	81.82%	77.27%	86.36%
	FPR	40.91%	40.91%	45.45%	45.45%	54.55%
	ACC	68.18%	65.91%	68.18%	65.91%	65.91%
R800	TPR	86.36%	90.91%	81.82%	77.27%	90.91%
	FPR	27.27%	31.82%	18.18%	22.73%	27.73%
	ACC	79.55%	79.55%	81.82%	77.27%	81.82%
S	TPR	87.88%	84.85%	86.36%	90.91%	86.36%
	FPR	21.21%	19.70%	22.73%	24.24%	21.21%
	ACC	83.34%	82.58%	81.82%	83.33%	82.58%
Average	TPR	84.63%	80.95%	84.42%	81.82%	85.71%
	FPR	28.35%	26.84%	25.97%	27.49%	27.12%
	ACC	78.14%	77.06%	79.22%	77.16%	79.33%

ANN: artificial neural networks; TPR: true positive rate; FPR: false positive rate; ACC: accuracy.

Table 6.

Execution time of ANN classifier in seconds.

Different road segments	Performance measures	ANN classifierNo. of neurons in the hidden layer
		10	20	30	50	100
L200	Training time	82.4973	82.3984	86.3599	88.6493	89.2121
	Classification time	0.0129	0.0126	0.0164	0.0124	0.0120
R200	Training time	97.3587	115.0316	126.2729	132.7357	161.6979
	Classification time	0.0153	0.0131	0.0164	0.0169	0.0187
L500	Training time	46.8019	46.8033	50.1238	62.3332	69.1656
	Classification time	0.0130	0.0130	0.0128	0.0131	0.0128
R500	Training time	115.8243	115.2418	124.8217	125.0037	145.2890
	Classification time	0.0140	0.0137	0.0186	0.0134	0.0177
L800	Training time	79.8741	78.2545	89.0627	104.0740	100.6951
	Classification time	0.0130	0.0141	0.0138	0.0129	0.0135
R800	Training time	119.8337	105.5011	115.9226	137.9824	170.2928
	Classification time	0.0147	0.0145	0.0140	0.0143	0.0147
S	Training time	107.5977	118.9140	120.8843	124.5985	108.8441
	Classification time	0.0175	0.0132	0.0167	0.0139	0.0128
Average	Training time	92.8268	94.5921	101.9211	110.7681	120.7424
	Classification time	0.0143	0.0135	0.0155	0.0138	0.0146

ANN: artificial neural networks.

Discussions

The classifier with the best performance measures was selected and compared for KNN, SVM, and ANN classifiers. For KNN classifier, the highest accuracy of 84.09%, with a TPR of 77.27% and an FPR of 9.09%, was achieved at curve segment R200. For SVM classifier, the highest accuracy of 88.64%, with a TPR of 86.36% and an FPR of 9.09%, was obtained at curve segment L500. For ANN classifier, the highest accuracy of 90.91%, with a TPR of 100% and an FPR of 18.18%, was obtained at curve segment L500. In general, classification time is more useful than training time in practical application because training can be performed offline and is only needed to be performed once. Accordingly, the average accuracy and classification time of three types of classifiers are compared in this article. The results are shown in Figures 5 and 6. The Friedman test was used to examine the significant difference in the accuracy among three classification methods because the data did not fit a normal distribution. The SPSS software and STAC³³ were used to perform statistical analysis. The results (the test statistic value χ ²(2, 7) = 9.556, p = 0.008 < 0.05) show that there is a significant difference among three classification methods. Accordingly, it is meaningful to conduct pairwise comparisons to identify the differences between the classification methods. Nemenyi post hoc test was used for pairwise comparisons. The results are shown in Table 7. We can see from Table 7 that there is no statistically significant difference in the accuracy between SVM and ANN. Significant differences from KNN are found for SVM (p = 0.015) and ANN (p = 0.048). We can find that the SVM classifier achieved the fastest classification time and the highest average accuracy (80.84%) when σ is equal to 15. The ANN classifier, created with 780 inputs and 1 output, came last in classification time. Compared to SVM, classification time of ANN is much longer. Overall, SVM and ANN are better choices for drowsiness detection.

Figure 5.

Average accuracy.

Figure 6.

Classification time.

Table 7.

Results of pairwise comparisons.

Pairwise comparisons	Statistic value	Adjusted p-value
KNN vs SVM	2.806	0.015
KNN vs ANN	2.405	0.048
SVM vs ANN	0.401	1.000

KNN: K-nearest neighbor; SVM: support vector machine; ANN: artificial neural networks.

The experimental results are encouraging. It can be seen that SVM and ANN classifiers achieve satisfied results in the classification. The study of Eskandarian et al.²⁹ reported that ANN had a TPR of 85%. S Lawoyin³⁴ detected drowsiness using SVM to analyze the steering wheel signals and achieved an accuracy of 81.25%. Our study is consistent with the results of these previous researches. SVM and ANN are effective classification methods for detecting drowsy driving with different road geometries.

Conclusion

This article reveals vehicle performance measures of drowsy driving on different road segments based on a driving simulator. Vehicle performance measures (speed, acceleration, brake pedal, gas pedal, steering angle, and lateral position) were analyzed and evaluated to examine their correlation with drowsiness on different road segment types. We find the waveforms by subtracting the road curvature from the steering angle in the curve segments were different from the waveforms of the straight segments. Furthermore, the less the radius of curvature, the more the difference. Therefore, the classifiers were constructed for each curve segment, respectively, and only one classifier was constructed for all straight segments. Finally, we found that the SVM classifier achieved the fastest classification time and the highest accuracy (80.84%). SVM and ANN are effective classification methods for detecting drowsy driving with different road geometries.

A limitation of this study is that only three types of curve radii were studied, which may not be sufficient to effectively develop a drowsy-driver detection system. Accordingly, future studies will use more roadway geometries. In addition, there might be a significant difference between the driving behavior–based measures during real driving and simulating driving. Hence, further studies are needed to analyze the performance of the classifiers based on a field study.

Footnotes

Academic Editor: Bryan(Byung-gyu) Kim

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

References

Eskandarian

Mortazavi

Sayed

. Drowsy and fatigued driving problem significance and detection based on driver control functions. In: Eskandarian

(ed.) Handbook of intelligent vehicles. London: Springer-Verlag, 2012, pp.941–974.

Brown

. Driver fatigue. Hum Factors 1994; 36: 298–314.

European Road Safety Observatory. Fatigue, http://www.erso.eu (2006, accessed 9 May 2008).

Rosekind

Gregory

et al . Crew factors in flight operations XIII: a survey of fatigue factors in corporate/executive aviation operations. NASA/TM-2000-209610, September 2000. Moffett Field, CA: Ames Research Center, National Aeronautics and Space Administration.

Klauer

Dingus

Neale

et al . The impact of driver inattention on near-crash/crash risk: an analysis using the 100-car naturalistic driving study data. Technical Report No. DOT HS-810 594, April 2006, https://www.nhtsa.gov/sites/nhtsa.dot.gov/files/driverinattention.pdf

Horne

Reyner

. Vehicle accidents related to sleep: a review. Occup Environ Med 1999; 56: 289–294.

Daza

Hernandez

Bergasa

et al . Drowsiness monitoring based on driver and driving data fusion. In: 14th international IEEE conference on intelligent transportation systems, Washington, DC, 5–7 October 2011, pp.1199–1204. New York: IEEE.

Zhu

Lan

. Real-time nonintrusive monitoring and prediction of driver fatigue. IEEE T Veh Technol 2004; 53: 1052–1068.

Bergasa

Nuevo

Sotelo

et al . Real-time system for monitoring driver vigilance. IEEE T Intell Transp 2006; 7: 63–77.

10.

Lal

Craig

Boord

et al . Development of an algorithm for an EEG-based driver fatigue countermeasure. J Safety Res 2003; 34: 321–328.

11.

Ueno

Kaneda

Tsukino

. Development of drowsiness detection system. In: Proceedings of the vehicle navigation and information systems conference, Yokohama, Japan, 31 August–2 September 1994, pp.15–20. New York: IEEE.

12.

Zhang

Wang

. Automated detection of driver fatigue based on entropy and complexity measures. IEEE T Intell Transp 2014; 15(1): 168–177.

13.

Wang

et al . EEG-based fatigue driving detection using correlation dimension. J Vibroeng 2014; 16(1): 407–413.

14.

Lee

Jung

et al . A vision-based method for detecting driver drowsiness and distraction in driver monitoring system. Opt Eng 2011; 50: 127202-1–127202-24.

15.

Wang

Guo

Tong

et al . Monitoring mouth movement for driver fatigue or distraction with one camera. In: 7th international IEEE conference on intelligent transportation systems, Washington, WA, 3–6 October 2004, pp.314–319. New York: IEEE.

16.

Azim

Jaffar

Mirza

et al . Fully automated real time fatigue detection of drivers through fuzzy expert systems. Appl Soft Comput 2014; 18: 25–38.

17.

Shen

Feng

. Multi-source information fusion application to driving fatigue detection. Computer Appl Softw 2012; 29: 271–274.

18.

Zhang

Cheng

Feng

. Real-time detection of driver drowsiness based on steering performance. J Tsinghua Univ 2010; 50(7): 1072–1076.

19.

Krajewski

Golz

Sommer

. Detecting sleepy drivers by pattern recognition based analysis of steering wheel behaviour. In: Lichtenstein

Stößel

Clemens

(eds) Der Mensch im Mittelpunkt technischer Systeme, 2009, pp.288–291. Düsseldorf: VDI-Verlag.

20.

Eskandarian

Mortazavi

. Unobtrusive driver drowsiness detection system and method. Patent application no. 12/613,306, USA, 2009.

21.

Sayed

Eskandarian

. Unobtrusive drowsiness detection by neural network learning of driver steering. Proc IMechE, Part D: J Automobile Engineering 2001; 215(9): 969–975.

22.

King

Mumford

Siegmund

. An algorithm for detecting heavy-truck driver fatigue from steering wheel motion. Richmond, BC, Canada: MacInnis Engineering Associates, 1998.

23.

Wierwille

Ellsworth

. Evaluation of driver drowsiness by trained raters. Accid Anal Prev 1994; 26(5): 571–581.

24.

Yeo

MVM

Shen

et al . Can SVM be used for automatic EEG detection of drowsiness during car driving? Safety Sci 2009; 47(1): 115–124.

25.

Zheng

. Driver drowsiness detection with eyelid related parameters by support vector machine. Expert Syst Appl 2009; 36(4): 7651–7658.

26.

Shen

Ong

et al . EEG-based mental fatigue measurement using multi-class support vector machines with confidence estimate. Clin Neurophysiol 2008; 119(7): 1524–1533.

27.

Yang

Mao

Tijerina

et al . Detection of driver fatigue caused by sleep deprivation. IEEE T Syst Man Cy A 2009; 39(4): 694–705.

28.

Liang

Reyes

Lee

. Real-time detection of driver cognitive distraction using support vector machines. IEEE T Intell Transp 2007; 8(2): 340–350.

29.

Eskandarian

Sayed

Delaigue

et al . Advanced driver fatigue research. Technical Report No. FMCSA-RRR-07-001, April 2007. Ashburn, VA: The George Washington University.

30.

Zhao

Zhang

et al . Effects of fatigue on driving performance under different roadway geometries: a simulator study. Traffic Inj Prev 2015; 16(5): 468–473.

31.

Dong

Uchimura

et al . Driver inattention monitoring system for intelligent vehicles: a review. IEEE T Intell Transp 2011; 12(2): 596–614.

32.

Hagan

Demuth

Beale

. Neural network design. Boston, MA: PWS Publishing, 1996.

33.

Rodríguez-Fdez

Canosa

Mucientes

et al . STAC: a web platform for the comparison of algorithms using statistical tests. In: Proceedings of the 2015 IEEE international conference on fuzzy systems, Istanbul, 2–5 August 2015. New York: IEEE.

34.

Lawoyin

. Novel technologies for the detection and mitigation of drowsy driving. PhD Dissertation, Virginia Commonwealth University, Richmond, VA, December 2014.