A fuzzy recurrent neural network for driver fatigue detection based on steering-wheel angle sensor data

Abstract

The study of the robust fatigue feature learning method for the driver’s operational behavior is of great significance for improving the performance of the real-time detection system for driver’s fatigue state. Aiming at how to extract more abstract and deep features in the driver’s direction operation data in the robust feature learning, this article constructs a fuzzy recurrent neural network model, which includes input layer, fuzzy layer, hidden layer, and output layer. The steering-wheel direction sensing time series sends the time series to the input layer through a fixed time window. After the fuzzification process, it is sent to the hidden layer to share the weight of the hidden layer, realize the memorization of the fatigue feature, and improve the feature depth capability of the steering wheel angle time sequence. The experimental results show that the proposed model achieves an average recognition rate of 87.30% in the fatigue sample database of real vehicle conditions, which indicates that the model has strong robustness to different subjects under real driving conditions. The model proposed in this article has important theoretical and engineering significance for studying the prediction of fatigue driving under real driving conditions.

Keywords

Fatigue driving features fuzzy recurrent neural network steering wheel angle robust learning

Introduction

Road traffic accidents are one of the main causes of human casualties. According to the World Health Organization (WHO),¹ traffic accidents will jump to the fifth place in the main cause of death in 2030, far higher than infectious diseases such as AIDS and tuberculosis. As the primary cause of traffic accidents, fatigue driving has attracted wide attention from all walks of life in recent years. In 2014, NHTSA’s FARS database had 846 driving-related fatal accidents, which was basically consistent with the death data of the past decade; between 2005 and 2009, there were an average of 83,000 accidents related to fatigue driving each year, and 886 among them were fatal accidents (accounting for 2.5% of all fatal accidents).² Therefore, research on identification of driver fatigue state and early warning technology under real vehicle conditions is conducive to reducing road traffic accidents caused by fatigue driving.

Fatigue driving usually refers to the driver’s mental and physiological dysfunction after a long-time driving, which naturally leads to a decrease in the driver’s driving control ability. According to the sensor data source of the fatigue monitoring systems, the existing driver fatigue monitoring systems mainly fall into two types: intrusive and non-intrusive. The intrusive fatigue monitoring system mainly analyzes the physiological characteristics of the driving process by collecting the physiological data of the driver and realizes the monitoring of the driver’s fatigue state. The data used in such systems mainly include electroencephalography (EEG) signals, electro-optical (EO) signals and electrocardiography (ECG) signals.^3–6 The results of other literature studies show that the main disadvantage of the contact fatigue monitoring system is that the system needs to install a data acquisition sensor in a certain part of the driver’s body, which may easily lead to distraction or discomfort of the driver, and thus its application in real conditions is restricted. Non-intrusive systems primarily monitor driver fatigue by extracting driver’s facial features and operational behavior characteristics.^7–11 During the monitoring process, the driver’s normal driving will not be disturbed, and the data acquisition and fatigue monitoring can objectively reflect the driver’s actual operating behavior or state.¹¹ At present, the non-intrusive method has become popular in the field of driver fatigue monitoring.¹²

A fatigue monitoring system based on driver’s steering wheel angle (SWA) information is a type of non-intrusive method. Studies have shown that after driving for a long time, the driver will suffer from lack of energy or fatigue. At this time, his ability to operate or control the vehicle will be greatly reduced, resulting in decline in steering wheel rotation accuracy and frequency.^13,14 Therefore, by collecting the real-time operation data of the driver’s steering wheel, the fatigue characteristics of the direction angle data change are mined, and the driver fatigue level identification model is constructed to realize the effective detection of the driver’s fatigue level.^15–17

Eskandarian and Mortazavi¹⁸ established a monitoring model and method for the driver’s fatigue level by analyzing the amplitude variation characteristics of the driver’s direction operation data, and the correct recognition rate of the fatigue level is 85%. Berglund¹⁹ used simulator data for fatigue driving experiments on 22 subjects, which collected state variables of 10 vehicles including SWA, steering wheel angular velocity, steering wheel torque, yaw rate, and vehicle lateral position. With the extracted 17 fatigue indicators and a linear regression model, the detection accuracy of fatigue detection reached 87%. Fukuda et al.²⁰ analyzed the statistical characteristics of driver’s SWA sequence and found that the periodicity of the time series has a certain internal relationship with the fatigue state, realizing the online identification of the driver’s fatigue level, and the correct recognition rate of the fatigue level reached from 76% to 88%. In addition, based on the statistical analysis of the driver’s direction angle data, Qu et al.² and others extracted 11 characteristic indicators reflecting the driver’s fatigue level, and constructed a SVM-based fatigue classification recognition model. The correct monitoring rate of the third-level fatigue hit 87.7%. However, most of the research data in the existing literature comes from the driving simulator environment, and the reliability of the fatigue monitoring system developed on this basis still lacks effective verification under real vehicle conditions.

The existing literature monitors a lot of work based on the fatigue of the driver’s direction angle, but these methods have extracted more statistical features of drivers’ operation time series, and shows little of their individual difference, which easily leads to the fact that the fatigue detection method is not robust enough under real vehicle conditions. Bittner et al.²¹ reported the effectiveness of the existing fatigue feature indicator under actual road conditions. It was found that angle standard deviation with good performance in the driving simulation environment is not effective in the fatigue detection under real vehicle conditions. This is because the steering characteristics of the real vehicles are not only related to the fatigue level of the driver, but also related to the external environment such as the operator’s driving habits, steering ability, speed, and actual road conditions; Meanwhile, due to the random jitter on the road, the probability of mixing noise and drift in the SWA data is greatly increased, which leads to more complicated analysis of the SWA characteristics, feature selection, and performance optimization under real vehicle conditions.

In addition to the feedforward connection structure of the traditional network, the recurrent neural network realizes the recurrent connection between the neurons in the network, that is, the hidden layer units are not independent of each other, and the hidden layers are not only related to each other, but also the time series input before the time of the hidden layer unit is accepted, so that, in this structure, drivers’ operational characteristics and fatigue features can be transmitted in the direction of the time series. This advantage enables it to make full use of historical information and achieve a series of remarkable results in solving problems such as time-series nonlinear system modeling.^22–25 This article makes use of the unique advantage of the recurrent neural network in processing time series data, designs the feature extraction model for the driver’s direction angle time series under real driving conditions, and deeply exploits the driver’s operational characteristics in the time direction. A robust driver fatigue feature learning model is constructed to solve the individual differences and random complexity faced by the driving fatigue detection system.

Background

Usually, the driving process can be considered as a typical nonlinear dynamic system. Mining the nonlinear characteristics of the time series data of the operational state is beneficial to the driver’s fatigue state analysis and identification. Under the real driving conditions, in order to ensure safe driving, the driver must constantly judge the driving state of the vehicle. Once the vehicle state is out of the estimation, it should be corrected in real time. When the driver is in a state of fatigue, the driver’s ability to perceive the environment, the ability to judge the situation, and the ability to control the vehicle will decrease, resulting in greater error tolerance to the vehicle and reduced control accuracy, and, consequently, the vehicle control variables and state variables will have a different amplitude or frequency with those in the awake state. Existing studies, Krajewski et al.²⁶ and Zhang et al.,²⁷ have shown that fatigued drivers’ behaviors in operating the steering wheel become obviously abnormal compared with those in a non-fatigued state. For instance, the correction time interval of the steering wheel becomes longer and the instantaneous correction range becomes larger. Therefore, the driver’s steering wheel operating characteristics can reflect the fatigue state of the driver, and the nonlinear characteristics, irregularities, and individual differences implied by the steering wheel operation time series data also vary under different fatigue states.

Traditional artificial neural networks, also known as feedforward neural networks (FNNs), consist of a series of simple neurons. A simple FNN usually consists of an input layer, a hidden layer, and an output layer. The neuron response in the FNN is usually derived from the different degrees of excitation of all neurons in the upper layer, that is, mathematically can be expressed as the sum of the weights of all the neurons in the upper layer, but there is no connection between the neurons in the same layer of the FNN. There is no loop on the network structure, and there is no feedback connection between the output of the network and the model itself. Data pass through the neural network layer by layer from the input layer until the final output layer. The observations of all the neuron cells in the same layer of the traditional artificial neural network are processed independently of each other. However, the data in many tasks are rich in context information, and there is a strong correlation between them in time, such as, audio, video, and text, so FNN still has a lot of limitations in many tasks, especially the time series features with strong correlation in mining time dimension and thus the performance of traditional artificial neural network is not promising. The biggest difference between recurrent neural networks and traditional artificial neural networks is that the neurons between the hidden layers have temporal storage and recursion. The output of neurons at a certain moment can be used as the input of the next layer of neurons. The network structure is well suited for time series data and can maintain dependencies between the data. The classical recurrent neural network structure is shown in Figure 1,²⁸ in which the H layer is an implicit layer, which is connected through the loops on the hidden layer, so that the network state at the previous moment can be transmitted to the current moment, and the current state can also be passed to the next moment. For the expanded recurrent neural network, a repetitive structure can be obtained and the parameters in the network structure are shared, greatly reducing the neural network parameters required for training. Therefore, the recurrent neural network has a good advantage in studying the behavioral characteristics of the driver and the learning of fatigue characteristics.

Figure 1.

Traditional recurrent neural network model.

Methodology

Combining the self-organizing, adaptive learning and selective long-term memory ability of self-organizing competitive network, a new fuzzy recurrent neural network is constructed to extract fatigue characteristics, and the fatigue feature space of the driver is obtained. In this article, a four-layer fuzzy recurrent neural network model is designed, and its structure is shown in Figure 2.

Figure 2.

An improved recurrent neural network model.

In Figure 2, the first layer is the input layer, and the input data are the spatio-temporal feature quantity of the SWA under real road conditions, and the input variable is $X_{i}, i = 1, 2, \dots, 60$ . The direction angle time series is processed with double windowing, as shown in Figure 3.

Figure 3.

Time window and sample window of SWA sensor data.

In Figure 3, time window $t_{i} = 1 s$ , $X_{i}$ is the average time window

X_{i} = \frac{1}{f} \sum SW A_{n} (n = 1, 2, \dots, f)

(1)

In equation (1), f is the sampling frequency of the direction angle time series, and $f = 100$ . $SW A_{n}$ is the steering-wheel Angle at a certain sampling point when the driver steering the wheel.

In Figure 2, the second fuzzified neuron is blurred by a smooth Gaussian function, and the fuzzification of the second layer is expressed by equation (2)

F_{i} = \frac{1}{1 + e^{- x_{i}}}

(2)

The third layer is the hidden layer of the recurrent neural network. The neurons are not only related to the excitation output of the previous layer but also realize the information transmission of the neurons in the same layer, making weight influence on the neighbors and depicting the precise relationship between state and time under the real vehicle conditions. The forward propagation of the hidden layer can be expressed by equation (3)

H_{i} = σ (W_{FH} F_{i} + W_{HH} H_{i - 1} + b_{H})

(3)

In equation (3), $W_{FH}$ is the weight matrix from the fuzzy layer to the hidden layer, $W_{HH}$ is the connection weight matrix between the hidden layer neurons, and $b_{H}$ is an offset vector. The recurrent neuron coefficients are trained by backpropagation through time (BPTT) algorithm. For gradient-based learning, there is usually a closed solution between the model parameter $θ$ and the loss function L. The loss function is minimized according to the error between the estimated value and the actual value. The calculated gradient information can be transmitted back to the model to modify the neural network parameters. Assume for the sequence $Z_{1}, Z_{2}, \dots, Z_{t}$ , the state of the previous moment $S_{t - 1}$ is mapped to the state of the next moment $S_{t}$ through $S_{t} = G_{θ} (S_{t - 1}, Z_{t})$ . The gradient of the hidden layer neuron loss function is shown in equation (4)

\nabla_{θ} L_{T} = \frac{\partial L_{T}}{\partial θ} = \sum_{t \leq T} \frac{\partial L_{T}}{\partial S_{T}} \frac{\partial S_{T}}{\partial S_{t}} \frac{\partial G_{θ} (S_{t - 1}, Z_{t})}{\partial θ}

(4)

According to the chain rule, the Jacobian matrix $\frac{\partial S_{T}}{\partial S_{t}}$ is decomposed as shown in equation (4)

\frac{\partial S_{T}}{\partial S_{t}} = \frac{\partial S_{T}}{\partial S_{T - 1}} \frac{\partial S_{T - 1}}{\partial S_{T - 2}} \dots \frac{\partial S_{t + 1}}{\partial S_{t}}

(5)

The fourth layer in Figure 2 is the competition layer, and the competition winner, output 1, is transmitted to the next neuron through the memory unit $S$ and constitutes a sequence of fatigue characteristic indicators. The algorithm for finding winning neurons is as follows: assume that the mode input through the input layer is $X = (X_{1}, X_{2}, \dots, X_{n})$ , where $X_{i} = [\underline{x_{i}}, \bar{x_{i}}] \in I [0, 1]$ , $i = 1, 2, \dots, n$ , and all the neurons of the competition layer correspond to the weight vector $w_{j} = (w_{j 1}, w_{j 2}, \dots, w_{jn}) (j = 1, 2, \dots, m)$ ; the weight vector most similar to X is judged as the winning neuron. Finally, adjust the weight according to the Winner-Take-All learning rule. The adjustment rules are

w_{j^{*}} (t + 1) = w_{j^{*}} (t) + α (f (w_{j^{*} 1}, X), f (w_{j^{*} 2}, X), \dots, f (w_{j^{*} n}, X))

(6)

f (w_{j^{*} i}, X) = sgn (\underline{x_{i}} - w_{j^{*} i}) d (w_{j^{*} i}, X)

(7)

In equation (6), $α \in [0, 1]$ is the learning rate. After fully learning with this method, the weight vector of the winning neuron becomes a clustering center of the input mode.

Experiment and results

Experiment platform and sample data

The structure of the real vehicle and road data acquisition platform is shown in Figure 4. Distributed sensors are mainly used for data acquisition of driver operating characteristics and vehicle status such as SWA, brakefroce, leftsteer, rightsteer, can_braking, can_thrott, and YawRate. The data collected by the sensor are processed by the VBOX system, to realize data storage, display and video monitoring, and so on.

Figure 4.

Acquisition platform for driver operation and vehicle data.

With the data acquisition platform shown in Figure 4, the subjects’ facial video images are obtained. This article uses the facial-video-based expert scoring method to construct the driver fatigue state sample database. The method scores the fatigue status of a driver based on the driver’s facial expression and head posture. Wierwille and Ellsworth²⁹ first cited this method in the driver fatigue state assessment. The specific operation steps are as follows: the driver’s facial video is divided into video clips; multiple experts score the video clips according to the random sequence according to the driver’s blinking, scratching, yawning, closing the eyes, adjusting the posture, and other fatigue characterizations. The evaluation result is a continuous value of 0–100 points; the average of multiple expert scores is used as the fatigue score of the video. In this article, the driver’s fatigue state is set to three levels, that is, awake, fatigue, and very fatigue level. With the driver’s facial video as the evaluation basis, the driver’s operation characteristics and vehicle state data collected by VBOX are taken as experimental data. Considering the complexity of multidimensional spatio-temporal data, this article only takes the driver’s SWA data as the research object to test the feasibility of the designed model in this article. The time series distribution of the SWA when the driver’s awake, fatigued, and very fatigued is, respectively, shown in Figures 5 –7 . As can be seen from Figure 5, when the driver is in the awake state, the frequency of the wheel-steering operated by the driver is relatively high, and the frequency of the substantial direction operation is low. When the driver is getting tired, the driver’s direction steering frequency is reduced, and the frequency in dramatic steering is increased as shown in Figure 6. When the driver is very tired, with the decrease of driver’s control ability, the time of non-operating the steering wheel increases, and at the same time, the driver quickly and dramatically turns the steering wheel, and the time series distribution is as shown in Figure 7. From the fatigue driving data of eight subjects, after data filtering processing, 790 fatigue driving samples were formed, and the fatigue state level distribution of each subject is shown in Figure 8. In the experiment of the article, the number of neural network layers is four layers, the input layer node, the fuzzy layer, the hidden layer, and the output layer node are all 60, and the learning rate is 0.42.

Figure 5.

Time series of SWA at awake level.

Figure 6.

Time series of SWA at fatigue level.

Figure 7.

Time series of SWA at very fatigue level.

Figure 8.

Samples distribution of driver fatigue state.

Experiment results

In this article, 60% of the samples are randomly selected from the 790 samples of eight subjects as training samples, and the remaining 40% are test samples. First, the “very fatigued” level is incorporated into the general “fatigued” level, that is, the fatigue level of the test sample is divided into two levels: awake and fatigued, and all the sample test results of the fatigue level “0” and “1” of the subjects are shown in Table 1. The test samples in Table 1 totals 315, including 222 cases of fatigue level “0” and 93 cases of fatigue level “1.” It can be seen from Table 1 that the correct detection rate of fatigue level “0” is 90.54%, and misjudgment rate is 9.46%. For the fatigue level “1,” the correct detection rate is 80.65% and the false positive rate is 19.35%. Second, the sample data are tested according to the three fatigue levels, that is, the fatigue states of the sample are “awake,”“fatigued,” and “very fatigued,” which are represented by “0,”“1,” and “2,” respectively. The results are as shown in Table 2: there are 315 experimental test samples, including 222 cases of awake state, 76 cases of fatigued state, 17 cases of very fatigued state, the correct recognition rate of samples in awake state is 90.05%, and the false rate is 9.91%, all of which are falsely judged as fatigue level 1. The correct recognition rate of the fatigue state sample is 80.26%, and the false rate is 19.74%. Among them, 10.21% fatigued samples are misjudged to awake, and 9.53% misjudged to very fatigued; the recognition rate of the very fatigued sample is 76.47%, and the false rate is 23.53%, among which 5.52% very fatigued samples are misjudged to awake, and 18.01% are misjudged to fatigued state. The average false positive rate is 9.68% and the average false negative rate is 13.99% from the all samples. Based on all the test samples in Tables 1 and 2, the average correct recognition rate is 87.30%.

Table 1.

Confusion matrix of the SWA based for detection drowsiness levels “0” and “l.”

		Detection results
		“Awake” (level 0)	“Fatigued” (level 1)
Expert classification	“Awake” (level 0)	90.54%	19.35%
Expert classification	“Fatigued” (level 1)	9.46%	80.65%
Samples		201	93

Table 2.

Confusion matrix of the SWA based for detection fatigue levels “0,”“l,” and “2.”

		Detection results
		“Awake” (level 0)	“Fatigued” (level 1)	“Very fatigued” (level 2)
Expert classification	“Awake” (level 0)	90.09%	9.21%	5.88%
	“Fatigued” (level 1)	9.91%	80.26%	17.65%
	“Very fatigued” (level 2)	0.00%	10.53%	76.47%
Samples		200	76	17

SWA: steering wheel angle.

Discussion

The method proposed in this article uses the SWA data to detect the driver’s fatigue state based on the fact that the driver’s mental state can directly reflect its operational behavior. The steering wheel rotation is the most frequent and sensitive operation of the driver. Using the SWA makes the system more robust and reliable. The fatigue detection method based on the SWA can identify the three levels of fatigue status: “awake,”“fatigued,” and “very fatigued.” The method proposed in this article takes the driving sample data of eight subjects as the experimental object. The correct rate for fatigue state detection reaches 87.02%. The reliability of the detection system is ensured by the fact that the direction angle data in obtained in the real road environment and more accurately reflects the mental state and operational behavior of the driver than the data in the laboratory simulation environment. At the same time, the detection system has a very low rate of false alarms of “fatigued” state (9.46%) from Table 1, and the rate of false alarms of “very fatigued” state (9.53%) from Table 2. This detection system is also robust because the evaluation criteria of the system comes from the consistent evaluation results of multiple experts and multiple methods, which is based on facial video information, rather than the direction angle information. The universality of the standard evaluation method is resulting from its combination of the driver’s facial expression, head posture, and mental state conditions to comprehensively score the driver’s fatigue rating. The recursiveness of the evaluation criteria makes the detection system very robust.

Unfortunately, there’s little existing literature reporting the direction-angle fatigue detection method based on real vehicle conditions. It is difficult to have an objective comparison of the superiority of the method proposed in this article. Compared with the author’s previous research, the correct recognition rate of the proposed method, 87.30%, is higher than the previous results, 82.07% and 78.01%.^15,16 Qu et al.,² using the simulation environment sample data under the laboratory platform, with the SWA data, has obtained a correct recognition rate of 86.1%, which is lower than the correct recognition rate of this article, 87.30%. Comparing the directional data characteristics of the real vehicle environment and the laboratory environment, it is found that in actual vehicle conditions, there is a serious drift in the steering wheel corner sequence due to the randomness of the vehicle vibration. The irregular drift of the time series exists in the original data as the statistical features of an illusion, making the statistical feature extraction of the directional angles extremely difficult. Although the correct rate of the driver’s fatigue state detection in the direction angle is lower than that of the biomedical signal (The accuracy of estimation is about 96.5%–99.5%),³ considering the actual driving environment, the method proposed in this article is more convenient and feasible.

Conclusion and future work

A fatigue driving characteristic learning model based on recurrent neural network structure is proposed. This model has solved the problem that the fatigue feature extraction of steering wheel operation data is difficult under real vehicle conditions. Considering the safety in real driving conditions, more researches were done in the past on the simulation angle data based on the laboratories and much less on the data based on the actual driving conditions. This article uses multi-distributed sensors installed in the real vehicle system, collecting data on driver operating characteristics and vehicle status such as SWA, brakefroce, leftsteer, rightsteer, can_braking, can_thrott, YawRate, and facial video, combining the time series distribution characteristics of steering data, and innovatively proposes a fuzzy circulatory neural network model. Taking 557 samples as the object, this method has achieved the fatigue state identification rate of 87.30%, surpassing the fatigue state detection based on laboratory simulation data.

Previous work of Qu et al.² has shown that the SWA, combined with vehicle yaw angle, vehicle lateral position, and other vehicle status data, has a higher fatigue state recognition rate in the laboratory simulation environment. Inspired by this idea, the integration of SWA, vehicle yaw angle, vehicle lateral position, and other vehicle status data under real vehicle conditions may improve the driver’s fatigue detection accuracy, which is a meaningful work for us.

Footnotes

Handling Editor: Xing Chen

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Natural Science Foundation of China under grant number 61873043 and, in part, by the Natural Science Foundation of Chongqing under grant number cstc2018jcyjAX0048.

ORCID iD

Zuojin Li

References

World Health Organization (WHO). Global status report on road safety: time for action. Geneva: WHO, 2009, http://www.who.int/violence_injury_prevention/road_safety_status/2009/en/

Cheng

Lin

, et al. Drowsy driving detection based on driver’s steering operation characteristics. Qiche Gongcheng/Automot Eng 2013; 35(9): 803–831.

Zhang

Wang

. Automated detection of driver fatigue based on entropy and complexity measures. IEEE T Intell Transp 2013; 15(1): 168–177.

Picot

Charbonnier

Caplier

. On-line detection of drowsiness using brain and visual information. IEEE T Syst Man Cy A 2012; 42(3): 764–775.

Jung

Shin

Chung

. Driver fatigue and drowsiness monitoring system with embedded electrocardiogram sensor on steering wheel. IEEE T Intell Transp 2014; 8(1): 43–50.

Correa

Orosco

Laciar

. Automatic detection of drowsiness in EEG records based on multimodal analysis. Med Eng Phys 2013; 36(2): 244–249.

Chen

. Detection of driver’ drowsiness using facial expression features. J Automot Saf Energy 2010; 13(5): 200–204.

Zhang

. Driver fatigue detection algorithm based on the states of eyes and mouth. Computer Appl Softw 2013; 30(3): 25–54.

Lin

Chen

, et al. Sleepy eye’s recognition for drowsiness detection. Int Symp Biom Secur Technol 2013; 14(7): 176–179.

10.

Wierwille

Wreggit

Kirn

, et al. Research on vehicle-based driver status/performance monitoring; development, validation, and refinement of algorithms for detection of driver drowsiness. Final Report No. DOT HS 808 638, September 1996. Washington, DC: National Highway Traffic Safety Administration.

11.

King

Siegmund

Montgomery

. Outfitting a freightliner tractor for measuring driver fatigue and vehicle kinematics during closed-track testing. J Commer Veh 1994; 103(2): 611–621.

12.

Wang

Zheng

, et al. Longitudinal collision mitigation via coordinated braking of multiple vehicles using model predictive control. Integr Comput-Aid E 2015; 22(2): 171–185.

13.

Wang

. Characterization of longitudinal driving behavior by measurable parameters. Transport Res Rec: J Transp Res Board 2010; 2185: 15–23.

14.

Wang

Zhang

, et al. An adaptive longitudinal driving assistance system based on driver characteristics. IEEE T Intell Transp 2013; 14(1): 1–12.

15.

, et al. Driver fatigue detection using approximate entropic of steering wheel angle from real driving data. Int J Robot Autom 2017; 32(3): 291–298.

16.

, et al. Online detection of driver fatigue using steering wheel angles for real driving conditions. Sensors 2017; 17(3): 495.

17.

Chen

Peng

, et al. Automatic detection of driver fatigue using driving operation information for transportation safety. Sensors 2017; 17(6): 1212.

18.

Eskandarian

Mortazavi

. Evaluation of a smart algorithm for commercial vehicle driver drowsiness detection. In: Proceedings of the 2007 IEEE intelligent vehicles symposium, Istanbul, 13–15 June 2007, pp.553–559. New York: IEEE.

19.

Berglund

. In-vehicle prediction of truck driver sleepiness: steering related variables. Institutionen För Systemteknik, 2007, https://pdfs.semanticscholar.org/67b9/48606bd8fbd2bc85dbb87b57bd3d0e7d7049.pdf?_ga=2.75121175.440716504.1565588886-1383787297.1561959603

20.

Fukuda

Akutsu

Aoki

. Estimation of driver’s drowsiness level using interval of steering adjustment for lane keeping. JSAE Rev 1995; 16(2): 197–199.

21.

Bittner

Smrcka

Vysoký

, et al. Detecting of fatigue states of a car driver. In: International symposium on medical data analysis, Frankfurt, 29–30 September 2000, pp.260–273. Berlin; Heidelberg: Springer-Verlag.

22.

Palangi

Deng

Shen

, et al. Deep sentence embedding using long short-term memory networks: analysis and application to information retrieval. IEEE/ACM IEEE T Audio Speech 2016; 24(4): 694–707.

23.

Nguyen

Anh

Yang

H-J

. Real-time event detection using recurrent neural network in social sensors. Int J Distrib Sens N 2019; 15(6): 1–15.

24.

Lee

. Deep learning-assisted real-time container corner casting recognition. Int J Distrib Sens N 2019; 15(1): 1–10.

25.

Shi

Wang

. The modeling and the sensor fault diagnosis of a continuous stirred tank reactor with a Takagi-Sugeno recurrent fuzzy neural network. Int J Distrib Sens N 2009; 5(1): 37.

26.

Krajewski

Golz

Sommer

. Detecting sleepy drivers by pattern recognition based analysis of steering wheel behaviour. In: Proceedings of the fifth international driving symposium on human factors in driver assessment, training and vehicle design, Big Sky, MT, USA, 22–25 June 2009, pp.118–224.

27.

Zhang

Cheng

Feng

. Real-time detection of driver drowsiness based on steering performance. Tsing Hua Univ (Sci & Tech) 2010; 50(7): 1072–1081.

28.

Elman

. Finding structure in time. Cognitive Sci 1990; 14(2): 179–211.

29.

Wierwille

Ellsworth

. Evaluation of driver drowsiness by trained observers. Accident Anal Prev 1994; 26(5): 571–581.