Towards a cognitive warning system for safer hybrid traffic

Abstract

Technological development brings increasingly closer the era of widely available self-driving cars. However, presumably there will be a time when human drivers and self-driving cars would share the same roads. In the current paper, we propose a cognitive warning system that utilizes information collected from the behaviour of the human driver and sends warning signals to self-driving cars in case of human related emergency. We demonstrate that such risk detection can identify danger earlier than an external sensor would, based on the behaviour of the human-driven vehicle. We used data from a simulator experiment, where 21 participants slalomed between road bumps in a virtual reality environment. Occasionally, they had to react to dangerous roadside stimuli by large steering movements. We used one-class SVM to detect emergency behaviour in both steering and vehicle trajectory data. We found earlier detection of emergency based on steering wheel data, than based on vehicle trajectory data. We conclude that tracking cognitive variables of the human driver means that we can utilize the outstanding power of the brain to evaluate external stimuli. Information about the result of this evaluation (be it steering action or saccade) could be the basis of a warning signal that is readily understood by the computer of a self-driving car.

Keywords

Warning system driver behaviour one-class SVM t-SNE

1. Introduction

Since 2009, when Google started testing Google Chauffeur driven cars, they accomplished driving over 1.5 million miles with only 22 documented minor accidents [1]. Interestingly, human error was found underlying all but one of these [2]. This warns to the fact that in spite of self-driving cars being a safer mode of transportation [3], a hybrid traffic of human-driven and self-driving cars is still prone to human faults. Human drivers are object to biological limitations (e.g. drowsiness) and tend to do multitasking in the car, thus providing suboptimal response in emergency situations [4]. Several in-car warning system designs have been implemented in order to reduce the risk of fatal outcomes [5]. In the present paper, we propose that these warning systems should not only raise the driver’s attention, but could be also used to inform other participants of the traffic, namely self-driving cars.

Widespread availably of passenger cars in the middle of the 20th century raised attention to traffic safety [9]. Since then, several different kinds of accident risk evaluation systems have been proposed. Amongst these we can distinguish three main types based on the source of data they use for estimation. These are (1) traffic data based, (2) car position based, and (3) driver behaviour based approaches. Traffic data-based approaches are typically based on traffic surveillance data and use that to evaluate the risk of accident depending on timeslot, traffic frequency and area (highway, intersection) [10, 11, 12, 13, 14]. Not entirely different from these systems [15, 16, 17] are those that work on the single car basis and use sensors of the master vehicle to predict risks of the peers. Current self-driving concept cars rely mostly on this technology [18]. The third type of risk evaluation systems is the set of systems that collect information from the driver. Driver behaviour-based models use gaze [19, 20], facial coding [19, 21], EEG [22, 23], and motion trajectories [24, 25, 26] recorded with various sensors. These solutions give very good real-time estimates that can be used to warn the driver for a potential risk of falling asleep [24, 27], driving through a red light [26], or for optimal lane-changing trajectory [28]. Here, we propose that these warnings could help the hybrid traffic of human-driven and self-driving cars in the future. This way they work more as a communication channel between two agents and not as a one-way sensor, hence the term cognitive in the title.

While a human driver may not be able to evaluate a warning message from a lead car in a couple milliseconds, this is not a problem for the processor of a self-driving car. Automated vehicles constantly monitor their surroundings with several sensors to provide the safest transportation possible [29]. Nonetheless, information collected inside the car’s cockpit may forego the externally detectable risk with tens or, sometimes, hundreds of milliseconds. This is true even if we take the steering wheel, where there is a few millisecond delay between the steering action and the chassis response [30]. Thus, these warnings may be extremely helpful for self-driving cars.

The proposed solution could be a good example of how biological and artificial cognitive agents could co-evolve [31, 32], emerging in a safer traffic infrastructure. The current proposal is not the first that promote consideration of cognitive factors in traffic safety [9, 33, 34], or increased communication between traffic participants [35, 36]. However, it is unique in its emphasis on human-to-machine information flow. Ongoing research [17, 29, 36, 37] is focusing on the design of optimal wireless communication between vehicles (vehicle to vehicle, V2V) and between vehicles and road-side units (vehicle to infrastructure, V2I). These communication links support efficiently the drivers’ situational awareness. Although situational awareness often refers to human situational awareness [38], it bears relevance also in human-machine (or possibly even in machine-machine) situations as a general concept of information availability and use in an interaction [39]. To demonstrate whether we can potentially facilitate situational awareness of a machine, we validate our idea by predicting abrupt steering wheel turn actions of a human driver in a virtual reality simulator paradigm. Here, from time to time the driver had to make emergency steering movements to roadside stimuli [40]. In the present analysis we used the car trajectory and the steering wheel angle data to investigate how early we can detect the initiation of an emergency steering behaviour only based on data from either external sensor.

In the current proof-of-concept implementation we used a one-class support vector machine (OC-SVM). SVM [41, 42, 43] is a set of machine learning models that uses support vectors (i.e. hyperplanes) in high dimensional space for classification and regression problems. Our choice of model was motivated by three main reasons. First, SVM solutions are fast and are often used in real-time applications [44]. Second, such a model can be extended, for example, a recent study presented a hybrid model of an OC-SVM and a deep belief network that outperformed a deep autoencoder in terms of speed on an anomaly detection task in high dimensional data [45]. Third, SVM can be trained even on computers with modest processing power. This latter argument is important since the current ideas may later give birth to an actual product. Presumably, people who cannot afford buying new self-driving cars would adhere to using human-driven cars, and thus would be the target audience of such an instrument. This facilitates the design of an efficient, yet inexpensive device.

We hypothesized that abrupt steering movements can be readily detected using both steering and car trajectory data. Moreover, we predicted that emergency events are detected earlier based on steering than on trajectory data. We aimed to propose a general anomaly detection system that could potentially use multidimensional data (e.g. EEG, eye-tracking etc.). These sensors could provide even earlier detection of an emergency [46]. Therefore we did not include any prior expectation of the dangerous events, only data of normal driving and hence the use of OC-SVM.

Figure 1.

The experimental design. (a) Participants had to slalom through road bumps on a rural road. (b) From time to time, a deer raised up its head from the bushes. If the animal was facing to the road they had to steer to the other end of the road. If the deer looked the other direction they did not have to do anything. The red rectangle serves illustrative purposes.

2. Methods

2.1 Participants

Twenty-three participants took part in the virtual reality experiment. Two of them experienced simulator sickness, therefore their data was excluded. The training and test data were extracted from the steering and trajectory data of the remaining 21 participants (age $M=$ 25.29, SD $=$ 5.54 years; age range: 19–37 years; 10 men and 11 women). All of them reported normal hearing and normal or corrected-to-normal vision. They were also tested for stereo vision (Randot test) and stereo-projection was adjusted accordingly with the interpupillary distance. All participants were right-handed. Neither of the participants had a history of neurological disorder or epilepsy. All of them had valid driving license and frequently drove a car in the past months. As inclusion criteria they had at least 50,000 km driving experience prior to the experiment. Participants were recruited volunteers from students of the Aix-Marseille University. Written informed consent was collected prior to the experiment, and the experimental protocol was designed according to the Declaration of Helsinki and was approved by the local ethical committee.

2.2 Experiment

The experiment took place in a cave automatic virtual environment (CAVE [47]) at the Centre de la Realité Virtuelle de la Mediterranean (CRVM), Aix-Marseille University. The CAVE consisted of three backprojected, 3 by 4 meter side screens and a fiberglass screen of 3 by 3 meter on the floor. Two Barco 5000 lumen projectors illuminated each screen. Participants sat in a custom built car simulator consisting of a car seat frame and a force feedback steering wheel (Logitech G27). Sounds were coming from two loudspeakers placed on both sides of the car frame.

We designed a driving simulator game in Unity 3D, where participants were told to drive on a rural road bounded by bushes on both sides. The road was flat and the scene did not contain other landmarks that may have distracted the driver’s attention. The experiment contained two kinds of tasks. Most of the time they had to slalom between road bumps. The task required continuous left/right steering movements. The road bumps appeared on both sides of the road to guarantee that only small steering movements were used, and the trial was only successful if the participant passed between the two road bumps (see Fig. 1). A green disk placed between the road bumps indicated the ideal position of passing. Running over a road bump was signalled by a small vibration on the steering wheel. This task was sometimes interrupted by an emergency event.

The emergency event was the appearance of a deer in the bushes, either on the left or on the right side of the road. The orientation of the deer’s jaw signalled whether a response was required or not (Go-NoGo task). If the deer was facing the road it signalled emergency (Go signal), if it turned away then no response was required (NoGo signal). In case of emergency, participants were instructed to steer to the other side (i.e. large steering movement) in order to avoid a collision. If the orientation of the deer did not implicate emergency, the participants were instructed to execute the primary task and not to react to the deer.

2.3 Procedure

The experiment started with a practice phase where participants were familiarized with the task. We looked for signs of simulator sickness to avoid unwanted discomfort caused by performing the task for a prolonged period. The data used in the current analysis was collected from four 5 minute-long blocks. The participants were free to take a rest, stand up, walk and drink between the blocks. The total duration of the experiment was approximately one hour, including breaks.

During the experiment, emergency events appeared with 20% chance. Time between road bumps varied between 300 and 1700 msec (distance: 5.9 m to 34 m at 70 km/h speed). Emergency events always followed a road bump with 650 to 700 msec and when they appeared they were the closest visual target stimuli. Emergency events were followed by road bump with 300 to 350 msec. This way the distance between the two road bumps bounding the emergency event was equal to the average distance of two road bumps. We used this configuration to avoid that participants could anticipate the emergency events.

2.4 Data preprocessing

Data preprocessing and modelling was done in Python [48] using Pandas [49], Scikit-learn [50], visualisation was done using Matplotlib [51] and Seaborn. Trajectory and steering angle data was logged in every 50 msec with high precision, according to the Unity environment internal physics. Normal driving data was extracted from the trajectories by selecting data points outside the emergency events. Emergency event onsets were defined as the moment when the deer become visible.

We defined the time window of the emergency events from $-$ 100 msec 1900 msec, 0 msec being the onset of the emergency stimulus. Both for the trajectory and for the steering angle we calculated first (speed), second (acceleration) and third order (jerk) derivatives using finite difference approximation, formulated as

$\displaystyle\bm{X}=\left[{\begin{array}[]{*{20}c}\vec{x}_{4}\\ \vec{x}_{5}\\ \vdots\\ \vec{x}_{n}\\ \end{array}}\right]\to\left[{\begin{array}[]{*{20}c}r_{4}&\nabla_{4}^{1}&% \nabla_{4}^{2}&\nabla_{4}^{3}\\ r_{5}&\nabla_{5}^{1}&\nabla_{5}^{2}&\nabla_{5}^{3}\\ \vdots&\vdots&\vdots&\vdots\\ r_{n}&\nabla_{n}^{1}&\nabla_{n}^{2}&\nabla_{n}^{3}\\ \end{array}}\right],$ $\displaystyle\quad∼{}\vec{x}_{i}\in\mathbb{R}$

where $\bm{X}$ is the input data of $\vec{x}$ vectors for $n$ time points. The dimensions of $\vec{x}_{i}$ are $r$ , which is either the raw measurement of steering wheel angle or vehicle position in the $i$ th time point, and $\nabla^{1}$ , $\nabla^{2}$ , $\nabla^{3}$ , which are the first, second and third order finite backward differences in that time point $i$ , respectively. The time points start at 4 because third order finite backward differences were defined only after 3 data points.

Consequently, we had a four dimensional vector available for every time point, which was used as the input of the risk prediction model. This way the model was able to handle short range dependencies of the time-series data.

In the following we will refer the normal driving data as no event and the emergency data as event. Thus data points were in theory either normal ( $S$ ) or emergency ( $\bar{S}$ ) points labels, these were denoted as $+$ 1 or $-$ 1 such as

$\displaystyle y=\begin{cases}+1\text{ if }\vec{x}\in S\\ -1\text{ if }\vec{x}\in\bar{S}\end{cases}$ $\displaystyle\quad∼{}\text{where }S=\{\text{no event}\}\text{ and }\bar{S}=\{% \text{event}\}$

This means that we could have used the $\bar{S}$ data points and train a binary classifier. However, our aim was to design a model that could detect any anomalies outside the normal range. Hence, we trained separate one-class support vector machine models(OC-SVM) for the steering angle and for the trajectory data. The OC-SVM is finding a hyperplane that identifies the boundaries of the training pattern from the origin of the feature space $\mathbb{F}$ [52]. Because this is often difficult in the original feature space, we mapped them using function $\Phi$ and using a Gaussian (RBF) kernel space transformation [53]. The kernel function was formulated as

$\exp(-\gamma\|\vec{x}-\vec{x}^{\prime}\|^{2}),\quad\gamma=0.25,$

where $\gamma$ is the kernel coefficient that defines how far the influence of a single training example reaches, where low values mean far and $\gamma\in\mathbb{R}|\gamma>0,\vec{x}^{\prime}$ are the centroids. During training, one needs to solve the quadratic programming problem of

$\min(\vec{\omega},\xi,\rho)\quad\!\frac{1}{2}\|\vec{\omega}\|^{2}+\frac{1}{\nu n% }\sum\limits_{i=4}^{n}{\xi_{i}-\rho},\quad\!\nu=0.1$

that is subject to

$\left(\vec{\omega}\cdot\Phi\left(\vec{x}_{i}\right)\right)\geqslant\rho-\xi_{i% },\quad\xi_{i}\geqslant 0$

here, $n$ is the number of samples, $\xi_{i}$ are the slack variables, $\vec{\omega}$ is the hyperplane weight vector, $\rho$ is the bias term. $\nu\in(0,1]$ and this regularization parameter adds an upper bound on the fraction of training errors and a lower bound on the fraction of resulting support vectors. If $\omega$ and $\rho$ solved the problem the following decision function is yielded

$\hat{y}=\,{\rm sign}\,(\left(\vec{\omega}\cdot\Phi(\vec{x})\right)-\rho)$

which yields positive values for $S$ . Parameters were chosen to generate the least amount of false alarms. However because we cannot be certain that the training set does not include any accidental anomalies (i.e. quick/large steering movements), we set the $\nu$ parameter so that the false alarm rate was around 5% (i.e. this would mean 1 package/sec on average with the 20 Hz sampling rate). This was used a fair trade-off between earlier detection of emergency and more false alarms. Shrinking heuristic was used in the training to speed up optimization [54].

Figure 2.

Detection time of Emergency from steering wheel and position data. We were able to predict emergency from steering data earlier than from lateral position because of the non-linear relation between steering angle and vehicle position. Whiskers show 95 % confidence intervals for the mean.

Figure 3.

Relationship between steering angle and vehicle position. It can be on the two dimensional histogram, that the position of the vehicle changes in a rather curvilinear manner relative to the steering angle (nova from the centres). The two dense centres are results of the slaloming task, where the car was either going slightly left or slightly right, the smaller circular pattern around the centres also resulted from the slaloming task. The histogram uses jet colormapping, which goes from blue through green to red.

3. Results

As a first step, we divided the whole no event data to training and validation sets by randomly assigning half of the time points to one and the other half to the other. Because our aim was to build a model that uses both general and personalized information, we did not split the data to two pools of participants. The model gave very small amount of false alarms on the validation set: 4.86% for the steering angle data and 4.06% for the trajectory data. After this, we used the support vectors of this model to detect the earliest anomaly point in the event data. We expected significantly high detection rate of the emergency events, and earlier detection of anomalies in the steering wheel data than in the trajectory data.

Emergencies were detected 645.15 ( $\pm$ 219.67) msec after the onset of the event. In total 2735 emergency events were detected and 8 remained undetected. As can be seen in Fig. 2 this is in the beginning of the trajectory curvature in the emergency trials meaning that we detected emergency very early in time. On the trajectory data anomalies were detected 734.54 ( $\pm$ 269.44) msec after the onset of the event, significantly later than in the steering angle data ( $t$ (1530) $=$ $-$ 17.24 $p<$ 0.001). The detection rate was not different: 2736 emergency events were detected and 7 were undetected. The reason why steering angle made earlier detection possible is the non-linear relationship between steering angle and vehicle position (see Fig. 3).

Figure 4.

t-SNE embedding of no event and earliest detected emergency event data. The embedding method clearly visualizes the decision boundaries between event and no event data. Only a fraction of 30.000 data points are displayed.

We visualized the anomaly detection thresholds based on the validation set and emergency event data points using the t-Distributed Stochastic Neighbour Embedding (t-SNE) method [55]. This method efficiently visualizes high-dimensional data by using joint probabilities of a low-dimensional embedding. The transformation was run using the Barnes-Hut approximation in order to perform calculation in quasi-linear time. The results of the t-SNE show that the no event and emergency event data points are easily differentiable (see Fig. 4).

Summarizing the results, we found that emergency events were readily detected both in wheel angle and in trajectory data using a OC-SVM. Steering data made possible earlier detection of emergency events than trajectory data.

4. Discussion

In the current work we proposed an in-car risk detection and warning system that could inform automatic vehicles on the road about the cautious actions of the human driver (e.g. abrupt steering movement, falling asleep). We illustrated the benefits of the risk detection component by predicting dangerous steering movements earlier from wheel angle data than from vehicle trajectory data, because of the non-linear relationship between steering angle and vehicle lateral position [56, 57].

We used one class support vector machine for learning and prediction. These type of models are common in outlier detection scenarios for various problems [45, 58, 59]. Note, that by controlling the sparsity parameter of the SVM we can limit the number of support vectors used for prediction [54], there are even solutions to find the optimal number of support vectors for a given problem [60]. Moreover, while training an SVM (and potentially multitude of SVMs for each car on the road) would be infeasible inside a master vehicle, our proposal leads to computational efficiency since training and prediction could run on the individual peer vehicles. This fact opens the door to highly individualized models.

We found earlier detection of risk in wheel angle data than in trajectory data. Although this is in line with the expectations (i.e. because of steering backlash, vehicle inertia, tire stiffness), a limitation of the current study is that it was done in virtual reality. While reactions in virtual reality are comparable to those in the real-world [61], the physics of the virtual environment are simpler than reality. Not speaking of the large variance of normal driver behaviour in real world scenarios. While in our case there were only two tasks, outside of the simulator the driver faces all the challenges of traffic. This necessitates further exploration under more naturalistic circumstances. Nonetheless, our choice of virtual reality was motivated by the fact that only this way we were able to generate large amount of clean and labelled data for training and test without real risk of accident. Further studies should evaluate the effectiveness of such a system with more degrees of freedom. Here participants were only able to control the steering wheel angle but not the speed of the car, in reality steering wheel angle changes depends on the speed of the car too, also manufacturers apply speed steering solutions in today’s cars [56].

Worthy to note, that the change of the steering wheel angle is indicative of rather distant elements of the perception-action cycle. Hence, presumably more benefit we earn from such a model when more proximal cognitive variables are tracked. Eye and face tracking in the cockpit could help detecting drowsiness very early in time [21], but also – in situations like the current experiment – could also help identifying saccades to certain stimuli inside and outside the car [8]. Wearable sensors can monitor heart rate, and therefore can be used to inform traffic peers of medical emergency. Moreover, given the increasing availability of consumer EEG headsets, it is promising that research shows electrophysiological patterns can be extremely helpful as well [22, 23].

Another interesting field of exploration is the study of information transmission and potentially further propagation of data in a vehicle network [17, 62, 63]. This way the risk information is not only locally useful but can change the state of the global network. For example, the network could start organizing detours even when an inevitable accident has not happened yet. On the one hand, creating such a one-directional inter-cognitive link between an artificial and a biological cognitive system is an important step forward from the perspective of the applied field of cognitive infocommunications [31]. On the other hand though, it raises important concerns regarding privacy and security. These systems would monitor the driver’s reactions and while communication is only intended in case of risk, it is still a potential data breach. Moreover, malicious attack is also possible against the automated car by sending large amount of risk notifications. The communication link therefore must be secured. Indeed, current research on intelligent automated traffic, smart cities and situation awareness of self-driving cars is aware of these challenges [17, 35, 64, 65].

Researchers working on self-driving cars say that fully automated cars are still years or even decades ahead [29, 66]. Meanwhile, semi-automatic solutions are increasingly available (automatic parking, highway autopilot) [67, 68]. Thus, roads are becoming more and more a niche of biological and artificial drivers. In this situation we may want artificial cognitive agents to co-evolve with our biological cognitive systems. In the present work we detailed one aspect of this endeavour, namely inter-cognitive warning systems. The core of arguments was the importance of communication of the human drivers’ cognitive and behavioural states to self-driving cars to increase road safety in the future.

Footnotes

Acknowledgments

The research leading to these results has received funding from the European Community’s Research Infrastructure Action – grant agreement VISIONAIR 262044-under the 7 ${}^{\rm th}$ Framework Programme (FP7/ 2007-2013). Á.T. was additionally supported by a Young Researcher Fellowship from the Hungarian Academy of Sciences. The authors would like to thank László Kovács for his valuable comments on an earlier version of this manuscript.

References

Google. Google Self-Driving Car Project. https://static.googleusercontent.com/media/, www.google.com/hu//selfdrivingcar/.2016.

LaFrance

. When Google Self-Driving Cars Are in Accidents, Humans Are to Blame. Atl2015.

Blanco

Atwood

Russell

Trimble

McClafferty

Perez

. Automated Vehicle Crash Rate Comparison Using Naturalistic Data [Internet]. Vtti. Virginia Tech Transportation Institute; 2016. Available from: http://www.apps.vtti.vt.edu/PDFs/Automated%5CnVehicle%5CnCrash%5CnRate%5CnComparison%5CnUsing%5CnNaturalistic%5CnData_Final%5CnReport_20160107.pdf%5Cn. http://www.vtti.vt.edu/featured/?p=422.

Brumby

Salvucci

Howes

. Focus on driving. Proceedings of the 27th international conference on Human factors in computing systems – CHI 09 [Internet]. New York, New York, USA: ACM Press; 2009 [cited 2 Nov 2014], 1629. Available from: http://dl.acm.org/citation.cfm?id=1518701.1518950.

Spence

Gray

. Looming auditory and vibrotactile collision warning for safe driving. 7th International Driving Symposium on Human Factors in Driver Assessment, Training, and Vehicle Design [Internet]; 2013 [cited 2 Nov 2014]. Available from: http://trid.trb.org/view.aspx?id=1263140.

Török

Tóth

Honbolygó

Csépe

. Integration of warning signals and signaled objects to a multimodal object: A pilot study. 2013 IEEE 4th International Conference on Cognitive Infocommunications (CogInfoCom) [Internet]. IEEE 2013 [cited 18 Sep 2014], 653-8. Available from: http://ieeexplore.ieee.org/articleDetails.jsp?arnumber=6719183.

Reed

Spence

. Multisensory in-car warning signals for collision avoidance. Hum Factors [Internet]. Dec 2007 [cited 2 Oct 2013]; 49(6): 1107-14. Available from: http://www.ncbi.nlm.nih.gov/pubmed/18074709.

Colonius

Diederich

. The Multisensory Driver: Contributions from the Time-Window-of-Integration Model. In: Cacciabue

Hjãlmdahl

Luedtke

Riccioli

, eds. Human Modelling in Assisted Transportation SE – 39 [Internet]. Springer Milan 2011; 363-71. Available from: http://dx.doi.org/10.1007/978-88-470-1821-1_39.

Koren

Borsos

. Is Smeed’s law still valid? A world-wide analysis of the trends in fatality rates. J Soc Transp Traffic Stud2013; 1(1): 64-76.

10.

Mountain

Fawaz

Jarrett

. Accident prediction models for roads with minor junctions. Accid Anal Prev Elsevier1996; 28(6): 695-707.

11.

Abdel-Aty

Radwan

. Modeling traffic accident occurrence and involvement. Accid Anal Prev Elsevier2000; 32(5): 633-42.

12.

Greibe

. Accident prediction models for urban roads. Accid Anal Prev Elsevier2003; 35(2): 273-85.

13.

Yun

Hwang

Han

. A comparative study on accident prediction model using nonlinear regression and artificial neural network, structural equation for rural 4-legged intersection. J Korean Soc Transp Korean Society of Transportation2014; 32(3): 266-79.

14.

Dunyao

Lixin

Pan

. The traffic accident hotspot prediction: Based on the logistic regression method. In: Transportation Information and Safety (ICTIS), International Conference on IEEE, 2015; 107-10.

15.

Xiao

Xie

Tan

. Traffic accident prediction using vehicle tracking and trajectory analysis. In: Intelligent Transportation Systems, Proceedings IEEE, 2003; 220-5.

16.

Xiao

Xie

Tan

Maybank

. Traffic accident prediction using 3-D model-based vehicle tracking. Veh Technol IEEE Trans IEEE2004; 53(3): 677-94.

17.

Jãmsã

Sukuvaara

Luimula

. Vehicle in a cognitive network. Intell Decis Technol IOS Press2015; 9(1): 17-27.

18.

Berger

Rumpe

. Autonomous Driving – 5 years after the urban challenge: The anticipatory vehicle as a cyber-physical system. Proc Inform (September)2012; 789-98.

19.

Yang

. Real-Time Eye, Gaze, and Face Pose Tracking for Monitoring Driver Vigilance. Real-Time Imaging [Internet]. Oct 2002 [cited Sep 17 2014]; 8(5): 357-77. Available from: http://www.sciencedirect.com/science/article/pii/S1077201402902792.

20.

Peng

Guo

Yuan

Wang

. Multi-parameter prediction of drivers’ lane-changing behaviour with neural network model. Appl Ergon Elsevier2015; 50: 207-17.

21.

Ueno

Kaneda

Tsukino

. Development of drowsiness detection system. Proceedings of VNIS’94 – 1994 Vehicle Navigation and Information Systems Conference [Internet]. IEEE 1994 [cited Sep 17 2014]; 15-20. Available from: http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=396873.

22.

Huang

K-C

Huang

T-Y

Chuang

C-H

King

J-T

Wang

Y-K

Lin

C-T

, et al. An EEG-Based Fatigue Detection and Mitigation System. Int J Neural Syst [Internet]. World Scientific 2016; 26(4): 1650018. Available from: http://www.worldscientific.com/doi/10.1142/S0129065716500180.

23.

Wang

Zhang

Shi

Wang

. Real-time EEG-based detection of fatigue driving danger for accident prediction. Int J Neural Syst World Scientific2015; 25(2): 1550002.

24.

Suzuki

Jansson

. An analysis of driver’s steering behaviour during auditory or haptic warnings for the designing of lane departure warning system. JSAE Rev [Internet]. Jan 2003 [cited Sep 17 2014]; 24(1): 65-70. Available from: http://www.sciencedirect.com/science/article/pii/S0389430402002473.

25.

Engström

Johansson

Östlund

. Effects of visual and cognitive load in real and simulated motorway driving. Transp Res Part F Traffic Psychol Behav [Internet]. Mar 2005 [cited Jun 20 2015]; 8(2 SPEC ISS): 97-120. Available from: http://www.sciencedirect.com/science/article/pii/S1369847805000185.

26.

Hoehener

Green

Del Vecchio

. Stochastic hybrid models for predicting the behavior of drivers facing the yellow-light-dilemma. American Control Conference (ACC), IEEE2015; 3348-54.

27.

Kozak

Pohl

Birk

Greenberg

Artz

Blommer

, et al. Evaluation of Lane Departure Warnings for Drowsy Drivers. Proc Hum Factors Ergon Soc Annu Meet [Internet], SAGE Publications Oct 1 2006 [cited 18 Sep 2014]; 50(22): 2400-4. Available from: http://pro.sagepub.com/content/50/22/2400.abstract.

28.

Liu

Song

. Dynamic modeling of driver control strategy of lane-change behavior and trajectory planning for collision prediction dynamic modeling of driver control strategy of lane-change behavior and trajectory planning for collision prediction. Intell Transp Syst IEEE Trans IEEE2012; 13(September): 1138-55.

29.

Waldrop

. Autonomous vehicles: No drivers required. Nature [Internet], 2 Feb 2015 [cited 6 Feb 2015]; 518(7537): 20-3. Available from: http://www.nature.com/news/autonomous-vehicles-no-drivers-required-1.16832?WT.ec_id=NATURE-20150206.

30.

Abe

. Vehicle Handling Dynamics: Theory and Application [Internet]. Elsevier Science 2015 [cited May 28 2016]; 322. Available from: https://books.google.com/books?id=yOzHBQAAQBAJ&pgis=1.

31.

Baranyi

Csapó

. Definition and synergies of cognitive infocommunications. Acta Polytech Hungarica2012; 9(1): 67-83.

32.

Baranyi

Csapo

Sallai

. Cognitive infocommunications (CogInfoCom). Cognitive Infocommunications (CogInfoCom) Springer2015; 1-219.

33.

Miletics

. Human decisions at irregular overtakings. Cognitive Infocommunications (CogInfoCom), 2015 6th IEEE International Conference on IEEE2015; 145-9.

34.

Chen

Ahn

Laval

Zheng

. On the periodicity of traffic oscillations and capacity drop: The role of driver characteristics. Transp Res part B Methodol Pergamon2014; 59: 117-36.

35.

Jãmsã

. Cognitive communication for traffic safety. 5th IEEE International Conference on Cognitive Infocommunications, CogInfoCom – Proceedings IEEE2014; 103-8.

36.

Sepulcre

Gozalvez

Hernandez

. Cooperative vehicle-to-vehicle active safety testing under challenging conditions. Transp Res Part C Emerg Technol [Internet], Jan 2013 [cited Sep 17 2014]; 26: 233-55. Available from: http://www.sciencedirect.com/science/article/pii/S0968090X12001258.

37.

Heikkilã

Kippola

Jãmsã

Nykãnen

Matinmikko

Keskimaula

. Active antenna system for cognitive network enhancement. 5th IEEE Int Conf Cogn Infocommunications, CogInfoCom – Proc IEEE2014; 19-24.

38.

Politis

Brewster

Pollick

. Evaluating multimodal driver displays under varying situational urgency. Proceedings of the 32nd Annual ACM Conference on Human Factors in Computing Systems – CHI ’14 [Internet]. New York, New York, USA: ACM Press, 2014 [cited 15 Oct 2014]; 4067-76. Available from: http://dl.acm.org/citation.cfm?id=2611222.2556988.

39.

Jãmsã

Pieskã

Luimula

. Situation awareness in cognitive transportation systems. Spec Issue Cogn Infocommunications Infocommun J2013; 5(4): 10-6.

40.

Kling

Török

Mestre

Pergandi

J-M

Mallet

Honbolygó

, et al. Effectiveness of warning signals in dual-task driving scenarios. In: Cognitive Science Arena III, 2015.

41.

Hearst

Dumais

Osman

Platt

Scholkopf

. Support vector machines. Intell Syst their Appl IEEE1998; 13(4): 18-28.

42.

Aizerman

Braverman

Rozoner

. Theoretical foundations of the potential function method in pattern recognition learning. Autom Remote Control1964; 25: 821-37.

43.

Cortes

Vapnik

. Support-vector networks. Mach Learn Springer1995; 20(3): 273-97.

44.

Michel

El Kaliouby

. Real time facial expression recognition in video using support vector machines. Proceedings of the 5th international conference on Multimodal interfaces ACM2003; 258-64.

45.

Erfani

Rajasegarar

Karunasekera

Leckie

. High-dimensional and large-scale anomaly detection using a linear one-class SVM with deep learning. Pattern Recognit [Internet]. Oct 2016; 58: 121-34. Available from: http://www.sciencedirect.com/science/article/pii/S0031320316300267.

46.

Steenken

Weber

Colonius

Diederich

. Designing driver assistance systems with crossmodal signals: Multisensory integration rules for saccadic reaction times apply. PLoS One [Internet]. Public Library of Science; May 6 2014; 9(5): e92666. Available from: http://dx.doi.org/10.1371%2Fjournal.pone.0092666.

47.

Cruz-Neira

Sandin

DeFanti

. Surround-screen projection-based virtual reality. In: Proceedings of the 20th Annual Conference on Computer Graphics and Interactive Techniques – SIGGRAPH ’93 [Internet]. New York, New York, USA: ACM Press 1993 [cited Mar 16 2015]; 135-42. Available from: http://dl.acm.org/citation.cfm?id=166117.166134.

48.

Van Rossum

. Python Programming Language. In: USENIX Annual Technical Conference,2007.

49.

McKinney

. Pandas: A Python data analysis library. 2012; 551. Online URL http://pandas.

50.

Pedregosa

Varoquaux

Gramfort

Michel

Thirion

Grisel

, et al. Scikit-learn: Machine learning in Python. J Mach Learn Res JMLR.org2011; 12: 2825-30.

51.

Hunter

. Matplotlib: A 2D graphics environment. Comput Sci Eng2007; 9(3): 90-5.

52.

Schölkopf

Williamson

Smola

Shawe-Taylor

Platt

. Support vector method for novelty detection. NIPS1999; 12: 582-8.

53.

Schölkopf

Smola

. Learning with kernels: Support vector machines, regularization, optimization, and beyond. MIT Press, 2002.

54.

Joachims

. Making large scale SVM learning practical. Universitãt Dortmund, 1999.

55.

Van der Maaten

Hinton

. Visualizing data using t-SNE. J Mach Learn Res2008; 9(2579–2605): 85.

56.

Minh

. Vehicle steering dynamic calculation and simulation. Proc 23rd Symp DAAAM Int Vienna2012; 237-42.

57.

Andrzejewski

Awrejcewicz

. Nonlinear dynamics of a wheeled vehicle. Vol. 10. Springer Science & Business Media, 2006.

58.

Huang

Erdogmus

Pavel

Mathan

Hild

. A framework for rapid visual image search using single-trial brain evoked responses. Neurocomputing2011; 74(12–13): 2041-51.

59.

Hassan

Lambert-Lacroix

Pasqualini

. Real-time fault detection in semiconductor using one-class support vector machines. Int J Comput Theory Eng IACSIT Press2015; 7(3): 191.

60.

Cotter

Shalev-Shwartz

Srebro

. Learning optimally sparse support vector machines. In: ICML, 2013; 266-74.

61.

Lloyd

. In Touch with the Future: The Sense of Touch from Cognitive Neuroscience to Virtual Reality. Presence Teleoperators Virtual Environ [Internet]. The MIT Press; Aug 4 2014 [cited Sep 10 2014]; 23(2): 226-7. Available from: http://www.mitpressjournals.org/doi/abs/10.1162/PRES_r_00182?journalCode=pres#.VBAm2vl_uMg.

62.

Karsai

Kivelã

Pan

Kaski

Kertész

Barabási

A-L

, et al. Small but slow world: How network topology and burstiness slow down spreading. Phys Rev E APS2011; 83(2): 25102.

63.

Wang

González

Hidalgo

Barabási

A-L

. Understanding the spreading patterns of mobile phone viruses. Science, American Association for the Advancement of Science2009; 324(5930): 1071-6.

64.

Gerla

Lee

E-K

Pau

Lee

. Internet of vehicles: From intelligent grid to autonomous cars and vehicular clouds. In: Internet of Things (WF-IoT), IEEE World Forum on 2014; 241-6.

65.

Hubaux

J-P

Capkun

Luo

. The security and privacy of smart vehicles. IEEE Secur Priv Mag2004; 2(LCA-ARTICLE-2004-007): 49-55.

66.

Urmson

. Google Self-Driving Car Project. SXSW Interactive2016. https://www.youtube.com/watch?v=Uj-rK8V-rik.

67.

Koo

Kwac

Steinert

Leifer

Nass

. Why did my car just do that? Explaining semi-autonomous driving actions to improve driver understanding, trust, and performance. Int J Interact Des Manuf Springer2015; 9(4): 269-75.

68.

Mok

BK-J

Johns

Lee

Ive

Miller

. Timing of unstructured transitions of control in automated driving. In: Intelligent Vehicles Symposium (IV), IEEE, 2015; 1167-72.