Abstract
Objective:
The aim of this study was to investigate the effects of task-induced fatigue in prolonged conditional automated driving on takeover performance.
Background:
In conditional automated driving, the driver can engage in non–driving related tasks (NDRTs) and does not have to monitor the system and the driving environment. In the event that the system hits its limits, the human driver must regain control of the car. To ensure safety, adequate driver fallback performance is necessary. Effects of the drivers’ state and the engagement in NDRTs need to be investigated.
Method:
Seventy-three participants experienced prolonged conditional automated rides and simultaneously had to engage in either an activating quiz or a fatiguing monitoring task (between subjects). After 50 minutes, a takeover situation occurred, and participants had to regain control of the car.
Results:
Prolonged conditional automated driving and simultaneously engaging in NDRTs affected the driver’s state and the takeover performance of the participants. Takeover performance was impaired when participants had to deal with monotonous NDRTs.
Conclusion:
An engagement in monotonous monitoring tasks in conditional automated driving affects the driver’s state and takeover performance when it comes to takeover situations. Especially in prolonged automated driving, an adequate driver state seems to be necessary for safety reasons.
Application:
The results of this study demonstrate that engagement in monotonous NDRTs while driving conditionally automated may negatively affect takeover performance. A monitoring of the driver state and adapted assistance in a takeover situation seems to be a good opportunity to ensure safety.
Keywords
Introduction
Advanced driver assistance systems (ADAS) support the driver in the driving task. Referring to the norm J3016 (SAE International, 2016) today’s ADAS represent SAE Type 2, partial driving automation. In this type of automation, the system executes steering and acceleration or deceleration. The human driver is required to monitor the driving environment.
In SAE Type 3, conditional driving automation, the system performs all aspects of the dynamic driving task with the expectation that the human driver will respond appropriately to a request to intervene (RtI) if necessary. When conditional automated driving (CAD) is activated, the driver can engage in non–driving related tasks (NDRTs). However, if the system detects a situation that it cannot handle, an RtI is executed, and the driver has to regain control of the vehicle.
CAD for roadway travel (e.g., on a highway or Autobahn) is expected to be introduced to the market within the next few years (Belz et al., 2017). As the human driver represents the fallback performance when it comes to an RtI, such takeover situations are currently the focus of research.
The available time budget for regaining control when it comes to an RtI depends on the specific situation: A distinction is made between long-term and short-term transitions (Gold, Naujoks, Radlmayr, Bellem, & Jarosch, 2018). Unlike the long-term takeover situations, when it comes to a short-term takeover situation, the driver has to react within seconds. Referring to Petermann-Stock, Hackenberg, Muhr, and Mergl (2013), the resulting time budget for the transition process should be between 5 s and 10 s.
In previous studies on takeover performance, drivers were often confronted with a large number of takeover situations during the short duration of the experiment (see e.g., Gold, Körber, Lechner, & Bengler, 2016; Hergeth, Lorenz, Krems, & Toenert, 2015; Kerschbaum, Lorenz, & Bengler, 2014). An accumulation of takeover situations in these experiments leads to training effects and attentive drivers. In reality, when these systems are available on road, the duration of automated driving can be increased and the drivers less attentive. Fridman et al. (2017) recommends a larger temporal window and examining entire trips to understand how humans interact with autonomous vehicles.
Prolonged times of automation bring up the concerns expressed by Bainbridge (1983) in “Ironies of Automation.” She claims that “the more advanced a control system is, so the more crucial may be the contribution of the human operator” (p. 775). Parasuraman and Riley (1997) characterize humans to be poor supervisors in the context of automation. With introduction of CAD, the human driver will switch his or her role from an active operator to a passenger who just has to intervene if requested by the system. Effects occurring due to prolonged supervising of automated systems were investigated in the past. Known effects are increased cognitive load and a performance decline in monitoring tasks. In an experiment conducted by Kessel and Wickens (1982), participants either had to control or monitor a system to detect changes in system dynamics. Participants who had to control the system were more accurate and faster compared to those who had to monitor it. Mackworth (1948) found that efficiency in a signal detection task, which was an irregular movement of a pointer, was lower when the subjects had been doing the task for about half an hour. In the following 1.5 hours of the experiment, the efficiency continued to decline.
According to May and Baldwin (2009), another problem in CAD may be emerging fatigue. Increasing automation and monotony are two known causations for fatigue. In a study conducted by Jamson, Merat, Carsten, and Lai (2013), symptoms of fatigue increased when the drivers drove automated compared to driving manually.
In manual driving, fatigue is associated with impaired driving performance. Transferred to conditional automated driving, fatigue may affect takeover performance and increase the crash risk when taking over control. Merat, Jamson, Lai, and Carsten (2012) found that driving performance was impaired after a period of underload condition, which is a known cause of passive task-related fatigue imposed by the vehicle automation.
Therefore, it is crucial to investigate the effects of prolonged conditional automated driving on the driver’s state and thus takeover performance. Different NDRTs seem to have an effect on the driver’s state and were used to affect the driver’s state during conditional automated driving (Jarosch, Kuhnt, Paradies, & Bengler, 2017; Schömig, Hargutt, Neukum, Petermann-Stock, & Othersen, 2015).
In the study reported in this article, we proposed that two issues be addressed: First, we investigated whether different NDRTs (monotonous vs. activating) affect the driver’s state in prolonged CAD. Second, effects of task-induced fatigue on takeover performance should be assessed.
Theoretical Issues
Fatigue in Manual Driving
In the fatigue model of May and Baldwin (2009), the consequences of fatigue in manual driving are increased crash risk and a decrease in driving performance. The model distinguishes between three different forms of fatigue:
Active task-related fatigue: Caused by increased task load (e.g., high traffic density, poor visibility or engaging in secondary tasks)
Passive task-related fatigue: Caused by underload conditions (e.g., monotony, extended driving periods or increasing automation)
Sleep-related fatigue: Caused by the circadian rhythm and time of day (e.g., sleep deprivation, sleep restriction, and untreated sleep disorders).
Due to severe consequences and the huge number of accidents caused by fatigue in manual driving, much research is focused on the topic. Jewett, Dijk, Kronauer, and Dinges (1999) as well as Saxby, Matthews, Warm, Hitchcock, and Neubauer (2013) demonstrated that fatigue negatively affects the reaction times of drivers. In many other studies, a deterioration in driving performance parameters was detected (e.g., Lenné, Triggs, & Redman, 1997; Matthews & Desmond, 2002; Philip et al., 2005).
Fatigue in CAD
With increasing automation and the introduction of CAD, major changes in the development of fatigue while driving are expected: Causes for active task-related fatigue may be reduced as the car executes the driving task itself, which leads to a lower mental workload for the driver. However, it is also expected that CAD can reinforce aspects of passive task-related fatigue (Neubauer, Matthews, Langheim, & Saxby, 2012). May and Baldwin (2009) mention increasing automation as one of the main causes of passive task-related fatigue.
In general, it is of interest whether the models that have been developed for fatigue and manual driving can be applied to automated driving situations.
In a study conducted by Feldhütter, Hecht, and Bengler (2018), 3 participants fell asleep in 60 min of automated driving when no NDRT was available. In another study conducted by Omae, Fujioka, Hashimoto, and Shimizu (2006), 8 out of 30 participants fell asleep after 1 hr of automated driving when they were told to monitor the system.
Positive effects of a quiz task to countermeasure fatigue in automated driving were reported. In an experiment, participants indicated significantly lower levels of fatigue when they had to deal with a quiz task for 15 minutes (Schömig et al., 2015). The author claims that an interesting and motivating activity has the potential to prevent drivers from experiencing an increase in fatigue. In a study conducted by Jarosch et al. (2017), participants either had to deal with a monotonous monitoring task or an activating quiz task. Significant differences in subjective and objective fatigue were obvious after 25 min of engagement in the NDRTs and automated driving. Percentage of eyelid closure over the pupil over time (PERCLOS), which was used as the objective measurement for fatigue, in the monitoring task group rose up to 11.5% compared to 4% in the quiz task condition. PERCLOS indicates whether the eyelid is closed to more than 80% and is a valid measurement for fatigue (Wierwille, Wreggit, Kirn, Ellsworth, & Fairbanks, 1994). After 25 min, takeover performance measured by time (reaction times) and quality (e.g., longitudinal and lateral acceleration) based metrics was not affected.
In prolonged automated driving, monotony may further increase. Thus, it can be expected that in prolonged automated driving, fatigue will further increase.
Assessment and Measurement of Fatigue
Fatigue can be assessed by using either subjective or objective data. A frequently used method for assessing subjective fatigue is the Karolinska Sleepiness Scale (KSS; Åkerstedt & Gillberg, 1990). Participants state their subjective fatigue on a 9-point Likert scale: extremely alert (1), alert (3), neither alert nor sleepy (5), sleepy—no difficulty remaining awake, (7) and extremely sleepy—fighting sleep (9).
For objectively assessing fatigue, the recording of physiological data (e.g., heart rate, Bishop & Hasser, 1985; heart rate variability, Manzey, 1997; and electroencephalography [EEG], Simon et al., 2011) as well as eyelid-closure-based metrics (PERCLOS; Wierwille et al., 1994), mean blink duration (Schleicher, Galley, Briest, & Galley, 2008), and blink frequency (Dinges & Mallis, 1998) have been frequently used in driving simulator studies.
Hypotheses
The aim of the study was to investigate the effects of two different NDRTs in CAD on driver fatigue. Participants either had to deal with a monotonous monitoring task or an activating quiz task. We hypothesized that (a) an engagement in the monotonous task leads to increased subjective and objective fatigue compared to the activating task and that (b) an activating quiz task can prevent emerging fatigue. The second aim of the study was to investigate the effects of fatigue provoked through the monotonous task on takeover performance. We expected that (c) fatigue leads to an impaired takeover performance when it comes to an RtI.
Materials and Methods
Participants
Seventy-three employees of the BMW Group voluntarily participated in the study. Of those, n = 4 participants were excluded from analysis due to simulator problems (e.g., no RtI). Another n = 2 participants were excluded because of missing eye-tracking data. One participant was excluded as he did not engage in the NDRT and was obviously waiting for a takeover situation, resulting in a sample size of 66. The sample consisted of 14 female (21.21%) and 52 male (78.78%) participants. The mean age of the participants was 31.36 years (SD = 9.86 years, maximum = 60 years, minimum = 20 years). The subjects were drivers with a mean driving experience of 13.83 years (SD = 9.68 years). The sample quoted their driving experience as “experienced” (83.33%) or “very experienced” (15.15%). Just 1 participant was “inexperienced” (1.5%) with driving.
Most participants had already experienced at least one driver assistance system (83.56%). Table 1 shows the experience with assistance systems for the two conditions and results of chi-square analysis indicating that there were no differences in their experiences with ADAS between the two NDRT groups before.
Experiences of the Participants With Driver Assistance Systems
Note. ACC = adaptive cruise control.
Before the experimental ride, the majority of participants indicated being very awake to awake (60.6%). A state between awake and tired, without problems staying awake was indicated by 34.85%, and 4.54% reported they were tired, without problems staying awake. KSS did not differ between the two conditions before the experimental ride, with M = 3.55 (SD = 1.52) for the Pqpd task and M = 3.72 (SD = 1.4) for the quiz task, t(64) = −.48, p = .63.
Apparatus
The study was conducted in a motion-based driving simulator at BMW Group laboratories (see Figure 1). Nine visual channels are rendered at 60 frames/s, predominantly at a resolution of 1920 × 1200. Seven forward channels provide a horizontal field of view of 240° × 45°. The two rear channels, providing the same resolution, can be seen through the side mirrors. The simulator uses a hexapod system with 6 degrees of freedom. Maximum translation is ±1.2 m and rotation ±30°. Maximum acceleration is 7 m/s².
Motion-based driving simulator (left) and vehicle interior with mounted Windows Surface tablet (right).
A BMW 520i touring with automatic transmission was used as the mockup. All necessary instrumentation was identical to a production car. Realistic steering torque was simulated by using a servo-motor and a steering model.
A Dikablis 3.0 head-mounted eye-tracker was used for measuring PERCLOS.
The NDRTs were presented on a Windows Surface tablet (10.8 in.) mounted in front of the central information display (CID; see Figure 1).
Experimental Setup
Five participants were tested per day (8:15 a.m., 10:15 a.m., 1:00 p.m., 3:00 p.m., and 5:00 p.m.). Each trial took about 1:45 hr. Upon arrival, participants were given a written description of the CAD. They were informed about the characteristics of the system as well as system boundaries including the possibility of RtIs. A confidentiality statement and a demographic form with questions about driving experience, age, gender, and experience with ADAS was filled out. After that, the head-mounted eye-tracker was calibrated for each participant.
This was followed by a familiarization ride to accustom participants with the driving simulator. In this session, participants first had to drive manually and then had to activate the CAD. After 2 min of automated driving, the examiner verbally explained that RtIs can occur. After this, an RtI was released, and participants had to take over control of the car. Thereupon, participants had to activate the system of conditional automated driving two more times and take control of the system by braking and pushing the button on the steering wheel. According to Hergeth, Lorenz, and Krems (2017), prior familiarization with the system of CAD and its system boundaries strongly influences takeover performance. Because some participants had previously participated in CAD experiments, the familiarization was necessary to get comparable data.
The training session was followed by the experimental ride. In the experimental ride, drivers had to deal with an NDRT for the entire ride. After 50 minutes of automated driving, the takeover situation occurred.
This research complied with the American Psychological Association Code of Ethics.
NDRTs
NDRTs were used to affect the driver’s fatigue level. Participants either had to deal with a monotonous monitoring task (Pqpd task), which should induce task-related fatigue, or an activating quiz task, which should prevent fatigue. Participants were told that the processing of the task has priority during CAD.
Both tasks were presented on a Windows Surface tablet computer (10.8 in) mounted in front of the CID and were visually and manually demanding. The NDRTs automatically started with activation of the CAD and stopped with the RtI (black screen).
In the Pqpd task, the letters P, q, p, and d were randomly presented for a variable amount of time between 10 s and 15 s. Whenever the p was presented, participants had to react and touch the screen. On average, a p was presented once a minute. In a study by Jarosch et al. (2017), this task induced significantly higher subjective and objective fatigue compared to a quiz task after 25 min of CAD.
The quiz task looked similar to popular quiz applications like Quizduell or Quizup. Correct or incorrect answers were highlighted green or red. To increase motivation, a total score was displayed. A quiz task had been used previously and successfully to prevent fatigue in CAD in driving simulator studies (Jarosch et al., 2017; Schömig et al., 2015). See Figure 2 for the two NDRTs. While the Pqpd task should represent a monitoring of the automated driving system, the quiz task was used to represent a task that the driver must constantly respond to.
Non–driving related tasks: Quiz and Pqpd task.
Specifics of Conditional Automated Driving
Participants were told to switch on the CAD after entering the highway and drive automated for the entire ride. The CAD could be activated by pressing a button on the steering wheel. By pressing the button, the car took over longitudinal and lateral control, and the drivers had to take hands off the steering wheel and feet from the pedals. The target speed of the system was set to 130 km/h (80.8 mph). Slower driving vehicles were overtaken by the system automatically.
The system could be deactived by pressing the button again or by a braking input. Once deactivated, the vehicle would decelerate with drag torque and drift in the current steering wheel direction. If the system detects a situation it cannot handle, an RtI is indicated.
Human-Machine Interface
The state of the CAD system was displayed in the vehicle’s main instrument cluster. Three different states could be displayed: When conditional automation was available, a grey steering wheel and the notice “Autopilot verfügbar” (English: Autopilot available) was shown. With activation of the system for conditional automated driving, the steering wheel icon in the main instrument cluster turned blue and signaled “Autopilot aktiv” (Autopilot activated). In case of an RtI, the steering wheel icon in the main instrument cluster turned red and showed “Bitte manuell fahren” (Please drive manually). Additionally, two hands grabbing the steering wheel were displayed (see Figure 3). An auditory signal was simultaneously presented, and the NDRT on the Windows Surface was stopped (black screen).
Human-machine interface of the conditional automated driving (from left: available, activated, request to intervene).
Scenario
The experimental ride was conducted on a three-lane highway with a hard shoulder. Traffic density was low, and guidance was mostly straight. There were two elongated curves and hardly any overtaking situations during the ride. Weather conditions were set to a clouded sky to create an ideally monotonous situation.
After 50 min of automated driving, a takeover scenario occurred. For testing of human performance in takeover situations, Gold et al. (2016) recommends high urgency, low predictability, high criticality, and medium to high complexity for the scenario.
The takeover was requested due to an unpredictable accident on the lane of the ego vehicle. A hard shoulder is located to the right of the ego vehicle. Left of the car, there are two additional lanes with two cars in the lane left of the ego vehicle. As there was a 100-m gap between the two vehicles, a braking or lane-change maneuver was possible to avoid an accident.
At the time of the RtI, the car drove 130 km/h (80.8 mph), and time to collision (ttc) was highly urgent at 7 s. In the event that the driver did not intervene, the system would not deactivate and crash into the accident. Thus, the scenario is highly critical.
Dependent Variables
Effects on the driver’s state
Different metrics of the driver’s state were recorded over the course of the experiment. As objective data, PERCLOS was tracked at 1-min intervals at four defined points of time (see Table 2) over the course of the experimental ride. The subjective fatigue (KSS) of the participants was also assessed at four defined moments of time. To not affect PERCLOS, KSS was verbally assessed after the PERCLOS measurement. According to Schmidt, Schrauf, Simon, Buchner, and Kincses (2011), a verbal assessment does not affect objective fatigue in the long term. In a study, measurements of objective fatigue returned to their pre-communication level after 2 min after a 1-min communication (Schmidt et al., 2011). In the current study, the one-item assessment was explicitly shorter. In the takeover situation, no KSS was assessed.
Procedure of the Experimental Ride.
Note. t = time; PERCLOS = percentage of eyelid closure over the pupil over time; KSS = Karolinska Sleepiness Scale. 
= stands for measurement (PERCLOS and afterwards KSS).
Effects on takeover performance
To measure the performance of the drivers in the takeover situation, time and quality aspects of the driver’s intervention were recorded. Reaction times can be measured for different sections of the takeover process. In the current experiment, the following reaction times and quality aspects were identified (see Table 3).
Dependent Variables for Takeover Performance
Results
The main objective of the study was to examine how engagement in NDRTs during prolonged CAD affects the fatigue state of the human driver over the course of the ride and how task-induced fatigue affects takeover performance.
Effects of Prolonged CAD and the Engagement in NDRTs on the Driver’s State
To examine the effects of the prolonged conditional automated ride in combination with the execution of different NDRTs, PERCLOS and KSS were evaluated using mixed ANOVAs. The factor time of measurement was used as the within-subjects factor, and the NDRT was used as the between-subjects factor. To evaluate the driver’s state dependent on the NDRTs, times of measurement (t1, t2, t3, and t4) before the RtI were examined.
Effects on PERCLOS
PERCLOS was highest in the Pqpd task group at t4 (M = 10.47, SD = 12.57) and lowest in the quiz task group at t1 (M = 2.83, SD = 2.39).
A mixed ANOVA with Greenhouse-Geisser correction showed that there was a statistically significant main effect for the factor time of measurement on PERCLOS, F(2.02, 129.15) = 8.47, p < .001, partial η2 = .12. There was also a significant effect of the NDRT, F(1, 64) = 7.95, p = .01, η2 = .11. Next to the main effects, there was also a significant interaction effect between time of measurement and the NDRT, F(2.02, 129.15) = 3.15, p < .05, partial η2 = .05. Thus, fatigue in the Pqpd task group increased faster and more intensively compared to the quiz task group. See Figure 4 for further information.
Percentage of eyelid closure over the pupil over time (PERCLOS) over the course of the ride for the two tasks.
Effects on KSS
KSS was highest in the Pqpd task group at t4 (M = 5.64, SD = 1.93) and lowest in the quiz task group at t1 (M = 3.15, SD = 1.25).
A mixed ANOVA with Huynh-Feldt correction showed that there was a statistically significant main effect for the factor time of measurement on KSS, F(2.53, 161.61) = 61.50, p < .001, partial η2 = .49. There was no significant effect of the NDRT on the KSS, F(1, 64) = 2.14, p = .15, η2 = .03. There was also no significant interaction effect between the time of measurement and the NDRT, F(2.53, 161.61) = 1.32, p = .27, partial η2 = .02. See Figure 5.
Karolinska Sleepiness Scale (KSS) over the course of the experiment.
Reaction After the RtI
To investigate effects of the NDRTs on the takeover performance, the driver’s reactions after the RtI were assessed. Due to the large variety of reactions by human drivers, we distinguished between three different types of possible reactions:
braking maneuver: stopping behind the accident (v < 20 km/h [12.4 mph])
lane change maneuver
accidents/losses of control
In Figure 6, the different driver reactions after the RtI are presented. In both NDRT groups, there were n = 33 participants. The difference was not statistically significant (p = .20, Fisher’s exact test).
Drivers’ reactions after the request to intervene.
Accidents occurred in both NDRT groups, whereas n = 4 drivers from the Pqpd task condition compared to n = 2 drivers from the quiz task condition lost control.
Results in Takeover Performance
To evaluate the effects of NDRTs on prolonged conditional automated driving, takeover performance measures after the RtI were analyzed. Different driving performance parameters and reaction times were examined using unpaired t tests. The dependent variable is the particular driving parameter. The two different NDRTs represent the factor of the group.
Driving Performance Parameters
Maximum longitudinal acceleration
Maximum longitudinal acceleration did not differ between the two groups, with the Pqpd group braking a bit stronger (M = −8.69 m/s², SD = 2.1 m/s²) compared with the quiz task group (M = −8.25 m/s², SD = 2.56 m/s²). A t test with Welch correction indicated that there is no statistically significant difference between the two tasks, t(61.70) = −.77, p = .44.
Maximum lateral acceleration
Maximum lateral accelerations did not differ between the two groups, with the quiz group steering a bit stronger (M = 3.56 m/s², SD = 2.79 m/s²) compared with the quiz task group (M = 3.11 m/s², SD = 2.28 m/s²). There was no statistically significant difference between lateral accelerations for the Pqpd group and the quiz group, t(61.59) = −.72, p = .47.
Reaction Times
Significant differences in the reaction times according to the NDRT could be found in the center-of-road fixation time and the first braking reaction. See Table 4 and Figure 7 for further details.
Reaction Times for Segments of the Takeover Process According to Tasks
p < .05. **p < .01.
Reaction times for the takeover process for the two task groups.
Discussion
The main objective of the current study was to examine the effects of the engagement in NDRTs in prolonged conditional automated driving on the driver’s state and how this affects takeover performance.
Therefore, participants either had to deal with a fatiguing monitoring task, which was used to fatigue the drivers, or a quiz task, which was used to prevent drivers from getting fatigued. The monitoring task is quite similar to system supervising of the automated system, whereas the quiz task is more like an activity with the infotainment system or an own device. Another focus of the experiment was whether emerging fatigue due to passive task-related fatigue affects takeover performance.
The manipulation of fatigue by the two NDRTs worked. Participants who had to engage in the monitoring task (Pqpd) showed higher PERCLOS measures compared with the participants who had to engage in the activating quiz task. The results of this experiment suggest that prolonged conditional automated driving and simultaneously engaging in a monitoring task (i.e., monitoring the automated driving system) can lead to fatigue. This may introduce a driver response delay and negatively affect takeover performance after an RtI. This is indicated on the one hand due to the worsened reaction times and on the other hand due to the higher number of accidents and worse takeover performances. A higher mean PERCLOS in the Pqpd task group was associated with slower reaction times after the RtI. The center-of-road fixation time and the first touch of the brake pedal showed significant differences related to the NDRT. Participants that engaged in the activating quiz task focused on the road center earlier compared to the Pqpd group. In addition, the first braking reaction was faster for the quiz group.
A fast glance reaction on the road as well as a fast braking reaction can be advantageous for the following processes in a takeover situation: Surrounding traffic can be detected earlier, situation awareness can be established faster, and with the fast braking reaction, the resulting time budget for the takeover situation increases. Thus, a lane change maneuver became more prevalent in the quiz group compared with the Pqpd group.
Next to deteriorated reaction times, a higher number of accidents after the RtI also occurred when participants had to engage in the monitoring task (n = 4) compared with the quiz task (n = 2). However, the majority of the participants could still avoid an accident after 50 min of automated driving.
As the current study was similar to the study by Jarosch et al. (2017) concerning the two NDRTs, the takeover scenario, and the driving environment and only differed with respect to the duration of automation, these effects may be attributed to the prolonged journey time. In the previous study, where the subjects had to drive conditionally automated and engage in NDRTs for 25 min, no negative effects on takeover performance have been observed.
One possible explanation for this impaired takeover performance in the monitoring task group can be emerged task-related fatigue due to sustained monitoring as opposed to active reasoning in the quiz task group. In the monotonous monitoring task condition, PERCLOS increased significantly over time. When people monitor the system for automated driving, similar effects could occur. According to the fatigue model of May and Baldwin (2009), all forms of fatigue in manual driving lead to a higher crash risk and impaired driving performance. Transferred to CAD, this means that all forms of fatigue lead to a deteriorated takeover performance. This becomes particularly relevant when drivers are already tired when they get into their automated vehicle and then switch on the automated driving system.
The measured PERCLOS values after 50 min were quite similar to the PERCLOS values Jarosch et al. (2017) found after 25 min of conditional automated driving and engaging in NDRTs (Pqpd = 11.5%, quiz = 4%). We suspect that the NDRTs also somehow prevented participants from experiencing extreme fatigue. In this experiment, no participant fell asleep compared to n = 3 participants that fell asleep after 60 min of automated driving and no task engagement (Feldhütter et al., 2018) or n = 8 sleeping participants in a study conducted by Omae et al. (2006).
When considering the KSS ratings over the course of the experiment, no significant difference between the two NDRTs could be found after 50 min. In the study conducted by Jarosch et al. (2017), significant differences in the self-reported KSS were obvious after an engagement in the same NDRTs of 25 min. Participants that had to engage in the activating quiz task reported lower levels of fatigue compared to the participants that had to engage in the monotonous monitoring Pqpd task. Similar results were reported by Schömig et al. (2015), who found an activating effect for an engagement in a quiz task for 15 min. Thus, it can be concluded that any given task is subjectively perceived as boring or fatiguing after a longer period of time.
In the current study, it was obvious that a time budget of 7 s was unfeasible for a part of the sample after prolonged conditional automated ride and simultaneously engaging in an NDRT. Six drivers lost control due to inadequate steering maneuvers, and others could not avoid an accident. These accidents were more prevalent in the monotonous task condition.
A possible explanation for this phenomenon can be a panic reaction due to the RtI. It is assumed that panic reactions let participants respond inadequately. Due to a rush of adrenaline, perception and control of the environment is attenuated (Jamson & Smith, 2003). Such panic reactions either lead to no reaction at all (Muir, Bottomley, & Marrison, 1996) or an overcompensating reaction (Dingus et al., 1998).
What is important to note is that prolonged automated driving led to a deterioration in takeover behavior. Therefore, drivers should increasingly be supported by assistance systems, especially in the case of takeovers after a longer automated driving period. It is also conceivable to be able to support the driver with different manifestations of the human-machine interface.
How the human-machine interface and driver assistance systems can support the human driver in such takeover situations should be examined in future experiments.
Limitations
This study was conducted in a motion-based driving simulator and focused on measuring human performance when it comes to an RtI after prolonged automated driving. Therefore, several assistance systems like lane keep assistance or emergency braking assistance were deactivated when the RtI was presented in order to measure the reaction times and the quality of the driver’s input. This system behavior is not in accordance with the current status of the CAD system or the system information provided in the instrument cluster and/or central information display at BMW. The two NDRTs were chosen to affect driver fatigue and do not represent a natural behavior.
Key Points
In a driving simulator experiment, 73 participants experienced prolonged conditional automated driving while engaging in either an activating or a fatiguing non–driving related task (NDRT).
The driver state (measured with percentage of eyelid closure over the pupil over time and Karolinska Sleepiness Scale) changed over the course of the ride depending on the NDRT.
After 50 min of automated driving, a takeover situation occurred, and participants had to regain control of the car.
Results suggest that reaction times and the takeover reaction can be impaired when task-related fatigue (e.g., induced from monitoring tasks) occurs in conditional automated driving.
Footnotes
Acknowledgements
This work is the result of the joint project Ko-HAF-Cooperative Conditional Automated Driving and was funded by the German Federal Ministry for Economic Affairs and Energy based on a resolution from the German Bundestag.
Oliver Jarosch studied human factors engineering with a focus on driving simulation and automotive engineering at Technische Universität München (TUM). Since February 2016, he has been working as a PhD student at BMW Research, New Technologiesm and Innovations as a researcher in the field of automated driving.
Hanna Bellem earned her master’s degree from Technische Universität Chemnitz in 2013. From 2013 to 2016, she worked as a doctorate student at the TU Chemnitz in cooperation with Daimler AG, and currently, she is employed as a research specialist at Robert Bosch GmbH in Heilbronn, Germany.
Klaus Bengler is the head of the Ergonomics Department at Technische Universität München, and he conducts research in the field of “micro-ergonomics.” The primary focus of his work is human-machine interactions, particularly in the area of driver assistance, software ergonomics, and cooperation between humans and robots.