Takeover Time in Highly Automated Vehicles: Noncritical Transitions to and From Manual Control

Abstract

Objective:

The aim of this study was to review existing research into driver control transitions and to determine the time it takes drivers to resume control from a highly automated vehicle in noncritical scenarios.

Background:

Contemporary research has moved from an inclusive design approach to adhering only to mean/median values when designing control transitions in automated driving. Research into control transitions in highly automated driving has focused on urgent scenarios where drivers are given a relatively short time span to respond to a request to resume manual control. We found a paucity in research into more frequent scenarios for control transitions, such as planned exits from highway systems.

Method:

Twenty-six drivers drove two scenarios with an automated driving feature activated. Drivers were asked to read a newspaper, or to monitor the system, and to relinquish, or resume, control from the automation when prompted by vehicle systems.

Results:

Significantly longer control transition times were found between driving with and without secondary tasks. Control transition times were substantially longer than those reported in the peer-reviewed literature.

Conclusion:

We found that drivers take longer to resume control when under no time pressure compared with that reported in the literature. Moreover, we found that drivers occupied by a secondary task exhibit larger variance and slower responses to requests to resume control. Workload scores implied optimal workload.

Application:

Intra- and interindividual differences need to be accommodated by vehicle manufacturers and policy makers alike to ensure inclusive design of contemporary systems and safety during control transitions.

Keywords

automation automated driving control transitions takeover requests driving performance task regulation

Introduction

Highly automated vehicles are becoming an engineering reality and will become commonplace on our roads in the very near future (Walker, Stanton, & Salmon, 2015). For example, Tesla released its Autopilot feature in 2015, with BMW, Mercedes, and Audi quickly following with similar technologies (Audi, 2014; BMW, 2013; “Intelligent Drive Concept,” 2015). It is a common misconception that these features are “highly automated” when they are in fact classified as conditional driving automation (SAE Level 3; SAE International, 2016). This classification means that they come with limitations, such as that the features may be intended for use only under certain operational design domains, for example, on highways, as well as require driver monitoring and intervention (Stanton, Young, & McCaulder, 1997; Wolterink, Heijenk, & Karagiannis, 2011).

When using a driver assistance system that is able to automate the driving task to such an extent that hands- and feet-free driving is possible (SAE Level 3; SAE International, 2016), the driver becomes decoupled from the operational and tactical levels of control (Michon, 1985; Stanton & Young, 2005), leaving the high-level strategic goals to be dealt with by the driver (until the point of resuming manual control). This system is a form of “driver-initiated automation,” where the driver is in control of when the system becomes engaged or disengaged (Banks & Stanton, 2015, 2016; Lu & de Winter, 2015). Indeed, according to Bainbridge (1983), two of the most important tasks for humans in automated systems are monitoring the system to make sure it performs according to expectations and to be ready to resume control when the automation deviates from expectation (Stanton & Marsden, 1996). Research has shown that vehicle automation has a negative effect on mental workload and situation awareness (Endsley & Kaber, 1999; Kaber & Endsley, 1997; Stanton et al., 1997; Stanton & Young, 2005; Young & Stanton, 2002) and that reaction times increase as the level of automation increases (Young & Stanton, 2007). This effect becomes problematic when the driver is expected to regain control when system limits are exceeded as a result of a sudden automation failure.

Failure-induced transfer of control has been extensively studied (see Desmond, Hancock, & Monette, 1998; Molloy & Parasuraman, 1996; Stanton et al., 1997; Stanton, Young, Walker, Turner, & Randle, 2001; Strand, Nilsson, Karlsson, & Nilsson, 2014; Young & Stanton, 2007). In one failure-induced control-transition scenario, Stanton et al. (1997) found that more than a third of drivers failed to regain control of the vehicle following an automation failure while using adaptive cruise control. Other research has shown that it takes approximately 1 s for a driver manually driving to respond to an unexpected and sudden braking event in traffic (Summala, 2000; Swaroop & Rajagopal, 2001; Wolterink et al., 2011). Young and Stanton (2007) report brake reaction times of 2.13 ± 0.55 s for drivers using adaptive cruise control (SAE Level 1) and brake reaction times of 2.48 ± 0.66 s for drivers with adaptive cruise control and assistive steering (SAE Level 2). By contrasting the results from Young and Stanton (2007), where drivers experienced an automation failure while a lead vehicle suddenly braked, with Summala’s (2000), it seems like it takes an additional 1.1 to 1.5 s to react to sudden events requiring braking while driving with driver assistance automation (SAE Level 1) and partial driving automation (SAE Level 2). This increase, in combination with headways as short as 0.3 s (Willemsen, Stuiver, & Hogema, 2015) and evidence that drivers are poor monitors (Molloy & Parasuraman, 1996), could actually cause accidents.

Evidently, automating the driving task seems to have a detrimental effect on driver reaction time (Young & Stanton, 2007). Therefore, as Cranor (2008) and Eriksson and Stanton (2016) proposed, the driver needs to receive appropriate feedback if he or she is to successfully reenter the driving control loop. Recent research efforts have been made to determine the optimal takeover-request lead time (TORlt; the lead time from a takeover request [TOR] to a critical event, such as a stranded vehicle) and takeover reaction time (TOrt; the time it takes the driver to take back control of the vehicle from the automated system when a TOR has been issued), with times varying from 0 to 30 s for TORlt and 1.14 to 15 s for TOrt, as shown in Table 1. A total of 25 papers reported either TORlt or TOrt and were included in the review (see Table 1).

Table 1:

Papers Included in the Review

Study Number	Authors	TORlt (in seconds)	TOrt (in seconds)	Modality
1	Gold, Korber, Lechner, & Bengler (2016)	7	2.47–3.61	—
2	Körber, Weißgerber, Kalb, Blaschke, & Farid (2015)	3	—	A
3	Louw, Merat, & Jamson (2015)	6.5	2.18–2.47	A
4	Zeeb, Buchner, & Schrauf (2016)	2.5, 4	—	V
5	Damböck, Bengler, Farid, & Tönert (2012)	4, 6, 8	—	A
6	Kerschbaum, Lorenz, & Bengler (2015)	7	2.22–3.09	V, A
7	Belderbos (2015)	10	5.86–5.87	V, A, H
8	Walch, Lange, Baumann, & Weber (2015)	4, 6	1.90–2.75	V, A
9	Lorenz, Kerschbaum, & Schumann (2014)	7	2.86–3.03	V, A
10	Merat, Jamson, Lai, Daly, & Carsten (2014)	0	10–15	—
11	Naujoks, Mai, & Nekum (2014)	—	2.29–6.90	V, A
12	Schömig, Hargutt, Neukum, Petermann-Stock, & Othersen (2015)	12	—	V, A
13	Louw, Kountouriotis, Carsten, & Merat (2015)	3	—	V, A
14	Zeeb, Buchner, & Schrauf (2015)	2.5, 3, 3.5, 12	1.14	V, A
15	Mok et al. (2015)	2, 5, 8	—	A
16	Gold, Lorenz, & Bengler (2014)	5	1.67–2.22	V, A
17	Radlmayr, Gold, Lorenz, Farid, & Bengler (2014)	7	1.55–2.92	V, A
18	Dogan, Deborne, Delhomme, Kemeny, & Jonville (2014)	3	—	V, A
19	Gold, Damböck, Lorenz, & Bengler (2013)	5, 7	2.06–3.65	V, A
20	van den Beukel & van der Voort (2013)	1.5, 2.2, 2.8	—	A
21	Melcher, Rauh, Diederichs, Widlroither, & Bauer (2015)	10	3.42–3.77	V, A, B
22	Naujoks & Nekum (2014)	0, 1, 2, 3, 4	—	V, A
23	Feldhütter, Gold, Schneider, & Bengler (2016)	6	1.88–2.24	A
24	Payre, Cestac, & Delhomme (2016)	2, 30	4.30–8.70	V, A
25	Körber, Gold, Lechner, & Bengler (2016)	7	2.41–3.66	A

Note. TORlt = takeover-request lead time; TOrt = takeover reaction time; A = auditory; V = visual; H = haptic; B = brake jerk.

The review showed that the mean TORlt was 6.37 ± 5.36 s (Figure 1) with a mean reaction time of 2.96 ± 1.96 s. The most frequently used TORlts tended to be 3 s with a mean TOrt of 1.14 ± 0.45 (Studies 2, 13, 14, 18, 22), 4 s with a mean TOrt of 2.05 ± 0.13 (Studies 4, 8, 22), 6 s with a mean TOrt of 2.69 ± 2.21 (Studies 5, 8, 23), and 7 s with a mean TOrt of 3.04 ± 1.6 (Studies 1, 6, 9, 17, 19, 25), as shown in Figure 2.

Figure 1.

The takeover-request lead time used in the reviewed papers. Several papers used a multitude of takeover request lead times and thus contributed on several points of the graph.

Figure 2.

Takeover reaction time averages for all the conditions in the reviewed studies. Some studies had more than one takeover event and are therefore featured multiple times.

TOrts stay fairly consistent, around 2 to 3.5 s, in most control transitions, with a few outliers, as seen in Figure 2. Belderbos (2015); Merat, Jamson, Lai, Daly, and Carsten (2014); Naujoks, Mai, and Nekum (2014); and Payre, Cestac, and Delhomme (2016) show longer TOrt compared with the rest of the reviewed papers. Merat et al. and Naujoks et al. had the control transition initiated without any lead time, whereas Belderbos and Payre et al. did. Merat et al. showed that there is a 10- to 15-s time lag between the disengagement of the automated driving system and resumption of control by the driver. Notably, the control transition was system initiated and lacked a preemptive TOR, which may have caused the increase in TOrt. Similarly, Naujoks et al. observed a 6.9-s TOrt from when a TOR was issued and the automation disconnected until the driver resumed control in situations where automation became unavailable due to missing line markings, the beginning of a work zone, or entering a curve. Based on personal communication with the author, the vehicle would have crossed the lane markings after approximately 13 s and would have reached the faded lane markings approximately 10 s after the TOR. The velocity in Naujoks et al. was 50 km/h, which is fairly slow compared with most other TOR studies, which used speeds over 100 km/h (Studies 1, 3, 4, 6, 9, 10, 12, 14, 17, 19, 21, 23, 24, 25) and may have had an effect on the perceived urgency.

Belderbos (2015) showed TOrts of 5.86 ± 1.57 to 5.87 ± 4.01 when drivers were given a TORlt of 10 s during unsupervised automated driving. Payre et al. (2016) utilized two different TORlts, 2 and 30 s. These TORlts produced significant differences in TOrt, the 2-s TORlt produced a TOrt of 4.3 ± 1.2 s, and the two scenarios that used the 30-s TORlt produced TOrts of 8.7 ± 2.7 s and 6.8 ± 2.5 s, respectively. The shorter TOrt of the two 30-s TOR events occurred after the 2-s emergency TOR and could have been affected by the urgency caused by the short lead time in the preceding, shorter TOR.

Merat et al. (2014) concluded, based on their observed TOrt, that there is a need for a timely and appropriate notification of an imminent control transition. This observation is in line with the current SAE guidelines that state that the driver “is receptive to a request to intervene and responds by performing dynamic driving task fallback in a timely manner” (SAE International, 2016, p. 20). In initial efforts to determine how long in advance the driver needs to be notified before a control transition is initiated, Damböck, Bengler, Farid, and Tönert (2012) and Gold, Damböck, Lorenz, and Bengler (2013) explored a set of TOR lead times. Damböck et al. (2012) utilized three TORlts—4, 6, and 8 s—and found that given an 8-s lead time, drivers did not differ significantly from manual driving. This finding was confirmed by Gold et al. (2013), who reported that drivers need to be warned at least 7 s in advance of a control transition to safely resume control. These findings seem to have been the inspiration for the TORlt of some recent work utilizing timings around 7 s (Studies 1, 6, 9, 17).

A caveat of a number of the reviewed studies is that the lead time given in certain scenarios, such as disappearing lane markings, construction zones, and merging motorway lanes, is surprisingly short, from 0 to 12 s (cf. Table 1), and will likely be longer in on-road use cases (Studies 4, 5, 11, 14, 15, 21). The reason for this difference in lead time is the increasing accuracy of contemporary GPS hardware and associated services, such as Google Maps. Such services are already able to direct lane positioning while driving manually as well as notify drivers of construction zones and alternate, faster routes. Thus, there is no evident gain of having short lead times in such situations.

Authors of several of the studies reviewed have explored the effect of TORs in different critical settings by issuing the TOR immediately preceding a time-critical event (Studies 1, 2, 3, 4, 6, 7, 8, 9, 13, 16, 17, 19, 20, 23, 24, 25). These authors have explored how drivers manage critical situations in terms of driving behavior, workload, and scanning behavior. Although it is of utmost importance to know how quickly a driver can respond to a TOR and what the shortest TOR times are in emergencies, there is a paucity of research exploring the time it takes a driver to resume control in normal, noncritical situations. We argue that if the design of normal, noncritical control transitions are designed based on data obtained in studies utilizing critical situations, there is a risk of unwanted consequences, such as drivers not responding optimally due to too-short lead time (suboptimal responses are acceptable in emergencies as drivers are tasked with avoiding danger), drivers being unable to fully regain situation awareness, and sudden, dramatic increases in workload. Arguably, these consequences should not be present in every transition of control as they pose a safety risk for the driver as well as other road users. Therefore, the aim of this study is to establish driver takeover time in normal traffic situations when, for example, the vehicle is leaving its operational design domain as they will account for most of the situations (Nilsson, 2014; SAE International, 2016). We also explore how TOR takeover time is affected by a nondriving secondary task, which was expected to increase the reaction time (Merat, Jamson, Lai, & Carsten, 2012).

Moreover, none of the papers included in the review mentioned the time it takes drivers to transition from manual to automated driving. Gaining an understanding of the time required to toggle an automated driving system on is important in situations such as entering an area dedicated to automated vehicles or engaging the automated driving mode in preparation for joining a platoon, as proposed by the SARTRE project (Robinson, Chan, & Coelingh, 2010). Therefore, the aim of this study was to establish the time it takes a driver to switch to automated driving when automated driving features become available. Ultimately, this research aims to provide guidance about the lead time required to get the driver back into, and out of, the manual vehicle control loop.

Method

Participants

Twenty-six participants (10 females, 16 males) between 20 and 52 years of age (M = 30.27, SD = 8.52) with a minimum 1 year and an average 10.57 years (SD = 8.61) of driving experience were asked to take part in the trial. Upon recruiting participants, we obtained their informed consent. The study complied with the American Psychological Association Code of Ethics and had been approved by the University of Southampton Ethics Research and Governance Office (ERGO No. 17771).

Equipment

The experiment was carried out in a fixed-based driving simulator located at the University of Southampton. The simulator was a Jaguar XJ 350 with pedal and steering sensors provided by Systems Technology Inc. as part of STISIM Drive® M500W Version 3 (http://www.stisimdrive.com/m500w) providing a projected 140° field of view. Rearview and side mirrors were provided through additional projectors and video cameras. The original Jaguar XJ instrument cluster was replaced with a 10.6-inch Sharp LQ106K1LA01B Laptop LCD panel connected to the computer via a RTMC1B LCD controller board to display computer-generated graphics components for TORs. The default configuration of the instrument cluster is shown in Figure 3.

Figure 3.

The instrument cluster in its default configuration.

When a TOR was issued, the engine speed dial was hidden and the request was shown in its place. The symbol asking for control resumption is shown in Figure 4, and the symbol used to prompt the driver to reengage the automation is shown in Figure 5.

Figure 4.

The takeover-request icon shown on the instrument cluster. The icon was coupled with a computer-generated voice message stating, “Please resume control.”

Figure 5.

The icon shown when the automation becomes available. The icon was coupled with a computer-generated voice message stating, “Automation available.”

The mode-switching human–machine interface was located on a Windows tablet in the center console, consisting of two buttons, used either to engage or to disengage the automation. To enable dynamic disengagement and reengagement of the automation, bespoke algorithms were developed and are reported elsewhere (cf. Eriksson, Spychala, De Winter, & Stanton, 2016).

Experiment Design

The experiment had a repeated-measures, within-subject design with three conditions: manual, highly automated driving (HAD), and HAD with a secondary task. The conditions were counterbalanced to counteract order effects. For the automated conditions, participants drove at 70 mph on a 30-km, three-lane highway with some curves, with oncoming traffic in the opposing three lanes separated by a barrier and moderate traffic conditions. The route was mirrored between the two automated conditions to reduce familiarity effects while keeping the roadway layout consistent.

In the secondary-task condition, drivers were asked to read (in their head) an issue of National Geographic while the automated driving system was engaged in order to remove them from the driving (and monitoring) task. During both conditions, drivers were prompted to either resume control from or relinquish control to the automated driving system. The control transition requests were presented as both a visual cue (cf. Figure 4 and Figure 5) and an auditory message, in line with previous research (Studies 6, 7, 8, 9, 11, 12, 13, 14, 16, 17, 18, 19, 22, 24), in the form of a computer-generated female voice stating, “Please resume control” or “Automation available.” No haptic feedback was included in this study, despite the findings from Petermeijer, de Winter, and Bengler (2016) and Scott and Gray (2008) showing shorter reaction times when vibrotactile feedback was used. The motivation for excluding the haptic modality was that it was underrepresented in the review, with only one paper in the review utilizing a form of haptic feedback. Furthermore, Petermeijer et al. concluded that haptic feedback is best suited for warnings, and as the current experimental design explored noncritical warnings, no motivation for including haptics could be found. The interval in which these requests were issued ranged from 30 to 45 s, thus allowing for approximately 24 control transitions, of which half were to manual control.

Procedure

Upon arrival, participants were asked to read an information sheet containing information regarding the study and the right to at any point abort their trial without any questions asked. After reading the information sheet, the participants were asked to sign an informed consent form. They were also told that they were able to override any system inputs via the steering wheel, throttle, or brake pedals. Drivers were reminded that they were responsible for the safe operation of the vehicle regardless of its mode (manual or automated) and thus needed to be able to safely resume control in case of failure. This requirement is in accordance with current legislation (United Nations, 1968) and recent amendments to the Vienna Convention of Road Traffic. They were informed that the system may prompt them to either resume or relinquish control of the vehicle and that when such a prompt was issued, they were required to adhere to the instruction but only when they felt safe doing so. This instruction was intended to reduce the pressure on drivers to respond immediately and to reinforce the idea that they were ultimately responsible for safe vehicle operation.

At the end of each driving condition, participants were asked to fill out the NASA-RTLX (Byers, Bittner, & Hill, 1989). They were also offered a short break before continuing the study. Reaction time data were logged for each transition to and from manual control.

Dependent Variables

The following metrics were collected for each condition per participant.

Reaction time to the control transition request was recorded from the onset of the TOR. The control transition request was presented in the instrument cluster coupled with a computer-generated voice to initiate a change in mode to and from manual control and was recorded in milliseconds.

Driving performance was measured by standard deviation of steering angular rate (degrees per second).

Subjective workload scores were collected via the NASA-TLX subscales at the end of each driving condition. Overall workload score was calculated through the summation of subscales (Byers et al., 1989; Hart & Staveland, 1988).

Analysis

The dependent measures were tested for normal distribution using the Kolmogorov-Smirnov test, which revealed that the data were non-normally distributed. To assess driving performance after control was handed back to the driver, a measure of the standard deviation of the absolute steering angular rate was used to capture corrective steering actions (Fisher, Rizzo, Caird, & Lee, 2011, chap. 40, p. 10). Furthermore, as the TOrt data are reaction time data, the median TOrt values for each participant were calculated, after which Wilcoxon signed-rank test was used to analyze the time and workload data. The box plots had their outlier thresholds adjusted to accommodate the log-normal distribution of the TOrt data by using the LIBRA library for MATLAB (Verboven & Hubert, 2005) and its method for robust boxplots for non-normally distributed data by Hubert and Vandervieren (2008). Effect sizes were calculated as r = abs(Z/√N).

Results

The results showed that it took approximately 4.2 to 4.4 ± 1.96 to 1.80 s (median) to switch to automated driving; see Table 2. No significant differences between the two conditions could be found when drivers transitioned from manual to automated driving (Z = −0.673, p = .5, r = .13). Control transition times from manual to automated driving in the two conditions are shown in Figure 6 and the distributions are shown in Figure 7. Individual transition times for each participant are available in the Appendix.

Table 2:

Descriptive Statistics of the Control Transition Times (in milliseconds) From Automated Driving to Manual Control, and From Manual Control to Automated Driving, as Well as Descriptive Statistics From the Presented TOrts From the Reviewed Articles

		From Automated to Manual		From Manual to Automated
Statistic	Metareview	No Secondary Task	Secondary Task	No Secondary Task	Secondary Task
Median	2,470	4,567	6,061	4,200	4,408
IQR	1,415	1,632	2,393	1,964	1,800
Minimum	1,140	1,975	3,179	2,822	2,926
Maximum	15,000	25,750	20,994	23,884	23,221

Note. TOrt = takeover reaction time; IQR = interquartile range.

Figure 6.

Adjusted box plot of control transition times from manual driving to automated driving. The dashed horizontal line indicates the maximum/minimum values assuming a normal distribution.

Figure 7.

A distribution plot of the takeover reaction time when drivers were prompted to engage the automation. The asterisk (*) marks the median value; the x-axis contains 160 bins.

The results showed a significant increase in control transition time of ~1.5 s when drivers were prompted to resume control while engaged in a secondary task (Z = −4.43, p < .01, r = .86). It took drivers approximately 4.46 ± 1.63 s to resume control when not occupied by a secondary task and 6.06 ± 2.39 s to resume control when engaged in a secondary task, as shown in Table 2. The transition times to manual driving in the two task conditions and as reported in the literature are shown in Figure 8, and the distributions of the transition data is shown in Figure 9.

Figure 8.

Adjusted boxplot of the takeover reaction time (TOrt) when switching from automated to manual control in the two experimental conditions contrasted with the TOrt of the reviewed papers.

Figure 9.

A distribution plot of takeover reaction time when drivers were prompted to resume manual control. The asterisk (*) marks the median value; the x-axis contains 160 bins. The amplitude of the reviewed papers is caused by the small number of values provided by the reviewed papers.

The results from analyzing the driving performance data from 0 to 18 s post–control transition showed nonsignificant differences in Standard deviation of absolute steering angular rate (Table 3) between the two task conditions.

Table 3:

Standard Deviation of Angular Rate (in degrees per second) for the Two Task Conditions From 0 to 18 Seconds Post-Takeover

	No Secondary Task	With Secondary Task
Time (seconds)	M (SD)	M (SD)	Z	p	r
0–3	.08 (.06)	.08 (.06)	−0.11	.91	.02
3–6	.03 (.03)	.03 (.03)	−0.14	.89	.03
6–9	.02 (.02)	.06 (.17)	−1.48	.13	.29
9–12	.02 (.02)	.11 (.44)	−0.98	.33	.19
12–15	.02 (.02)	.11 (.45)	−0.39	.69	.08
15–18	.03 (.07)	.13 (.51)	−0.42	.67	.08

The analysis of the subjective ratings for driver mental workload showed that the secondary-task condition has marginally higher scores overall, as shown in Table 4. Only temporal demand had a statistically significant difference (Z = −3.11, p < .05, r = .61), with higher rated demand in the secondary-task condition, as shown in Figure 10.

Table 4:

Overall Workload Scores as Well as Individual Workload Ratings for the Two Conditions

	Without Secondary Task	With Secondary Task
Scale	Median (IQR)	Median (IQR)	Z	p	r
Overall workload	5 (7.33)	6.2 (6.5)	−1.953	.051	.38
Mental demand	7.5 (10)	10.5 (9)	−1.41	.16	.28
Physical demand	4 (5)	5.5 (6)	−1.93	.054	.38
Temporal demand	3 (6)	8.5 (8)	−3.11	.00**	.61
Performance	6 (7)	4.5 (5)	−0.47	.63	.09
Effort	5 (9)	7 (8)	−0.23	.82	.05
Frustration	4 (9)	6.5 (7)	−1.04	.3	.2

Note. IQR = interquartile range.

p < .01.

Figure 10.

Box plot of subjective estimations of workload in the two conditions.

Discussion

Relinquishing Control to Automation

In this study we subjected drivers to multiple control transitions between manual and automated control in a highway scenario. Upon reviewing the literature, we found no mention of how long the driver takes to engage an automated driving system, making this study a first of a kind. We found that drivers take between 2.82 and 23.8 s (Mdn = 4.2–4.4) to engage automated driving when the system indicates that the feature is available. No significant differences between the two conditions were found, but as Figure 6 shows, there was large range in the time it takes to relinquish control. It is clear from Figure 7 that designing for the median or average driver effectively excludes a large part of the user group, which could have severe implications for drivers who fall outside of the mean or median. It has been common practice in human factors and anthropometrics to design for 90% of the population, normally through accommodating the range between the 5th percentile female and the 95th percentile male (Porter, Case, Marshall, Gyi, & Sims, 2004). Thus, it is important that vehicle manufacturers are made aware of the intraindividual differences, as such differences have a large effect on the larger traffic system if drivers are expected to toggle automated driving systems within a certain time frame.

An example of potential situations when the driver would need to toggle the automated driving system could be in HAD-dedicated areas. Moreover, it may be that the time it takes to engage the automated driving system depends on external factors, such as perceived safety, weather conditions, traffic flow rates, presence of vulnerable road users, roadwork, and so on. If the driver deems a situation unsafe, or has doubts as to how well the automation would perform in a situation, the driver may hold off on completing a transition until the driver feels that the system can comfortably handle the situation.

Resuming Control From Automation

Previous research was reviewed and it was found that most studies utilized system-paced transitions, where the automated driving system warns in advance of failure or reduced automation support with relatively short lead times, from 3 s (Studies 2, 13, 14, 18, 22) to 7 s (Studies 1, 6, 9, 17, 19, 25). It has previously been shown that although it takes approximately 2.47 ± 1.42 s on average, it can take up to 15 s to respond to such an event (Merat et al., 2014). We argue that this use case, albeit important, does not reflect the primary use case for control transitions in HAD. When comparing the range of TOrt in the literature to the user-paced (no secondary task) condition in this study, a great deal of overlap can be seen. The observed values in the current study are closer to the higher values observed by Merat et al. (2014), whereas the median range of 4.56 to 6.06 s is closer to the range of times suggested by Gold et al. (2013) and Damböck et al. (2012). It is evident that there is a large spread in the TOrt, which when designing driving automation should be considered, as the range of performance is more important than the median or mean, as these exclude a large portion of drivers.

When subjecting drivers to TORs without time restrictions, we found that drivers take between 1.97 and 25.75 s (Mdn = 4.56) to resume control from automated driving in normal conditions and between 3.17 and 20.99 s (Mdn = 6.06) to do so while engaged in a secondary task preceding the control transition. This finding shows that there is a median 2-s difference in control transition times in the reviewed manuscripts compared with the user-paced control transitions. There was a large effect of secondary-task engagement on TOrt, showing an increase in driver control resumption times when engaged in a secondary task. This finding might be explained in part by the nature of the secondary task, as the driver had to allocate time to put down the magazine he or she was asked to read while the automated driving feature was activated. It could also be partly attributed to driver task adaptation, by holding off transferring control until he or she has had time to switch between the reading task and driving task. This explanation is supported by research indicating that drivers tend to adapt to external factors, such as traffic complexity, to allow for more time to make decisions (Eriksson, Lindström, Seward, Seward, & Kircher, 2014) by, for example, slowing down when engaged in secondary tasks (Cooper, Vladisavljevic, Medeiros-Ward, Martin, & Strayer, 2009) or the expectation of resuming control (Young & Stanton, 2007). In light of these results, there is a case for “adaptive automation” that modulates TORlt by, for example, detecting whether the driver gaze is off road for a certain period and providing the driver with a few additional seconds before resuming control.

Furthermore, the 1.5-s increase of control resumption time when engaged in a reading task is similar to the reaction time increase caused by the introduction of automated driving features observed by Young and Stanton (2007) compared with reaction time in manual driving (Summala, 2000). Therefore, a further increase of reaction times when drivers are engaged in other tasks will have to be expected, but measures must be taken to reduce the increase in reaction time, for example, through the addition of informative displays, to reduce the risk of accidents (Cranor, 2008; Eriksson & Stanton, 2016).

There was a significant increase in perceived temporal demand when drivers were tasked with reading while the automation was engaged. This increase in perceived temporal demand may have been caused by the TOR and the driver not being fully sure as to how long the vehicle could manage before a forced transition would occur (which was not a possibility in the current experiment). This increase in perceived temporal demand could also be attributed to the pace of the experiment and that the drivers were required to pick up, and put down, the magazine whenever a control transition was issued. Overall there are few differences in workload, and the median workload in both conditions was approximately at the halfway point on the scale, implying optimal loading (Stanton, Dunoyer, & Leatherland, 2011).

These results, combined with the nonsignificant, small effect of task condition on driving performance, indicated that the drivers were able to self-regulate the control transition process by adapting the time needed to resume control (Eriksson et al., 2014; Kircher, Larsson, & Hultgren, 2014) and therefore maintaining optimal levels of workload, minimizing the severity of the aftereffects observed in studies by, for example, Gold et al. (2013).

Conclusion

Relinquishing Control to Automation

The literature on control transitions in HAD is absent in research reports on transitions from manual to automated vehicle control. In a first-of-a-kind study, we found that it takes drivers between 2.8 and 23.8 s to switch from manual to automated control. This finding has some implications for the safety of drivers merging into automated-driving-dedicated lanes or other infrastructure while in manual mode. Such an event may require certain adaptations for traffic already occupying such a lane. Adaptations may include increasing time headway or reducing speed to accommodate the natural variance in human behavior to avoid collisions or discomfort for road users in such a lane. Moreover, it may be that part of the variance could be reduced by designing merging zones on straight, uncomplicated road sections, as drivers may otherwise hold off transferring control to the automated driving system until the driver feels it safe to hand control to the automated driving system.

Resuming Control From Automation

A review of the literature showed that most papers tend to report the mean TOrt and often fail to report standard deviation and range (cf. Figure 2); thus the variance in control transition times remains unknown (median and interquartile range for each participant in this study can be found in the Appendix). Additionally, the reviewed papers tended to give drivers a lead time of between 0 and 30 s between the presentation of a TOR and a critical event, with the main part of the reviewed papers using a 3- or 7-s lead time. In this study we found that the range of time in which drivers resume control from the automated driving system was between 1.9 and 25.7 s depending on task engagement. The spread of TOrt in the two conditions in this study indicates that mean or median values do not tell the entire story when it comes to control transitions. Notably, the distribution of TOrt approaches platykurtic (cf. Figure 9) when drivers are engaged in a secondary task. This finding implies that vehicle manufacturers must adapt to the circumstances, providing more time to drivers who are engaged in secondary tasks while in HAD mode to avoid excluding drivers at the tail of the distribution.

In light of this consideration, designers of automated vehicles should not focus on the mean or median driver when it comes to control transition times. Rather, they should strive to include the larger range of control transitions times so they do not exclude users that fall outside the mean or median. Moreover, policy makers should strive to accommodate these inter- and intraindividual differences in their guidelines for “sufficiently comfortable transition times.” When drivers were allowed to self-regulate the control transition process, few differences could be found in both driving performance and workload between the two conditions. This finding lends further support to the argument for designing for the range of transition times rather than the mean or median in noncritical situations.

Last, based on the large decrease in TOrt kurtosis when drivers were engaged in a secondary task, it may also be the case that future automated vehicles need to adapt the TORlt to account for drivers engaged in other, nondriving tasks and even adapt TORlt to accommodate external factors, such as traffic density and weather.

Key Points

Large differences between control transitions times reported in the literature and the no-secondary-task condition were found.

Drivers take longer to resume control from automation when engaged in a secondary task prior to the control transition.

An inclusive design approach is needed to accommodate the observed variance as the mean or median response times are not sufficient when it comes to designing control transitions in automated driving.

Footnotes

Appendix

	From Manual To Automated				From Automated to Manual
	No Secondary Task		Secondary Task		No Secondary Task		Secondary Task
Participant	Median	IQR	Median	IQR	Median	IQR	Median	IQR
1	3531	442	3865	486	3519	404	4811	354
2	3437	453	3854	1423	4748	752	6256	1337
3	7226	6055	4855	964	5554	2720	8267	1743
4	3557	314	4340	1434	5334	2188	9075	2267
5	3433	464	3132	186	3123	381	4904	838
6	4461	397	4461	397	6733	1553	6733	1553
7	6226	929	6249	5805	5809	3011	6924	942
8	3673	666	4014	1000	3998	1317	5484	769
9	4522	465	4288	581	4670	414	6624	710
10	4277	346	4259	994	4509	478	5076	1382
11	8262	2625	8606	7988	8579	1577	11697	2491
12	8998	5370	5474	2459	6012	3382	6716	2263
13	4043	572	4532	1306	4808	854	6559	1080
14	4816	1942	4427	1923	5082	951	5443	710
15	4096	1103	4312	1023	4955	920	7494	2433
16	3299	587	4555	679	4315	554	6226	437
17	4864	2417	5066	4614	3710	1692	6701	2190
18	4467	1433	5612	5880	6836	2996	10136	3650
19	4992	2664	4273	1835	4363	1398	8945	1536
20	5287	2094	4337	1039	6223	1211	6576	1635
21	4052	626	3994	578	3875	202	4925	659
22	3780	463	3908	244	3874	303	5693	527
23	3916	538	4422	592	5025	933	6244	496
24	5663	482	5403	1787	4480	429	5550	1005
25	3915	738	4978	2278	4120	1369	5354	839
26	4772	1126	3528	1094	3151	628	4201	447

Note. Times shown are in milliseconds. IQR = interquartile range.

Acknowledgements

This research has been conducted as a part of the European Marie Curie ITN project HF-Auto: Human Factors of Automated Driving (PITN-GA-2013-605817).

Alexander Eriksson received his MSc in cognitive science from Linköping University in 2014 and is currently a Marie Curie Research Fellow in the EU-funded Marie Curie International Training Network (ITN) on Human Factors in Highly Automated Vehicles (HF-Auto) within the Faculty of Engineering and the Environment at the University of Southampton. His research focus is on human–automation interaction, specifically how automated vehicles hands back control to the driver.

Neville A. Stanton holds the Chair in Human Factors Engineering in the Faculty of Engineering and the Environment at the University of Southampton where he received his DSc in 2013 and was awarded his PhD in Human Factors from Aston University in 1993. He is leading the Engineering and Physical Sciences Research Council/Jaguar Land Rover–funded project Human Interaction: Designing Automated Vehicles (HI:DAVe) and is a partner in the Marie Curie ITN project HF-Auto.

References

Audi. (2014). Audi piloted driving. Retrieved from http://www.audi.com/com/brand/en/vorsprung_durch_technik/content/2014/10/piloted-driving.html

Bainbridge

(1983). Ironies of automation. Automatica, 19, 775–779.

Banks

V. A.

Stanton

N. A.

(2015). Contrasting models of driver behaviour in emergencies using retrospective verbalisations and network analysis. Ergonomics, 58, 1337–1346.

Banks

V. A.

Stanton

N. A.

(2016). Keep the driver in control: Automating automobiles of the future. Applied Ergonomics, 53, 389–395.

Belderbos

(2015). Authority transition interface: A human machine interface for taking over control from a highly automated truck. Delft, Netherlands: Delft University of Technology.

BMW. (2013). ConnectedDrive. Retrieved from http://www.bmw.co.uk/en_GB/topics/ownership/connecteddrive/highly-automated-driving.html

Byers

J. C.

Bittner

Hill

(1989). Traditional and raw task load index (TLX) correlations: Are paired comparisons necessary. In Mital

(Ed.), Advances in industrial ergonomics and safety I (pp. 481–485). London, UK: Taylor & Francis.

Cooper

J. M.

Vladisavljevic

Medeiros-Ward

Martin

P. T.

Strayer

D. L.

(2009). An investigation of driver distraction near the tipping point of traffic flow stability. Human Factors, 51, 261–268.

Cranor

L. F.

(2008, April). A framework for reasoning about the human in the loop. Paper presented at the Conference on Usability, Psychology, and Security, San Francisco, CA.

10.

Damböck

Bengler

Farid

Tönert

(2012). Übernahmezeiten beim hochautomatisierten Fahren [Takeover times for highly automated driving]. Tagung Fahrerassistenz, 15, 16–28.

11.

Desmond

P. A.

Hancock

P. A.

Monette

J. L.

(1998). Fatigue and automation-induced impairments in simulated driving performance. Human Performance, User Information, and Highway Design, 1628, 8–14.

12.

Dogan

Deborne

Delhomme

Kemeny

Jonville

(2014, April). Evaluating the shift of control between driver and vehicle at high automation at low speed: The role of anticipation. Paper presented at the Transport Research Arena 5th Conference: Transport Solutions from Research to Deployment, Paris, France.

13.

Endsley

M. R.

Kaber

D. B.

(1999). Level of automation effects on performance, situation awareness and workload in a dynamic control task. Ergonomics, 42, 462–492.

14.

Eriksson

Lindström

Seward

Kircher

(2014). Can user-paced, menu-free spoken language interfaces improve dual task handling while driving? In Kurosu

(Ed.), Human–computer interaction: Advanced interaction modalities and techniques (pp. 394–405). Berlin, Germany: Springer.

15.

Eriksson

Spychala

De Winter

J. C. F.

Stanton

N. A.

(2016, July). A highly automated driving algorithm implementation on STISIM. Paper presented at the Applied Human Factors and Ergonomics Conference, Orlando. FL.

16.

Eriksson

Stanton

N. A.

(2016, April). The chatty co-driver: A linguistics approach to human-automation-interaction. Paper presented at the Ergonomics and Human Factors 2016 Conference, Staverton Park, Daventry, Northamptonshire.

17.

Feldhütter

Gold

Schneider

Bengler

(2016, March). How the duration of automated driving influences take-over performance and gaze behavior. Paper presented at Arbeit in komplexen Systemen–Digital, vernetzt, human?! 62. Kongress der Gesellschaft für Arbeitswissenschaft, Aachen, Germany.

18.

Fisher

D. L.

Rizzo

Caird

Lee

J. D.

(2011). Handbook of driving simulation for engineering, medicine, and psychology. Boca Raton, FL: CRC Press.

19.

Gold

Damböck

Lorenz

Bengler

(2013). “‘Take over!’ How long does it take to get the driver back into the loop?” In Proceedings of the Human Factors and Ergonomics Society 57th Annual Meeting (pp. 1938–1942). Santa Monica, CA: Human Factors and Ergonomics Society.

20.

Gold

Korber

Lechner

Bengler

(2016). Taking over control from highly automated vehicles in complex traffic situations: The role of traffic density. Human Factors, 58, 642–652.

21.

Gold

Lorenz

Bengler

(2014, June). Influence of automated brake application on take-over situations in highly automated driving scenarios. Paper presented at the FISITA 2014 World Automotive Congress, Maastricht, Netherlands.

22.

Hart

S. G.

Staveland

L. E.

(1988). Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. Advances in Psychology, 52, 139–183.

23.

Hubert

Vandervieren

(2008). An adjusted boxplot for skewed distributions. Computational Statistics & Data Analysis, 52, 5186–5201.

24.

Intelligent drive concept for new Mercedes-Benz GLC. (2015). Safecarnews.com. Retrieved from http://safecarnews.com/intelligent-drive-concept-for-new-mercedes-benz-glc_ju6145/

25.

Kaber

D. B.

Endsley

M. R.

(1997). Out-of-the-loop performance problems and the use of intermediate levels of automation for improved control system functioning and safety. Process Safety Progress, 16, 126–131.

26.

Kerschbaum

Lorenz

Bengler

(2015). A transforming steering wheel for highly automated cars. In 2015 IEEE Intelligent Vehicles Symposium (IV). doi:10.1109/IVS.2015.7225893

27.

Kircher

Larsson

Hultgren

J. A.

(2014). Tactical driving behavior with different levels of automation. IEEE Transactions on Intelligent Transportation Systems, 15, 158–167.

28.

Körber

Gold

Lechner

Bengler

(2016). The influence of age on the take-over of vehicle control in highly automated driving. Transportation Research Part F: Traffic Psychology and Behaviour, 39, 19–32.

29.

Körber

Weißgerber

Kalb

Blaschke

Farid

(2015). Prediction of take-over time in highly automated driving by two psychometric tests. Dyna, 82, 195–201.

30.

Lorenz

Kerschbaum

Schumann

(2014). Designing take over scenarios for automated driving How does augmented reality support the driver to get back into the loop? In Proceedings of the Human Factors and Ergonomics Society 58th Annual Meeting (pp. 1681–1685). Santa Monica, CA: Human Factors and Ergonomics Society.

31.

Louw

Kountouriotis

Carsten

Merat

(2015, November). Driver inattention during vehicle automation: How does driver engagement affect resumption of control? Paper presented at the Fourth International Conference on Driver Distraction and Inattention, Sydney, Australia.

32.

Louw

Merat

Jamson

(2015, June). Engaging with highly automated driving: To be or not to be in the loop? Paper presented at the Eighth International Driving Symposium on Human Factors in Driver Assessment, Training, and Vehicle Design, Salt Lake City, Utah.

33.

de Winter

J. C. F.

(2015). A review and framework of control authority transitions in automated driving. Procedia Manufacturing, 3, 2510–2517.

34.

Melcher

Rauh

Diederichs

Widlroither

Bauer

(2015). Take-over requests for automated driving. Procedia Manufacturing, 3, 2867–2873.

35.

Merat

Jamson

A. H.

Lai

F. C.

Carsten

(2012). Highly automated driving, secondary task performance, and driver state. Human Factors, 54, 762–771.

36.

Merat

Jamson

A. H.

Lai

F. C. H.

Daly

Carsten

O. M. J.

(2014). Transition to manual: Driver behaviour when resuming control from a highly automated vehicle. Transportation Research Part F: Traffic Psychology and Behaviour, 26(Part A), 1–9.

37.

Michon

J. A.

(1985). A critical view of driver behavior models: What do we know, what should we do? In Evans

Schwing

R. C.

(Eds.), Human behavior and traffic safety (pp. 485–519). New York, NY: Plenum Press.

38.

Mok

Johns

Lee

K. J.

Miller

Sirkin

Ive

(2015, September). Emergency, automation off: Unstructured transition timing for distracted drivers of automated vehicles. Paper presented at the 2015 IEEE 18th International Conference on Intelligent Transportation Systems (ITSC), Canary Islands, Spain.

39.

Molloy

Parasuraman

(1996). Monitoring an automated system for a single failure: Vigilance and task complexity effects. Human Factors, 38, 311–322.

40.

Naujoks

Mai

Nekum

(2014, July). The effect of urgency of take-over requests during highly automated driving under distracted conditions. Paper presented at the 5th International Conference on Applied Human Factors and Ergonomics, Krakow, Poland.

41.

Naujoks

Nekum

(2014). Timing of in-vehicle advisory warnings based on cooperative perception. In Proceedings of the Human Factors and Ergonomics Society Europe Chapter Annual Meeting. Available from http://hfes-europe.org

42.

Nilsson

(2014). Safe transitions to manual driving from faulty automated driving system. Gothenburg, Sweden: Chalmers University of Technology.

43.

Payre

Cestac

Delhomme

(2016). Fully automated driving: Impact of trust and practice on manual control recovery. Human Factors, 58, 229–241.

44.

Petermeijer

S. M.

de Winter

J. C. F.

Bengler

K. J.

(2016). Vibrotactile displays: A survey with a view on highly automated driving. IEEE Transactions on Intelligent Transportation Systems, 17, 897–907.

45.

Porter

J. M.

Case

Marshall

Gyi

Sims

(2004). “Beyond Jack and Jill”: Designing for individuals using HADRIAN. International Journal of Industrial Ergonomics, 33, 249–264.

46.

Radlmayr

Gold

Lorenz

Farid

Bengler

(2014). How traffic situations and non-driving related tasks affect the take-over quality in highly automated driving. In Proceedings of the Human Factors and Ergonomics Society 58th Annual Meeting (pp. 2063–2067). Santa Monica, CA: Human Factors and Ergonomics Society.

47.

Robinson

Chan

Coelingh

(2010, October). Operating platoons on public motorways: An introduction to the SARTRE Platooning Programme. Paper presented at the 17th World Congress on Intelligent Transport Systems, Busan, Korea.

48.

SAE International. (2016). Taxonomy and definitions for terms related to driving automation systems for on-road motor vehicles (J3016_201609). Warrendale, PA: Author.

49.

Schömig

Hargutt

Neukum

Petermann-Stock

Othersen

(2015). The interaction between highly automated driving and the development of drowsiness. Procedia Manufacturing, 3, 6652–6659.

50.

Scott

Gray

(2008). A comparison of tactile, visual, and auditory warnings for rear-end collision prevention in simulated driving. Human Factors, 50, 264–275.

51.

Stanton

N. A.

Dunoyer

Leatherland

(2011). Detection of new in-path targets by drivers using stop & go adaptive cruise control. Applied Ergonomics, 42, 592–601.

52.

Stanton

N. A.

Marsden

(1996). From fly-by-wire to drive-by-wire: Safety implications of automation in vehicles. Safety Science, 24, 35–49.

53.

Stanton

N. A.

Young

M. S.

(2005). Driver behaviour with adaptive cruise control. Ergonomics, 48, 1294–1313.

54.

Stanton

N. A.

Young

McCaulder

(1997). Drive-by-wire: The case of driver workload and reclaiming control with adaptive cruise control. Safety Science, 27, 149–159.

55.

Stanton

N. A.

Young

M. S.

Walker

G. H.

Turner

Randle

(2001). Automating the driver’s control tasks. International Journal of Cognitive Ergonomics, 5, 221–236.

56.

Strand

Nilsson

Karlsson

I. C. M.

Nilsson

(2014). Semi-automated versus highly automated driving in critical situations caused by automation failures. Transportation Research Part F: Traffic Psychology and Behaviour, 27(Part B), 218–228.

57.

Summala

(2000). Brake reaction times and driver behavior analysis. Transportation Human Factors, 2, 217–226.

58.

Swaroop

Rajagopal

K. R.

(2001). A review of constant time headway policy for automatic vehicle following. In Proceedings of the IEEE Intelligent Transportation Systems Conference (pp. 65–69). New York, NY: IEEE.

59.

United Nations. (1968). Convention on road traffic: Amendment 1. Retrieved from http://www.unece.org/fileadmin/DAM/trans/conventn/crt1968e.pdf

60.

van den Beukel

A. P.

van der Voort

M. C

. (2013, October). The influence of time-criticality on situation awareness when retrieving human control after automated driving. Paper presented at the 16th International IEEE Conference on Intelligent Transportation Systems, the Hague, Netherlands.

61.

Verboven

Hubert

(2005). LIBRA: A MATLAB library for robust analysis. Chemometrics and Intelligent Laboratory Systems, 75, 127–136.

62.

Walch

Lange

Baumann

Weber

(2015). Autonomous driving: Investigating the feasibility of car-driver handover assistance. In Proceedings of the 7th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (pp. 11–18). New York, NY: ACM.

63.

Walker

G. H.

Stanton

N. A.

Salmon

P. M.

(2015). Human factors in automotive engineering and technology. Aldershot, UK: Ashgate.

64.

Willemsen

Stuiver

Hogema

(2015, June). Automated driving functions giving control back to the driver: A simulator study on driver state dependent strategies. Paper presented at the 24th International Technical Conference on the Enhanced Safety of Vehicles, Gothenburg, Sweden.

65.

Wolterink

W. K.

Heijenk

Karagiannis

(2011, June). Automated merging in a cooperative adaptive cruise control (CACC) system. Paper presented at the Fifth ERCIM Workshop on eMobility, Catalonia, Spain.

66.

Young

M. S.

Stanton

N. A.

(2002). Malleable attentional resources theory: A new explanation for the effects of mental underload on performance. Human Factors, 44, 365–375.

67.

Young

M. S.

Stanton

N. A.

(2007). Back to the future: Brake reaction times for manual and automated vehicles. Ergonomics, 50, 46–58.

68.

Zeeb

Buchner

Schrauf

(2015). What determines the take-over time? An integrated model approach of driver take-over after automated driving. Accident Analysis & Prevention, 78, 212–221.

69.

Zeeb

Buchner

Schrauf

(2016). Is take-over time all that matters? The impact of visual-cognitive load on driver take-over quality after conditionally automated driving. Accident Analysis & Prevention, 92, 230–239.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.19 MB