A Systematic Review and Meta-Analysis of Takeover Performance During Conditionally Automated Driving

Out-of-the-Loop Performance Problem and Time Budget

The out-of-the-loop performance problem is a phenomenon in which operators of automated systems are less capable of performing a task after they take over control of the system compared to manual operators (Endsley & Kiris, 1995). It is likely to occur when drivers are not required to monitor the roadway environment during driving automation (Gold et al., 2013; Körber et al., 2015; Louw et al., 2015). Endsley and Kiris (1995) attribute much of the out-of-the-loop performance problem to a loss of situation awareness. When operators take over control of the system, they must acquire information about the state of the system and environment before making an adequate decision and response, which requires time. In the context of conditionally automated driving, the time given to drivers to take over before reaching the system limit is called the time budget, and it is defined as the time between the moment the TOR is issued and the time at which the vehicle would have reached the system limit assuming no driver response (Gold et al., 2014; Happee et al., 2017).

For drivers to get back in the loop, they will require a sufficient time budget to acquire information on the state of the driving environment before making an adequate driving response to any roadway hazards. Eriksson and Stanton (2017) contend that noncritical takeovers will be more common than urgent takeovers, and for noncritical takeovers, the ADS could predict well in advance when a system limit is approaching. For example, if the ODD of an ADS is limited to a divided highway, the ADS will know that the system limit for this trip is the highway exit and can issue a TOR with ample time to take over (Miller et al., 2015; Petermeijer, Bazilinskyy, et al., 2017). However, the ADS will not always be able to predict a system limit far in advance. For example, debris in the road could reflect a system limit if the ODD assumes good roadway conditions, and the ADS may have to rely on its onboard sensors to detect this (e.g., camera, radio detection and ranging [RADAR], light detection and ranging [LIDAR]; Petermeijer, Bazilinskyy, et al., 2017). In these critical takeover situations, prior research has compared the takeover performance of a relatively shorter time budget (e.g., 5 and 6 s) and a relatively longer time budget (e.g., 7 and 8 s) to determine a minimum time budget required for a safe takeover (Gold et al., 2013; Mok et al., 2017; Wandtner et al., 2018). Whereas some studies found that relatively shorter time budgets were sufficient (Mok, Johns, Lee, et al., 2015), other studies found relatively longer time budgets were needed (Mok et al., 2017). More generally, narrative reviews suggested that only relatively longer time budgets are sufficient (Eriksson & Stanton, 2017; Radlmayr & Bengler, 2015).

Engaging in Non-Driving Related Tasks

Another factor that may affect takeover performance is engagement in non-driving related tasks (NDRTs). A recent meta-analysis found that hands-free cell phone use has moderate performance costs on manual driving (Caird et al., 2018). These driving performance costs putatively occur because the driver’s attention is diverted away from the driving task toward the cell phone conversation (Strayer & Johnston, 2001; Strayer et al., 2003). If drivers engage in NDRTs, such as a cell phone conversation, during conditionally automated driving, their attention would be diverted away from the roadway. A narrative review by de Winter et al. (2014) indicated that as level of automation increased, engagement in NDRTs increased, suggesting that drivers are more willing to engage in NDRTs during conditionally automated driving compared to lower levels of driving automation and manual driving. Although drivers are not required to monitor the roadway during conditionally automated driving, engaging in NDRTs may result in drivers becoming more out of the loop compared with not being engaged in NDRTs, because attention to the roadway environment is further reduced while engaging with NDRTs (Gold et al., 2016; Miller et al., 2015). Additionally, if a driver is engaged in an NDRT when a TOR is issued, the driver must switch tasks from the NDRT to manually controlling the vehicle. Research on task switching revealed a switch cost: responses are slower and more erroneous when switching between tasks (Monsell, 2003). This switch cost is attributed to the process of task set reconfiguration, which involves “shifting attention, retrieving task-specific goals and rules, or inhibiting and clearing out a prior task set” (Zeeb et al., 2017, p. 66). Thus, prior research on distracted driving and task switching suggests that drivers engaged with any NDRT will experience performance decrements during takeovers.

Such decrements may be exacerbated when the NDRT has overlapping resource demands with the driving task, according to multiple resource theory (Wickens, 1980, 2008; Wickens & Hollands, 2000, Chapter 11). NDRTs with a visual input modality will overlap more with roadway monitoring (also visual) than nonvisual NDRTs. In other words, drivers must take their eyes off the roadway during conditionally automated driving to engage in a visual NDRT, leading to reduced situation awareness and poorer takeover performance (Radlmayr et al., 2014). Although monitoring the roadway during conditionally automated driving is not required, the driver could choose to do it, nonetheless. Similarly, if NDRTs continue during the takeover, NDRTs with visual input or manual response demands will overlap with the manual driving task, resulting in poorer takeover performance (Wandtner et al., 2018). In short, multiple resource theory predicts that NDRTs with overlapping resource demands will cause more of a takeover performance decrement than NDRTs without these overlapping resources.

Alternatively, engaging in NDRTs may improve takeover performance. According to malleable attentional resources theory, mental underload leads to performance decrements because the operator’s attentional resources shrink in response to low task demands, leaving the operator unable to cope with sudden critical events (Young & Stanton, 2002). Automated driving has been shown to reduce workload (de Winter et al., 2014) and could reduce workload so much that mental underload is induced, resulting in diminished attentional resources. Consequently, the driver is unable to effectively cope with a takeover. However, engaging in NDRTs could counteract mental underload by increasing workload (Clark & Feng, 2017; Miller et al., 2015). Mental underload can also lead to performance decrements when low mental workload induces drowsiness. When a driver is not engaged in NDRTs during conditionally automated driving, they are likely to become drowsy (Johns et al., 2014; Schömig et al., 2015), but when drivers engage in NDRTs, they exhibit significantly fewer signs of drowsiness (Schömig et al., 2015). In short, according to malleable attentional resources theory, engagement in NDRTs during conditionally automated driving should lead to better takeover performance compared with not engaging with NDRTs.

Takeover Request Information Support

Another factor that may affect takeover performance is the level of information provided to the driver by the human–machine interface when a TOR is issued. Parasuraman et al. (2000) proposed a model in which four stages of human information processing—sensory processing, perception/working memory, decision-making, and response selection—have analogous functions in automated systems: information acquisition, information analysis, decision and action selection, and action implementation, respectively. These classes of functions can be automated and replace the human operator to various degrees, resulting in a continuum of levels of automation (Parasuraman et al., 2000). For conditionally automated driving, all four of these classes of functions are automated by the ADS when automation is engaged, but when a TOR is issued, the ADS will only continue implementing actions until the driver takes over or a limit is reached (e.g., time limit; Eriksson et al., 2019). However, the ADS may still provide information to the driver about the other three classes of functions to facilitate information processing during the takeover (Eriksson et al., 2019). For example, the human–machine interface could highlight the cause of the TOR (information acquisition), indicate whether the adjacent lane is open (information analysis), or recommend a specific action such as braking (decision and action selection). In contrast, a lower level of automation only informs the driver of the need to takeover with an auditory alert (Eriksson et al., 2019). The higher levels of information help the driver process the current roadway environment and gain situation awareness (Eriksson et al., 2019; Lorenz et al., 2014). Thus, higher levels of information support should lead to better takeover performance.

Previous Reviews and Meta-Analyses

There are several narrative reviews on the takeover of vehicle control from an ADS (Eriksson & Stanton, 2017; Körber & Bengler, 2014; Radlmayr & Bengler, 2015; de Winter et al., 2014). These narrative reviews have experts in the field read the relevant studies, summarize the findings of the studies, and make conclusions. However, narrative reviews are limited, because they have a lack of transparency about subjective decisions involved (e.g., what studies to include, how findings are weighted while drawing conclusions; Borenstein et al., 2009, Preface), may have difficulty resolving conflicting results (Rosenthal & DiMatteo, 2001), and often focus on statistical significance, which has often misled conclusions that narrative reviewers make (Rosenthal & DiMatteo, 2001). In contrast, a meta-analysis involves an exhaustive search for eligible studies, quantitative procedures that can resolve inconsistent results, a focus on effect sizes instead of statistical significance, and reporting guidelines that emphasize transparency (Borenstein et al., 2009; Rosenthal & DiMatteo, 2001). Thus, the conclusions of a meta-analysis are more accurate and credible than a narrative review (Rosenthal & DiMatteo, 2001).

Only one meta-analysis currently exists on the effects of conditionally automated driving on takeover performance (Zhang et al., 2019). They found that drivers take over more slowly when provided a longer time budget or when drivers were engaged in NDRTs with visual or manual resource demands. Moreover, they found that a directional TOR (i.e., a TOR with information acquisition) did not affect takeover time. However, Zhang et al.’s meta-analysis has a few notable limitations. First, it included studies that examined the takeover of both level 2 and level 3 driving automation systems and collapsed its meta-analysis across these levels of automation. This is problematic because the driver’s responsibilities are different for these levels while automation is engaged. In particular, the driver is responsible for completing the dynamic driving task by monitoring the driving environment during level 2 driving automation, but the driver is not responsible for monitoring the driving environment or any other part of the dynamic driving task during level 3 driving automation (SAE International, 2018). Consequently, the issues the driver must confront during each level of automation may be different (e.g., vigilance decrement for level 2, out-of-the-loop performance problem for level 3), and Zhang et al.’s results may be harder to interpret because a given effect may differ based on level of automation (e.g., large effect for level 2 but a small effect for level 3). Second, Zhang et al.’s meta-analysis only considered takeover time as a measure of takeover performance even though there are many other measures. This is problematic because takeover time does not indicate how well a takeover is performed (i.e., neither a short nor long takeover time is itself indicative of better takeover performance). Only takeover quality can provide information about how well a takeover is performed (Gold et al., 2014). Prior research has emphasized the need to consider both takeover timing and takeover quality measures to obtain a complete view of takeover performance (Zeeb et al., 2016, 2017). Excluding these other measures in a meta-analysis limits conclusions that can be derived. Third, it is unclear how Zhang et al.’s meta-analysis handled multiple effect sizes from the same study. Such effect sizes introduce dependencies that violate the assumptions of traditional meta-analysis models, which can produce invalid estimates (Fisher & Tipton, 2015; Tanner-Smith et al., 2016).

Current Study

The current meta-analysis improves upon the previous meta-analysis by Zhang et al. (2019), by focusing on automated driving in which the driver is not required to monitor the roadway (i.e., level 3) and including as many takeover timing and quality measures as possible. Three important questions are addressed: (1) Does time budget affect takeover performance? (2) Does engaging in NDRTs affect takeover performance? (3) Does providing the driver with information support when the TOR is issued affect takeover performance? There are four hypotheses relating to the research questions:

Data Sources and Search Strategy

In an initial search, several popular scholarly databases (PsycINFO, PsycArticles, MEDLINE, Web of Science) were queried through October 2017 with the following search terms connected with the OR Boolean operator: “automated driving,” “automated vehicle,” “autonomous driving,” “autonomous vehicle,” “vehicle automation,” and “transition of vehicle control.” For Web of Science, the search results were further narrowed to relevant results by specifying the following categories: ergonomics, psychology applied, behavioral sciences, psychology, psychology experimental, and psychology multidisciplinary. Additionally, targeted journals (The Transportation Research Record: Journal of the Transportation Research Board), conference proceedings (Human Factors and Ergonomics Society; Association for Computing Machinery Special Interest Group on Human–Computer Interaction; International Driving Symposium on Human Factors in Driving Assessment, Training and Vehicle Design; and Transportation Research Board), and government publications (National Highway Traffic Safety Administration; Swedish National Road and Transport Research Institute) were queried with the same search terms to ensure a comprehensive search that included “gray” literature (i.e., original research from government, technical, and research reports that may or may not be peer-reviewed; American Psychological Association, 2020). To be thorough, all the references from the articles that fulfilled the inclusion criteria, regardless of whether they addressed the specific research questions, were considered for inclusion as well. Because of a limited number of relevant articles returned in the initial systematic search, a second systematic search was conducted. The original data sources were queried again with a date restriction of since October 2017 (or only the year 2017 if restricting the search by month was not permitted) through June 2019. Additionally, all the studies included in the meta-analysis by Zhang et al. (2019) were included as an additional source.

Study Selection

In total, 8446 articles were considered for inclusion in the meta-analysis. In the first round, the abstracts from all articles captured by the search were screened using an initial criterion. Specifically, studies were required to examine a transfer of vehicle control of the entire dynamic driving task from an ADS to the human driver following a takeover request. If there was uncertainty as to whether a paper met this initial criterion based on the abstract alone, then that paper was passed on to the next round of review where the full-text was reviewed.

In the second round, the full-text papers were reviewed using an additional set of inclusion criteria. First, nonexperimental research (e.g., surveys, observational studies) were excluded. Second, a study had to have a measure of takeover time or takeover quality following the takeover. Third, papers had to be written in English. Finally, studies were coded into the meta-analysis if they addressed at least one of the research questions. Accordingly, they needed to have at least one of the following manipulations:

When multiple articles used the same data, the most complete one was retained unless the subsequent article provided additional analyses, in which case it was attached to the original article as an addendum. The number of articles that progressed through each round of review is illustrated in Figure 2.

OPEN IN VIEWER

Figure 2

Flow chart of the inclusion and exclusion of articles.

Data Extraction and Coding

The current meta-analysis focused on takeover performance, which can be measured by takeover timing or quality measures. Takeover timing assesses the length of time between the start of a TOR and a specified action by the driver (Kerschbaum et al., 2014). Takeover quality, on the other hand, assesses how well a takeover is performed (Gold et al., 2014). It provides information about how much danger is present during the takeover process (Kerschbaum et al., 2014). As can be seen in Table 1, takeover timing and quality can be divided into smaller measurement categories. For studies eligible for inclusion, takeover performance measures shown in Table 1 were coded.

OPEN IN VIEWER

Table 1

Categories of Takeover Performance Measures Coded in the Meta-Analysis

Category	Definition	Example Measure(s)
Takeover timing a
Gaze reaction time	Time until the first saccade stirs from the NDRT	Gaze reaction time
Road fixation time	Time until the driver’s first glance at the roadway	Road fixation time
Hands-on time	Time until the driver has their hands on the steering wheel	Hands-on time
Takeover time	Time until the driver’s first input	Time to first driver input, time to first braking input, and time to first steering input
Takeover quality
Minimum distance	Smallest physical or temporal distance between ego-vehicle and a forward hazard	Minimum time to collision and minimum headway
Braking magnitude	Degree to which driver reduces longitudinal velocity	Maximum longitudinal deceleration, minimum velocity, and average deceleration
Steering magnitude	Degree to which driver steers to go around an obstacle	Maximum lateral acceleration, maximum steering wheel angle, and maximum lateral velocity
Lane positioning	Variability of the ego-vehicle’s position in a lane	Standard deviation of lane position, standard deviation of steering wheel position, and mean absolute lateral position
Collisions	Number of times driver crashed into another object	Collisions

Note. NDRT = non-driving related tasks. Ego-vehicle refers to the participant’s vehicle.

^aAll takeover timing measures start from the beginning of the TOR.

For the studies that were eligible for inclusion in the meta-analysis, various statistics relevant to the comparison of interest (e.g., mean, standard deviation,) were extracted to calculate the effect sizes, raw mean difference (D), and Cohen’s d (Borenstein et al., 2009). Raw mean difference quantifies the effect size, yet retains the original units. This is especially useful for interpretability when the dependent measure has meaningful units. Cohen’s d also quantifies the effect size but on a standardized scale, which becomes useful when one wants to compare two effect sizes that are calculated from measures not using the same scale. For specific takeover timing measures, both raw mean difference and Cohen’s d are reported because takeover timing is always measured in seconds, an interpretable unit of measure. For takeover quality measures, only Cohen’s d is reported because there are many different units, which can only be combined using a standardized effect size. For all measurement categories except for collisions, raw mean difference and Cohen’s d was calculated for each study and then input into the meta-analysis. However, for collisions, the effect size log odds ratio was calculated for each study because either a collision occurs, or it does not (i.e., binary variable). The log odds ratio from each study was then input into the meta-analysis. The output was converted to Cohen’s d in the reported results for comparability.

When the mean and standard deviations were available, they were used to calculate effect size (Caird et al., 2018; Simmons et al., 2017). If not reported in the text or in a table, estimates of mean and standard deviation were extracted from figures using WebPlotDigitizer (Rohatgi, 2019). If needed, the standard deviation was calculated from the standard error of the mean or confidence intervals of the mean (Cochrane Collaboration, 2011a). If the mean and standard deviation were still not available, multiple attempts were made to contact authors so that they can provide the necessary information. All 51 studies reported statistics in the text or in a table that was used to conduct the meta-analysis. We additionally needed to extract statistics from figures for 18 studies and requested information from the authors of 26 studies. Authors provided the requested information in all but two cases.

Data Analysis

Effect sizes were calculated in R using the formulas from Borenstein et al. (2009, Equations 4.2–4.4, 4.6, 4.12–4.14, 4.16, 4.18–4.21, 4.26, 4.28, 4.29, 5.8–5.11, 7.1, 7.2), which took into account experimental design. When there were at least two studies per measurement category, the calculated effect sizes were meta-analyzed using a random-effects model. Many studies that met the inclusion criteria contributed multiple effect sizes to the same measurement category. When this occurs, the effect sizes are not independent because they come from the same sample of participants. To handle these dependent effect sizes, the robust variance estimation method was employed to meta-analyze the data using the robumeta R package (Fisher & Tipton, 2015; Hedges et al., 2010; Tanner-Smith et al., 2016). This method can handle statistically dependent effect sizes in a meta-analysis without the loss of information that occurs with other methods for handling dependent effect sizes (e.g., creating one aggregate effect size per study). Using this method, mean effect sizes, 95% confidence intervals, and 95% credibility intervals were computed. A mean effect size is the weighted mean of the effect sizes from each study (Borenstein et al., 2009). A 95% confidence interval is a measure of the accuracy of the mean effect size such that the population mean effect size falls within the confidence interval in 95% of cases and indicates whether the estimated population mean effect size is significantly different from zero based on whether zero is captured in the confidence interval (Borenstein et al., 2009). A 95% credibility interval is a measure of the dispersion of effect sizes, such that the true effect in a new study will fall within the credibility interval in 95% of cases (Borenstein et al., 2009). Credibility intervals reflect the heterogeneity between studies and indicate if unique or uncoded moderators are present. Wide credibility intervals suggest the presence of moderators (Schmidt & Hunter, 2015).

Study Characteristics

In total, 48 articles containing 51 experiments were included in the meta-analysis as shown in Table 2. These studies were published between 2013 and 2019 and reported data from 1972 participants ranging in age from 15 to 86 years old (M = 33.59, SD = 14.80). For the studies that reported sex, there were 1139 males and 749 females.

OPEN IN VIEWER

Table 2

Studies Coded in the Meta-Analysis

No.	Study	N	Sample Characteristics a	NDRT b	Group Comparison	Dependent Measures
1	Blommer et al. (2015)	40	Age: 24 < 45 years old <16; Sex: 21 M, 19 F	Watch video, listen to radio	Vis. vs. nonvis. NDRT	Takeover time
2	Borojeni et al. (2016)	21	Age: M(SD) = 26.33 (4.02); Sex: 11 M, 10 F	Spatial 1-back task	Less vs. more information support	Takeover time, minimum distance
3	Borojeni et al. (2017)	9	Age: M(SD) = 28.11 (8); Sex: 3 M, 6 F	Spatial 1-back task	Less vs. more information support	Takeover time, minimum distance
4	Borojeni et al. (2018)	18	Age: 20–35, M(SD) = 27.94 (4.95); Sex: 9 M, 9 F	Complex reading span task	Less vs. more information support	Takeover time, minimum distance, collisions
5	Braunagel et al. (2017)	81	Age: 20–58, M(SD) = 38 (11); Sex: 45 M, 36 F	Read news/ watch video, none	NDRT vs. no NDRT	Takeover time, minimum distance, lane positioning
6	Cohen-Lazry et al. (2017)	16	Age: M(SD) = 25.13 (2.05); Sex: 11 M, 5 F	Game of Simon	Less vs. more information support	Takeover time, collisions
7	Dogan et al. (2019)	38	Age: M(SD) = 37.11 (8.40); Sex: 25 M, 13 F	Write emails, watch videos	Man. vs. nonman. NDRT	Takeover time, minimum distance, braking magnitude, steering magnitude
8	Eriksson et al. (2019)	25	Age: 19–36, M(SD) = 25.7 (3.9); Sex: 14 M, 11 F	Angry Birds	Less vs. more information support	Road fixation, hands-on time, takeover time
9	Feldhütter et al. (2017)	30	Age: 21–28, M(SD) = 24.17 (2.09); Sex: 16 M, 14 F	Surrogate reference task, none	NDRT vs. no NDRT	Takeover time, minimum distance, braking magnitude, steering magnitude
10	Feldhütter et al. (2019)	42	Age: 23–77, M(SD) = 46.0 (20.2); Sex: 27 M, 15 F	Driver’s choice of activity, none	NDRT vs. no NDRT	Takeover time, minimum distance, braking magnitude, steering magnitude
11	Gold et al. (2015)	24	Age: 21–53, M(SD) = 27.67 (7.13); Sex: 15 M, 9 F	Surrogate reference task, fill-in-the-blank text, cognitive-motoric task, 2-back	Vis. vs. nonvis. NDRT, man. vs. nonman. NDRT, vis. man. vs. nonvis. nonman. NDRT	Takeover time, minimum distance, braking magnitude, steering magnitude
12	Gold et al. (2013); Happee et al. (2017)	32	Age: 19–57, M(SD) = 27.6 (8.7); Sex: 24 M, 8 F	Surrogate reference task	Short vs. long time budget	Gaze reaction time, road fixation time, hands-on time, takeover time, braking magnitude
13	Gold et al. (2016)	72	Age: 19–79, M(SD) = 45.0 (22.2); Sex: 58 M, 14 F	20-questions task, none	NDRT vs. no NDRT	Hands-on time, takeover time, minimum distance, braking magnitude, steering magnitude, collisions
14	Ito et al. (2016)	14		Input numbers task	Short vs. long time budget, NDRT vs. no NDRT	Takeover time, collisions
15	Kim and Yang (2017)	30	Age: 20–30; Sex: 20 M, 10 F	Watch videos, none	NDRT vs. no NDRT	Takeover time, braking magnitude, steering magnitude, lane positioning, collisions
16	Kim et al. (2018)	36	Age: 20–50; Sex: 19 M, 17 F	Image search, paced auditory serial addition task, none	NDRT vs. no NDRT, vis. vs. nonvis. NDRT	Road fixation time, hands-on time
17	Ko and Ji (2018) Exp. 2	32	Age: 23–39, M(SD) = 28.22 (4.43); Sex: 19 M, 13 F	Read article, watch video	Man. vs.nonman. NDRT	Road fixation time, hands-on time, takeover time
18	Körber, Gold, et al. (2016); Körber, Radlmayr, et al. (2016)	72	Age: 19–79, M(SD) = 44.98 (22.16); Sex: 58 M, 14 F	20-questions task, none	NDRT vs. no NDRT	Takeover time, minimum distance, braking magnitude, steering magnitude
19	Lorenz et al. (2014)	46	Age: 20–54, M(SD) = 31.7 (10.1); Sex: 36 M, 10 F	Surrogate reference task	Less vs. more information support	Takeover time, braking magnitude, steering magnitude
20	Louw et al. (2015)	16	Age: 19–26, M(SD) = 21 (1.54); Sex: 8 M, 8 F	Read aloud text	NDRT vs. no NDRT	Takeover time, braking magnitude, steering magnitude
21	McNabb (2017) Exp. 4	13	Age: M = 22.4	Watch video	Less vs. more information support	Takeover time
22	Miller et al. (2016)	36	Age: 15–81, M(SD) = 39.5 (25.96); Sex: 17 M, 19 F	Read book, watch movie	Man. vs. nonman. NDRT	Braking magnitude, collisions
23	Mok, Johns, Lee, Miller, et al. (2015)	30	Age: 19–33, M(SD) = 21.67 (2.76); Sex: 15 M, 15 F	Watch video	Short vs. long time budget	Lane positioning, collisions
24	Mok et al. (2017)	30	Age: 17–59, M(SD) = 32.3 (10.8); Sex: 15 M, 15 F	Temple Run 2 (tablet game)	Short vs. long time budget	Lane positioning, collisions
25	Mok, Johns, Lee, Ive, et al. (2015)	27	Age: 19–81, M(SD) = 28.9 (19.1); Sex: 14 M, 13 F	None	Short vs. long time budget	Lane positioning, collisions
26	Naujoks et al. (2019); Berghöfer et al. (2018)	34	Age: 26–78, M(SD) = 55 (14); Sex: 28 M, 6 F	Read magazine, play Tetris, search task, audio book	Vis. vs. nonvis. NDRT, man. vs. nonman. NDRT, vis. man. vs. nonvis. nonman. NDRT	Road fixation, takeover time, lane positioning
27	Payre et al. (2017)	113	Age: 19–70, M(SD) = 40.3 (13.2); Sex: 55 M, 58 F	Solve anagrams/solve labyrinth, none	NDRT vs. no NDRT	Takeover time
28	Petermeijer, Doubek, and de Winter (2017)	101	Age: 17–39, M = 24.7; Sex: 70 M, 31 F	Watch video, 20-questions task, read newspaper	Vis. vs. nonvis. NDRT, man. vs. nonman. NDRT, vis. man. vs. nonvis. nonman. NDRT	Road fixation time, hands-on time, takeover time
29	Petermeijer, Cieler, and de Winter (2017)	18	Age: 22–78, M(SD) = 43.0 (15.2); Sex: 13 M, 5 F	N-back, none	NDRT vs. no NDRT	Hands-on time, takeover time
30	Politis et al. (2018)	49	Age: 17–86, M(SD) = 45.51 (17.36); Sex: 25 M, 24 F	Concentration memory game	Less vs. more information support	Takeover time, lane positioning
31	Radlmayr et al., 2018	57	Age: M(SD) = 23.88 (2.89); Sex: 36 M, 21 F	Balloon popping game	NDRT vs. no NDRT	Takeover time, minimum distance, braking magnitude, steering magnitude, lane positioning
32	Radlmayr et al. (2019)	50	Age: M(SD) = 31.7 (15.6); Sex: 26 M, 24 F	Surrogate reference task, N-back task, shape sorter ball	Vis. vs. nonvis. NDRT, man. vs. nonman. NDRT, vis. man. vs. nonvis. nonman. NDRT	Gaze reaction time, takeover time, minimum distance, braking magnitude, steering magnitude, lane positioning
33	Radlmayr et al. (2014); Happee et al. (2017); Körber, Radlmayr, et al. (2016)	48	Age: M(SD) = 33.5 (9.0); Sex: 38 M, 10 F	Surrogate reference task, N-back task	Vis. vs. nonvis. NDRT, man. vs. nonman. NDRT, vis. man. vs. nonvis. nonman. NDRT	Takeover time, minimum distance, braking magnitude, steering magnitude, collisions
34	Roche et al. (2019)	40	Age: 22–65, M(SD) = 27 (7); Sex: 20 M, 20 F	Questions-and-answers game	Vis. vs. nonvis. NDRT, man. vs. nonman. NDRT, vis. man. vs. nonvis. nonman. NDRT	Takeover time, minimum distance, braking magnitude, steering magnitude
35	Schmidt et al. (2017)	20	Age: 21–61, M(SD) = 44 (12); Sex: 11 M, 9 F	None	Short vs. long time budget	Hands-on time
36	Telpaz et al. (2015); Telpaz et al. (2017)	26	Age: M(SD) = 30 (10.6); Sex: 14 M, 12 F	Texting	Less vs. more information support	Takeover time, braking magnitude, collisions
37	van der Heiden et al. (2017)	24	Age: M(SD) = 32.5 (9.6); Sex: 12 M, 12 F	Video transcription, enter calendar events, none	NDRT vs. no NDRT	Takeover time
38	Vogelpohl et al. (2018)	60	Age: 20–76, M(SD) = 36.0 (15.2); Sex: 38 M, 22 F	Read a magazine, Tetris	NDRT vs. no NDRT	Road fixation time, hands-on time, takeover time, lane positioning
39	Wan and Wu (2018)	36	Age: 18–49, M(SD) = 25.4 (7.5); Sex: 18 M, 18 F	Read, type, play games, watch videos	Man. vs. nonman. NDRT	Takeover time, minimum distance
40	Wandtner (2018) Exp. 1	20	Age: 20–44, M(SD) = 27.6 (6.2); Sex: 10, 10 F	Texting, none	NDRT vs. no NDRT	Hands-on time, takeover time, lane positioning
41	Wandtner (2018) Exp. 2	30	Age: M(SD) = 29.17 (6.38); Sex: 15 M, 15 F	Visual-manual sentence repetition task, visual-vocal sentence repetition task, auditory-vocal sentence repetition task, none	NDRT vs. no NDRT, vis. vs. nonvis. NDRT, man. vs. nonman. NDRT, vis. man. vs. nonvis. nonman. NDRT	Hands-on time, takeover time, minimum distance, collisions
42	Wandtner (2018) Exp. 3	14	Age: M(SD) = 32.0 (10.6); Sex: 9 M, 5 F	Visual-manual sentence repetition task, Visual-manual sentence alphabetization task, auditory-vocal sentence repetition task	Short vs. long time budget, vis. vs. nonvis. NDRT, man. vs. nonman. NDRT, vis. man. vs. nonvis. nonman. NDRT	Hands-on time, takeover time, braking magnitude, collisions
43	Wandtner (2018) Exp. 4	36	Age: 20–57, M(SD) = 34.11 (11.50); Sex: 21 M, 15 F	Visual-manual sentence repetition task, auditory-vocal sentence repetition task	Vis. vs. nonvis. NDRT, man. vs. nonman. NDRT, vis. man. vs. nonvis. nonman. NDRT	Hands-on time, takeover time, braking magnitude, steering magnitude, collisions
44	Wiegand et al. (2018)	20	Age: M = 25.3; Sex: 12 M, 8 F	Watch movie	Less vs. more information support	Steering magnitude
45	Wintersberger et al. (2018)	18	Age: M(SD) = 25.72 (5.69); Sex: 10 M, 8 F	Texting, none	NDRT vs. no NDRT	Takeover time
46	Wright et al. (2017, 2018)	57	Age: M(SD) = 21.17 (2.23)	Read drivers’ manual	Less vs. more information support	Braking magnitude
47	Yang et al. (2018)	50	Age: 18–72, M(SD) = 27.48 (13.09); Sex: 34 M, 16 F	Surrogate reference task	Less vs. more information support	Takeover time, minimum distance, braking magnitude, steering magnitude, lane positioning
48	Yoon and Ji (2019)	27	Age: 24–38, M(SD) = 29.1 (3.78); Sex: 17 M, 10 F	Interact with entertainment console, interact with smartphone, watch video	Man. vs. nonman. NDRT	Road fixation time, hands-on time, takeover time
49	Yoon, Kim, and Ji (2019)	20	Age: 24–42, M(SD) = 30.1 (4.77); Sex: 15 M, 5 F	Phone conversation, interact with smartphone, watch video, none	NDRT vs. no NDRT, vis. vs. nonvis. NDRT, man. vs. nonman. NDRT	Road fixation time, hands-on time, takeover time
50	Zeeb et al. (2016)	79	Age: M(SD) = 39.5 (10.3); Sex: 44 M, 35F	Write email, read news, watch video, none	NDRT vs. no NDRT, man. vs. nonman. NDRT	Road fixation time, hands-on time, takeover time, steering magnitude, lane positioning
51	Zeeb et al. (2017)	95	Age: M(SD) = 38.3 (10.8); Sex: 48 M, 47 F	Read news, none	NDRT vs. no NDRT	Hands-on time, takeover time, minimum distance, lane positioning

Note. Exp. = experiment; F = female; M = male; man. = manual; NDRT = non-driving related task; N = number of participants; vis. = visual.

^aSample characteristics include when available, range, mean, and standard deviation of the participants’ age and the number males and females in the sample.

^bThe visual and manual operator resource demands of NDRTs were coded using the description of the task(s) employed by each study. NDRTs were coded as visual if they required the driver to look at the NDRT for successful completion; otherwise, NDRTs were coded as nonvisual. NDRTs were coded as manual if they required the use of the driver’s hands for successful completion; otherwise, NDRTs were coded as nonmanual.

Synthesis of Results

The results from each comparison, including the number of effects (k), number of independent samples (s), total participants (N), weighted mean of Cohen’s d (d), weighted mean of raw mean difference (D), 95% confidence intervals, and 95% credibility intervals, are shown in Table 3. To interpret Cohen’s d, values of 0.2, 0.5 and 0.8 are used to reflect small, medium, and large effect sizes, respectively (Cohen, 1988).

OPEN IN VIEWER

Table 3

Meta-Analysis of the Effects of Time Budget, Non-Driving Related Task, and Information Support on Takeover Timing and Quality Measures

Variable	k	s	N	d	D	95% CI		95% CrdI
						Lower	Upper	Lower	Upper
Short vs. long time budget
Takeover timing a	48	4	80	−0.46  †		−1.07	0.14	−2.02	1.10
Hands-on time a	11	3	66	−0.39	−0.22	−1.40	0.63	−2.90	2.12
Takeover time a	35	3	60	−0.62	−0.42	−2.44	1.20	−4.30	3.06
Takeover quality	45	6	147	0.17		−0.77	1.11	−3.18	3.52
Braking magnitude a	2	2	46	3.99		−53.55	61.52	−94.20	102.17
Lane positioning a	6	3	87	−0.07		−3.33	3.19	−8.11	7.97
Collisions a	37	5	115	0.49		−0.51	1.50	−0.51	1.50
NDRT vs. no NDRT
Takeover timing	455	20	927	0.38**		0.12	0.64	−1.02	1.78
Road fixation time a	4	2	96	3.75	0.68	−46.59	54.09	−87.17	94.67
Hands-on time	180	9	430	0.34	0.27	−0.12	0.79	−1.11	1.79
Takeover time	271	19	891	0.33**	0.25**	0.13	0.53	−0.74	1.40
Takeover quality	238	14	698	0.29*		0.05	0.54	−1.04	1.63
Minimum distance	63	8	479	−0.20		−0.55	0.14	−1.32	0.91
Braking magnitude	59	7	319	0.17		−0.32	0.67	−1.73	2.08
Steering magnitude	62	8	398	0.06		−0.25	0.36	−0.80	0.91
Lane positioning	16	7	422	0.55**		0.05	1.05	−0.77	1.87
Collisions a	38	4	146	0.19		−0.76	1.15	−0.76	1.15
Visual vs. nonvisual NDRT
Takeover timing	443	12	473	0.47**		0.22	0.72	−0.78	1.73
Road fixation time a	5	2	70	0.83 †	0.77	−0.58	2.23	−11.02	12.67
Hands-on time a	113	5	201	0.50 †	0.29 †	−0.14	1.15	−1.22	2.23
Takeover time	317	11	437	0.30**	0.25**	0.14	0.47	−0.59	1.20
Takeover quality	561	8	276	0.14 †		−0.02	0.30	−1.65	1.92
Minimum distance a	187	5	192	−0.28 †		−0.57	0.00	−2.77	2.20
Braking magnitude a	174	5	198	0.23 †		−0.06	0.52	−0.90	1.36
Steering magnitude a	162	5	198	0.13*		0.01	0.25	−1.68	1.94
Lane positioning a	10	2	84	0.22		−1.57	2.02	−5.76	6.21
Collisions a	28	4	128	0.54		−0.31	1.39	−0.31	1.39
Manual vs. nonmanual NDRT
Takeover timing	527	15	609	0.24 †		−0.02	0.51	−1.15	1.64
Road fixation time	66	6	293	0.24	0.30	−0.62	1.09	−2.03	2.51
Hands-on time	127	8	339	0.28	0.19	−0.24	0.81	−1.33	1.90
Takeover time	326	15	609	0.10	0.11	−0.16	0.36	−1.16	1.36
Takeover quality	614	12	465	0.12 †		−0.02	0.26	−1.17	1.41
Minimum distance	189	7	266	−0.17		−0.43	0.09	−1.83	1.49
Braking magnitude	206	7	272	0.13		−0.09	0.35	−0.77	1.02
Steering magnitude	164	7	315	0.18		−0.06	0.41	−1.30	1.65
Lane positioning a	10	3	163	0.04		−1.23	1.31	−2.55	2.63
Collisions a	45	5	164	0.41*		0.07	0.75	0.07	0.75
Visual, manual vs. nonvisual, nonmanual NDRT
Takeover timing	212	9	377	0.58**		0.25	0.90	−0.83	1.98
Hands-on time a	13	4	181	0.68 †	0.38*	−0.01	1.37	−0.91	2.27
Takeover time	193	9	377	0.38**	0.34**	0.23	0.53	−0.46	1.22
Takeover quality	530	8	276	0.17		−0.05	0.39	−1.65	1.99
Minimum distance a	174	5	192	−0.36*		−0.68	−0.03	−2.90	2.19
Braking magnitude a	168	5	198	0.23  †		−0.04	0.51	−0.87	1.33
Steering magnitude a	156	5	198	0.16*		0.01	0.30	−1.70	2.01
Lane positioning a	5	2	84	0.38		−0.42	1.17	−0.42	1.17
Collisions a	27	4	128	0.56		−0.37	1.48	−0.37	1.48
Less vs. more information support
Takeover timing	63	10	273	0.05		−0.56	0.66	−2.66	2.75
Takeover time	39	10	273	0.05	−0.38	−0.56	0.67	−2.64	2.75
Takeover quality	35	10	312	0.03		−0.35	0.41	−1.31	1.37
Minimum distance a	13	4	98	0.00		−0.81	0.80	−1.62	1.62
Braking magnitude  a	7	4	179	0.04		−0.93	1.01	−1.92	2.00
Steering magnitude a	4	3	116	0.26		−1.25	1.77	−2.50	3.02
Lane positioning a	5	2	99	−0.14		−3.32	3.04	−10.09	9.81
Collisions a	6	3	60	0.36		−0.75	1.46	−0.75	1.46

Note. CI = confidence interval for Cohen’s d; CrdI = credibility interval for Cohen’s d; D = weighted mean of raw mean differences; d = weighted mean of Cohen’s ds; k = number of effect sizes; N = total number of participants; s = number of independent samples. The D column for takeover timing is blank because it combines timing measures with different end points; the D column for takeover quality measures is blank because these measures have different units. For all effect sizes except for takeover quality, a positive effect size indicates a larger value for the first condition (i.e., the condition that appears to the left of vs.) compared to the second condition (i.e., the condition that appears to the right of vs.); for takeover quality, a positive effect size indicates poorer takeover quality for the first group compared to the second group. Effect sizes could only be meta-analyzed when there were at least two independent samples (i.e., studies). When there were less than two independent samples that contributed effect sizes for a particular dependent measure, that dependent measure was omitted from the table. Information about how much a given study influenced the results is reported in the supplementary materials online.

^aThe degrees of freedom using the Satterthwaite approximation was less than four for these analyses. When this occurs, the distribution of the test statistic no longer approximates the t-distribution, and type I error can be larger than implied (Tanner-Smith et al., 2016). In these cases, a stricter α of .01 is used as suggested by Tanner-Smith et al. (2016).

^†p < .10. ^*p < .05. ^**p < .01.

Publication Bias

Publication bias is the phenomenon where studies with statistically significant results are more likely to be published than studies that do not have statistically significant results (Borenstein et al., 2009). Although publication bias in the current meta-analysis was minimized using thorough search methods, its presence was still checked by testing whether sample size (N) could predict effect size (Cohen’s d) using meta-regression within the robust variance estimation framework (Hedges et al., 2010; Tanner-Smith et al., 2016). Given small studies are more vulnerable to not being published because they are less likely to reach statistical significance, a significantly negative relationship is indicative of publication bias because it suggests there are studies with small sample sizes and small effect sizes missing from the published body of literature. This method requires at least 10 studies for a given comparison (Cochrane Collaboration, 2011b). Accordingly, only comparisons with at least 10 studies were assessed using this method. As seen in Table 4, two-tailed t-tests showed the coefficient (b) for sample size was not significantly different from zero in 10 out of the 11 analyzes, meaning sample size did not significantly predict effect size. However, there was a negative trend for takeover quality for the less versus more information support comparison. Altogether, there was little evidence that publication bias was present in the current meta-analysis.

OPEN IN VIEWER

Table 4

Meta-Regression With Sample Size as Predictor

Variable	b	df	t	p
NDRT vs. no NDRT
Takeover timing	0.004	6.86	1.46	.190
Takeover time	0.003	6.80	1.29	.241
Takeover quality	<.001	7.39	0.03	.974
Visual vs. nonvisual NDRT
Takeover timing a	<.001	1.65	−0.05	.968
Takeover time a	<.001	1.67	−0.25	.827
Manual vs. nonmanual NDRT
Takeover timing a	−0.009	2.18	−2.15	.154
Takeover time a	−0.011	2.18	−1.21	.341
Takeover quality a	−0.002	1.91	−0.45	.697
Less vs. more information support
Takeover timing	−0.020	4.25	−0.76	.487
Takeover time	−0.020	4.25	−0.76	.486
Takeover quality	−0.017	5.72	−2.02	.092

Note. NDRT = non-driving related tasks.

^aThe degrees of freedom using the Satterthwaite approximation was less than four for these analyses. When this occurs, the distribution of the test statistic no longer approximates the t-distribution, and type I error can be larger than implied (Tanner-Smith et al., 2016). In these cases, a stricter α of .01 is used as suggested by Tanner-Smith et al. (2016).

Moderator Analysis

Many of the credibility intervals in Table 3 are wide, indicating there are moderators present (Schmidt & Hunter, 2015). Moderators are third variables that affect the relationship between an independent variable and a dependent variable. For example, Hypothesis 2.2 predicts that the relationship between NDRT engagement and takeover performance is affected by NDRT resource demands. Specifically, Hypothesis 2.2 predicts that when the NDRT has overlapping resource demands with the driving task, takeover performance will be worse compared with when the NDRT has no overlapping resource demands. This relationship is tested below. At least 10 studies are needed to test for moderators using meta-regression (Cochrane Collaboration, 2011b), which limited the moderator analyses that could be performed.

For the comparison of engaging in NDRTs versus not engaging in NDRTs, meta-regression was used to test whether effect size (Cohen’s d) was moderated by overlapping resource demands with the driving task (i.e., visual or manual resource demands) for takeover timing, takeover time, and takeover quality. To create the meta-regression model, overlapping resource demands were dummy coded such that NDRTs without overlapping resource demands were coded as the reference category (i.e., the intercept). As shown in Table 5, when the NDRT had overlapping resource demands the effect size (Cohen’s d) trended 0.50 larger for takeover timing and 0.38 larger for takeover time compared with when the NDRT had nonoverlapping resource demands. Takeover quality was not significantly different between conditions.

OPEN IN VIEWER

Table 5

Meta-Regression for the Effect of Overlapping Resource Demands

Variable	k	s	N	d	D	95% CI		95% CrdI
						Lower	Upper	Lower	Upper
Takeover timing
Intercept a	66	5	228	−0.01		−0.44	0.43	−2.08	2.07
Overlapping resource demands	386	15	642	0.50 †		−0.04	1.03	−1.28	2.28
Takeover time
Intercept a	50	4	192	0.02	0.04	−0.42	0.45	−1.87	1.90
Overlapping resource demands a	218	14	606	0.38 †	0.26	−0.09	0.85	−1.13	1.89
Takeover quality
Intercept a	146	3	174	0.02		−0.57	0.61	−4.05	4.09
Overlapping resource demands a	85	10	431	0.34		−0.32	1.01	−2.29	2.97

Note. CI = confidence interval for Cohen’s d; CrdI = credibility interval for Cohen’s d; D = weighted mean of raw mean differences; d = weighted mean of Cohen’s ds; k = number of effect sizes; N = total number of participants; s = number of independent samples. The D column for takeover timing is blank because it combines timing measures with different end points; the D column for takeover quality measures is blank because these measures have different units. For all effect sizes except for takeover quality, a positive effect size indicates a larger value for the first condition (i.e., the condition that appears to the left of vs.) compared to the second condition (i.e., the condition that appears to the right of vs.); for takeover quality, a positive effect size indicates poorer takeover quality for the first group compared to the second group. The intercept reflects nonoverlapping resource demands (reference category).

^a The degrees of freedom using the Satterthwaite approximation was less than four for these analyses. When this occurs, the distribution of the test statistic no longer approximates the t-distribution, and type I error can be larger than implied (Tanner-Smith et al., 2016). In these cases, a stricter α of .01 is used as suggested by Tanner-Smith et al. (2016).

^† p < .10. ^*p < .05. ^**p < .01.

For the comparison of less versus more information support, meta-regression was used to test whether the effect size (Cohen’s d) was moderated by level of higher information support (high information acquisition, information analysis, and decision selection) for takeover timing, takeover time, and takeover quality. To create the meta-regression model, high information acquisition was coded as the reference category (i.e., the intercept), and information analysis and decision selection each had their own dummy variable. As shown in Table 6, high information acquisition was not significantly different from information analysis or decision selection in terms of takeover timing, takeover time, or takeover quality.

OPEN IN VIEWER

Table 6

Meta-Regression for the Effect of Level of Information Support

Variable	k	s	N	d	D	95% CI		95% CrdI
						Lower	Upper	Lower	Upper
Takeover timing
Intercept	30	8	246	0.08		−0.80	0.97	−3.11	3.28
Information analysis a	14	2	71	0.03		−2.98	3.03	−10.86	10.91
Decision selection a	19	3	52	−0.18		−2.55	2.19	−5.03	4.66
Takeover time
Intercept	22	8	246	0.09	−0.70	−0.80	0.97	−3.10	2.38
Information analysis a	6	2	71	0.02	0.79	−2.98	3.03	−10.85	10.88
Decision selection a	11	3	52	−0.18	0.57	−2.55	2.19	−5.01	4.66
Takeover quality
Intercept	19	8	285	−0.04		−0.51	0.43	−1.65	1.57
Information analysis	2	1	46	−0.04		−0.51	0.43	−1.65	1.57
Decision selection a	14	2	27	0.34		−2.66	3.34	−4.15	4.83

Note. CI = confidence interval for Cohen’s d; CrdI = credibility interval for Cohen’s d; D = weighted mean of raw mean differences; d = weighted mean of Cohen’s ds; k = number of effect sizes; N = total number of participants; s = number of independent samples. The D column for takeover timing is blank because it combines timing measures with different end points; the D column for takeover quality measures is blank because these measures have different units. For all effect sizes except for takeover quality, a positive effect size indicates a larger value for the first condition (i.e., the condition that appears to the left of vs.) compared to the second condition (i.e., the condition that appears to the right of vs.); for takeover quality, a positive effect size indicates poorer takeover quality for the first group compared to the second group. The intercept reflects high information acquisition (reference category).

^aThe degrees of freedom using the Satterthwaite approximation was less than four for these analyses. When this occurs, the distribution of the test statistic no longer approximates the t-distribution, and type I error can be larger than implied (Tanner-Smith et al., 2016). In these cases, a stricter α of .01 is used as suggested by Tanner-Smith et al. (2016).

^†p < .10. ^*p < .05. ^**p < .01.

Research Question 1: Does Time Budget Affect Takeover Performance?

Overall, the results from each of the measures for the time budget comparison suffered from a limited number of studies There were three independent samples or fewer contributing to most measures. This could explain why none of the mean effect sizes were significantly different from zero. Descriptively, the mean effect size for takeover timing was faster when drivers had to take over with the shorter time budget compared to the longer time budget, and the mean effect size for takeover quality was a little worse when the time budget was shorter compared to longer, though neither was significantly different from zero. The meta- analysis by Zhang et al. (2019) also found drivers took over faster when given a shorter time budget compared a longer time budget. Gold et al. (2016) suggest that longer takeover times are indicative of better takeover performance because drivers take the time to regain situation awareness before taking over. The pattern of mean effect sizes from the current meta-analysis tentatively supports this argument and Hypothesis 1 because when drivers were given extra time to take over, they tended to take over more slowly, which corresponded with better takeover quality. These results appear to reflect a speed-accuracy tradeoff such that faster takeovers can only be achieved if takeover quality is sacrificed. This support is tenuous, though, because the results of this meta-analysis did not eliminate the possibility that time budget had no effect on takeover performance. Additionally, our conclusions are specific to the time budgets analyzed (5–9 s) and may not hold for time budgets beyond this range.

Research Question 2: Does Engaging in NDRTs Affect Takeover Performance?

The NDRT versus no NDRT results showed takeover timing was longer and takeover quality was worse when drivers engaged in NDRTs compared to not engaging in NDRTs. In particular, drivers took over vehicle control a little slower and had moderately more variability in lane positioning when engaging in NDRTs. Unlike the first research question results, a longer takeover time coincides with worse takeover quality. This finding suggests longer takeover times are not always better, and takeover time has only limited utility in assessing takeover performance. One theoretical explanation for this is that the extra time the driver took to take over was not used to regain situation awareness and instead was used to switch tasks, which may involve putting away the NDRT, shifting attention, and retrieving task-specific goals. Drivers may also become more out of the loop when engaging in NDRTs and consequently take longer to regain situation awareness. This could be tested in future research by querying drivers’ situation awareness (Durso & Dattel, 2004; Endsley, 1988) or using eye-tracking based methods to measure situation awareness (Samuel et al., 2016). Thus, even though the driver is engaging in the NDRT while they are not responsible for the driving task, engaging in NDRTs does impair takeover performance, supporting Hypothesis 2.1. This means that even if drivers were trying to optimize takeover performance by shedding NDRTs, they did not do so effectively because they still experienced a takeover performance decrement. The subsequent results show the degree of the takeover performance decrement depends on whether the NDRT has overlapping resource demands with the driving task.

The visual versus nonvisual NDRT results and the visual, manual versus nonvisual, nonmanual NDRT results showed that takeover timing was moderately longer and takeover time was a little longer when drivers engaged in NDRTs with overlapping resource demands compared to NDRTs without overlapping resource demands. Takeover timing for the manual versus nonmanual NDRT comparison was a little longer when drivers engaged in NDRTs with overlapping resource demands compared to NDRTs without overlapping resource demands as well, but it only trended in that direction. The moderator analysis also showed a trend that takeover timing was moderately longer, and takeover time was a little longer when drivers engaged in NDRTs with overlapping resource demands compared to NDRTs without overlapping resource demands. Thus, drivers take longer to take over when NDRTs has overlapping resource demands with the driving task.

Specific takeover quality measures and takeover quality broadly did not reach significance for any of the three analyses that examine the effect of overlapping resource demand, but several measures were trending in one direction. In particular, steering magnitude was slightly greater for visual NDRTs compared to nonvisual NDRTs, collisions were slightly greater for manual NDRTs compared to nonmanual NDRTs, minimum distance was slightly shorter for visual-manual NDRTs compared to nonvisual, nonmanual NDRTs, and steering magnitude was slightly greater for visual-manual NDRTs compared to nonvisual, nonmanual NDRTs. Additionally, the visual versus nonvisual NDRT and manual versus nonmanual NDRT analyses showed takeover quality broadly was slightly worse when the NDRT had overlapping resource demands compared to NDRTs without overlapping resource demands. These findings favor the conclusion that engaging in NDRTs with overlapping resource demands with the driving task leads to worse takeover performance as predicted by Wickens’ multiple resource theory, supporting Hypothesis 2.2.

Implementing laws that restrict NDRT use during level 3 automated driving is one practical implication that might be drawn from these results. However, implementing such laws would undermine one of the objectives of level 3 automated vehicles, which is to free the driver from the driving task so that they can complete other tasks. Moreover, drivers may choose to ignore such laws and engage in NDRTs anyway. A better approach would be to ensure drivers have enough time to get back in the loop even if they are engaging in a visual-manual NDRT. This follows a user-centered design approach in which the system is designed around the user. Such an approach should be recommended by standards organizations such as SAE International (2018), which currently gives little guidance on how to determine how much time to give drivers.

One way to ensure drivers have enough time to get back in the loop is investing in the development and deployment of technologies that increase an ADS’s ability to predict system limits (Zhang et al., 2019). For example, implementing vehicle-to-everything (V2X; Chen et al., 2017; Harding et al., 2014) services would allow vehicles to wirelessly communicate with each other and roadway infrastructure about roadway conditions ahead that are not yet in sight (e.g., collisions, construction), and a level 3 automated vehicle or above could use this information to determine whether a system limit is approaching. For level 4 automated vehicles, automobile manufacturers could incorporate technologies to monitor the driver’s preparedness to take over vehicle control, taking into account engagement in NDRTs and their resource demands, and dynamically decide whether to issue a TOR or implement a minimal risk maneuver based on which poses less risk. For instance, if the ADS can no longer handle the driving task, having a driver who is listening to a podcast (which has no overlapping resource demands with the driving task) take over vehicle control in the middle of a multi-lane highway may pose less risk than having the ADS bring the vehicle to a stop in the middle of a multi-lane highway, which poses considerable risk.

Research Question 3: Does Providing the Driver With Information Support Affect Takeover Performance?

Overall, there was a lack of evidence that information support had an effect on takeover performance. None of the measures analyzed were significantly different from zero even when measures were collapsed into the broad categories takeover timing and takeover quality. Additionally, the mean effect sizes for the broad categories takeover timing and takeover quality were negligible. Thus, no support for Hypothesis 3 was found. In line with the current findings, Zhang et al. (2019) found information support had no effect on takeover time, though they examined a narrower question about whether TORs that indicate the direction of a hazard affect takeover time. They acknowledged that this could have been because information support only affects takeover quality, which they did not meta-analyze, but the current results show no support for this hypothesis.

It is possible that not all levels of information support assist the driver with processing information and coming back into the loop. In other words, it is possible that recommending a specific action such as braking (decision selection) supports the driver, but highlighting the cause of the TOR (high information acquisition) and indicating whether the adjacent lane is open (information analysis) does not support the driver. A moderator analysis was conducted to test this, and it found no significant differences between levels of information support. Interestingly, the two studies that compared decision selection to low information acquisition and measured takeover quality had conflicting results (Borojeni et al., 2017, 2018). Thus, it is unclear whether instructing the driver to initiate a specific action at takeover improves takeover performance. This situation in which the ADS instructs the driver to perform a specific action at takeover is paradoxical, though, because if the ADS knows the appropriate maneuver to perform, it is unclear why it would not perform the maneuver itself.

Another point to consider is the possibility that the specific implementations of information support that prior research has examined were inadequate, yet some other unexamined implementation of information support would improve takeover performance. The current results cannot resolve this issue and can only suggest that specific implementations of information support examined by prior research had no effect on takeover performance. Future research should explore this issue further before definitive conclusions are made.