Driving Performance After Self-Regulated Control Transitions in Highly Automated Vehicles

Abstract

Objective:

This study aims to explore whether driver-paced, noncritical transitions of control may counteract some of the aftereffects observed in the contemporary literature, resulting in higher levels of vehicle control.

Background:

Research into control transitions in highly automated driving has focused on urgent scenarios where drivers are given a relatively short time span to respond to a request to resume manual control, resulting in seemingly scrambled control when manual control is resumed.

Method:

Twenty-six drivers drove two scenarios with an automated driving feature activated. Drivers were asked to read a newspaper or monitor the system and relinquish or resume control from the automation when prompted by vehicle systems. Driving performance in terms of lane positioning and steering behavior was assessed for 20 seconds post resuming control to capture the resulting level of control.

Results:

It was found that lane positioning was virtually unaffected for the duration of the 20-second time span in both automated conditions compared to the manual baseline when drivers resumed manual control; however, significant increases in the standard deviation of steering input were found for both automated conditions compared to baseline. No significant differences were found between the two automated conditions.

Conclusion:

The results indicate that when drivers self-paced the transfer back to manual control they exhibit less of the detrimental effects observed in system-paced conditions.

Application:

It was shown that self-paced transitions could reduce the risk of accidents near the edge of the operational design domain. Vehicle manufacturers must consider these benefits when designing contemporary systems.

Keywords

automation automated driving control transitions takeover requests driving performance task regulation distributed cognition cognitive systems engineering

Introduction

Automated vehicles show promise in reducing road accidents (Eriksson, Banks, & Stanton, 2017; World Health Organisation, 2015) but in their current form are no panacea for road safety (Eriksson & Stanton, 2017b; Gold, Damböck, Lorenz, & Bengler, 2013; Gold, Korber, Lechner, & Bengler, 2016; Kalra & Paddock, 2016). As drivers engage a contemporary automated driving system, they are decoupled from the operational and tactical levels of control, leaving only high-level strategic goals to be dealt with by the driver (Bye, Hollnagel, & Brendeford, 1999; Michon, 1985) while still being expected to resume control when the vehicle reaches the limits of its operational design domain (ODD; the ODD may include geographic, roadway, environmental, traffic, speed, and/or temporal limitations of automated driving availability; SAE J3016, 2016) or when a system failure or sudden, unexpected event forces a transition back to manual control (SAE J3016, 2016). This intermediate form of automation has been deemed hazardous as drivers are required to monitor and be able to regain control at all times (Casner & Schooler, 2015; Seppelt & Victor, 2016; Stanton, 2015). This is a form of “driver initiated automation,” where the driver is in control regardless of whether the system is engaged or disengaged (Banks & Stanton, 2015, 2016; Lu, Happee, Cabrall, Kyriakidis, & de Winter, 2016; SAE J3016, 2016) contrary to “system initiated automation” where the controlling agent, be it driver or the automated driving system is in control, is determined by artificial intelligence (Gordon, Kozloski, Kundu, Malkin, & Pickover, 2017) in a MABA-MABA fashion (Dekker & Woods, 2002). The intermittent transitions of control and the sharing of task-relevant information between the driver and driving automation can be described in terms of distributed cognition (DCOG) (Hollan, Hutchins, & Kirch, 2000; Hutchins, 1995). DCOG offers a paradigm shift in describing how a human interacts with other humans, artifacts, and artificial agents, describing it as a “system” where cognition, knowledge, and mental models are distributed between agents in the joint cognitive system, henceforth referred to as system (Hollnagel & Woods, 2005). In such a system, coordination and communication between system entities are of the utmost importance (Christoffersen & Woods, 2002; Eriksson & Stanton, in press; Hutchins, 1995; Stanton, 2014). The functioning of a cognitive system has been described by Hollnagel and Woods (2005) in the contextual control model (COCOM) as a cybernetics inspired tracking loop. The COCOM model describes how the control of a system can be lost and gained and how different levels of control influence performance. As time progresses in a task such as driving, dynamic shifting between four different control modes can be made, dependent on the time horizon, and the predictability of the situation.

Hollnagel and Woods (2005) describes four control modes:

In the scrambled control mode, control actions are selected at random in a trial-and-error fashion, often urgently.

In the opportunistic control mode, the next action to be carried out is determined by the salient features of the current context, such as a dominating part of the human-machine interface, with limited forward planning and anticipation. This control mode relies on internal heuristics and may be inefficient compared to higher levels of control.

In the tactical control mode, the next action is usually preplanned as the operator has a longer time horizon, thus enabling the use of rules and procedures to carry out actions. In the tactical control mode, the operator is still heavily influenced by the immediacy of the situation and will therefore still be influenced by the interface to some extent (Stanton, Ashleigh, Roberts, & Xu, 2001).

In the highest level control mode, the strategic control mode, the time horizon is longer (Figure 1), which enables long-term planning and anticipation. Operators in this control mode have evaluated the relationship between cause and effect more precisely and will therefore have more overall control of the situation (Stanton et al., 2001).

Figure 1.

The relationship between Hollnagel and Woods (2005) control levels and available time and predictability of a situation.

Driver assistance systems on Level 1/2 (SAE J3016, 2016) have shown detrimental effects on driver behavior when drivers were asked to resume control compared to manual driving. Young and Stanton (2007) observed an increase in brake reaction time of approximately 1 second when a leading vehicle suddenly braked requiring driver intervention when using adaptive cruise control, which is the approximate time it takes a driver to respond to a sudden braking event while engaged in fully manual driving (Summala, 2000). An additional 0.3-second increase was found when adaptive steering was added (SAE J3016, 2016, Level 2). It is evident that the introduction of automated driving systems on Levels 1 and 2 have detrimental effects on driver readiness and driving performance. In a review of the literature of transitions of control in Level 3 automated driving systems, Eriksson and Stanton (2017b) found that it takes 1.14 to 15 seconds to respond to a request to intervene in an externally paced transition (system or event paced). In a previous analysis of the data from the experiment presented in this article, it was also found that drivers took between 1.97 and 25.75 seconds to respond to a request to resume control when they were able to pace the transitions (Eriksson & Stanton, 2017b). The increase in response times may be attributed to the fact that the control activities involved in driving are normally “automatic” activities that require little or no conscious effort to be executed (Norman, 1976, p. 70). When these activities are disrupted, by for example automation requesting manual control, conscious control is required to the detriment of “manual-driving” performance.

Russell et al. (2016) showed that drivers who are unaware of changes to vehicle driving characteristics (the steering torque profile) show declined steering performance, which may lead to over- or undershoot, indicating scrambled control. Merat, Jamson, Lai, Daly, and Carsten (2014) showed that it takes approximately 40 seconds for the driver to regain control stability after a control transition. Notably, the control transition used in the experiment of Merat et al. (2014) was system initiated and lacked a takeover request (TOR), which is featured in other recent research into control transitions in automated driving (e.g., Damböck, Bengler, Farid, & Tönert, 2012; Eriksson & Stanton, 2017b). The lack of human-machine interface (HMI) to convey the need to resume control may have contributed to scrambled control behavior in the first 40 seconds after resuming control. Similarly, Desmond, Hancock, and Monette (1998) found larger heading errors and poorer lateral control in the first 20 seconds after resuming control from automated driving following a failure compared to compensating for a wind gust in manual driving, hinting at scrambled control. Moreover, Gold et al. (2013) showed that drivers that subjected to short lead times (5 vs. 7 seconds) for the TOR elicited shorter response times, but performance post takeover was characterized by harsh braking, rapid lane changing, and unnecessary full stops, indicating that drivers were at the scrambled level of control. Damböck et al. (2012) found that an 8-second lead time for TORs produced driving performance at the same level as manual driving, indicating that drivers experienced a higher, operational or tactical level of control. Evidently, there may be a relationship between the available time to resume control from the automated vehicle and the resulting level of control performance when control has been handed back to the driver.

Eriksson and Stanton (2017b) and Eriksson et al. (2017) argue that “driver-paced” transitions will be a commonly occurring type of control transition in SAE J3016 (2016) Level 3 and 4 systems, where there is enough foresight when it comes to identifying system limitations (e.g., through the fleet learning feature of Tesla Autopilot version 8; Tesla Motors, 2016), which in turn will increase the lead time between TOR and a transition to manual control (Eriksson & Stanton, 2017b). The increase in the time between a TOR and a transition to manual control extends the time horizon, which in turn could enable drivers to attain a higher tactical level of control compared to the externally paced transitions reported in the literature (Damböck et al., 2012; Desmond et al., 1998; Gold et. al., 2013; Merat et al., 2014; Russell et al., 2016). Eriksson et al. (in press) showed that when the reason for a TOR was highlighted through an augmented reality overlay, drivers exhibited opportunistic control by braking to buy time. This was not observed when the augmented reality display showed higher levels of semantics, such as arrows indicating safe paths, indicating a higher level of control. Hollnagel (1993) emphasized that the “essence of control is planning” (p. 168), which means that a sudden, forced transition to manual control likely has detrimental effects on driving performance unless appropriate support is given (Cranor, 2008; Eriksson & Stanton, 2017b, in press; Stanton, 2015; Stanton & Young, 1998).

In light of this, this study aims to explore whether there are any differences in driving aftereffects following a transition from automated to manual control in two conditions, one with and one without a secondary task, compared to baseline manual driving. This research aims to provide knowledge on whether the resulting level of control (in terms of driving performance) is affected by control transitions when the transition is driver initiated (Banks & Stanton, 2015, 2016; Lu et al., 2016) and whether the additional time available for self-regulation (Cooper, Vladisavljevic, Medeiros-Ward, Martin, & Strayer, 2009; Eriksson, Lindström, Seward, Seward, & Kircher, 2014; Kircher, Eriksson, Forsman, Vadeby, & Ahlstrom, 2017; Wandtner, Schumacher, & Schmidt, 2016) of the transition process has a positive effect on driving performance.

Method

Participants

Twenty-six participants (10 women, 16 men) between 20 and 52 years of age (M = 30.27, SD = 8.52) with a minimum one year of driving experience and an average of 10.57 years of experience (SD = 8.61) took part in the study. Upon recruiting participants, we obtained their informed consent. The study complied with the American Psychological Association Code of Ethics and had been approved by the University of Southampton Ethics Research and Governance Office (ERGO No. 17771).

Equipment

The experiment was carried out in a fixed based simulator at the University of Southampton. The simulator consisted of a Jaguar XJ 350 vehicle with pedal and steering sensors provided by Systems Technology Inc. as part of the STISIM Drive M500W system (http://www.stisimdrive.com/m500w). The driving environment is powered by the STISIM Drive Version 3 software engine providing a projected 140° field of view. Rearview and side mirrors were provided through additional projectors and video cameras. The Jaguar XJ instrument cluster was replaced with a 10.5” LCD panel to display computer-generated graphics components, in the case of this experiment, takeover requests.

When a takeover request was issued, the engine speed dial was hidden, and the request was shown in its place. The symbol asking for control resumption is shown in Figure 2a, and the symbol used to prompt the driver to reengage the automation is shown in Figure 2b.

Figure 2.

(a) On the left-hand side is a control transitions request to automated vehicle control presented in the instrument cluster, coupled with a computer-generated voice message stating “automation available.” (b) Right-hand side, the graphic is shown in the instrument cluster of a takeover request. The visual TOR was coupled with a computer-generated voice message stating “please resume control.”

The mode switching human-machine interface was located on a Windows tablet in the center cluster, consisting of two buttons, either to engage or disengage the automation. To enable dynamic dis- and reengaging of the automation, something STISIM currently does not support, bespoke algorithms for automated longitudinal and lateral control developed in Visual Basic 6 for the STISIM Open Module API platform were used (Eriksson, Spychala, De Winter, & Stanton, 2016; Eriksson & Stanton, 2017a).

Experiment Design

The experiment had a repeated-measures, within-subject design with three counterbalanced conditions: highly automated driving while passively monitoring the system, highly automated driving while engaged in a secondary task, and a manual baseline drive. In both automated conditions, the participants drove for approximately 20 minutes, at 70 mph on a 3-km, six-lane motorway, three lanes either side of a barrier separating the direction of travel, with mild curves and moderate traffic conditions. The route was mirrored between the two automated conditions to reduce familiarity effects while keeping the roadway layout consistent (Figure 3). In the manual driving conditions, drivers drove a shortened version (10 minutes) of the route in Figure 3 under identical traffic and road layout conditions as in the automated conditions. In the secondary task condition, drivers were asked to read an issue of National Geographic while the automated driving system was engaged to effectively remove them from the driving task. The drivers started in manual driving mode at the start of each automated driving condition, and the timer to trigger the prompts to transition between control modes was triggered after 2 minutes of manual driving. Consequentially, drivers spent between 2 minutes and 30 seconds and 2 minutes and 45 seconds in manual mode before being asked to engage the automated driving system (Figure 4).

Figure 3.

A bird’s eye view of the road layout in the passive monitoring condition (route was mirrored for the secondary task condition).

Figure 4.

Takeover procedure during an experimental drive. Each manual and automated driving section lasted between 30 and 45 seconds.

Following initial engagement of the automated driving system, the drivers were prompted to either resume control from or relinquish control to the automated driving system at a randomized interval ranging from 30 to 45 seconds, thus allowing for approximately 24 control transitions, of which half were to manual control.

Dependent Variables

The following metrics were calculated for each condition and participant for the duration of 20 seconds post transition to manual control in the automated condition. For the manual driving comparison, a section of road was randomly selected, and 20 seconds of driving performance data were extracted for comparison between the automated and manual conditions.

Standard deviation of steering wheel angle (degrees): This metric is related to driver workload. In normal driving conditions, drivers tend to make continuous small steering corrections to adjust their lane position as driving conditions change. When workload increases, these corrections decrease in frequency, resulting in the need for larger steering inputs to correct the lane positioning (Liu, Schreiner, & Dingus, 1999). This metric is defined in the following equation (cf. Knappe, Keinath, Bengler, & Meinecke, 2007, p. 2).

Mean absolute lateral position (centimeters): This metric describes lane keeping accuracy and is calculated in the following way: $M L P = | \frac{\sum_{i = 1}^{n} d_{i}}{n} |$ , where d_i is the distance measured from the center of the vehicle to the lane center.

While a root mean square (RMS) of the lateral position may provide insights into large movements in lane position, mean absolute lateral position is used in combination with plotting the standard deviation as this not only provides the mean lateral position but also the RMS through the shaded standard deviation area shown in Figures 5 through 7. Furthermore, the standard deviation of the SDSWA metric is not included in Figures 8 through 10 as this effectively is a calculated standard deviation of a standard deviation metric and thus does not provide any additional information.

Figure 5.

A time-series analysis of the difference in the mean absolute lane position after resuming control from an automated driving system after engaging in passive monitoring compared to manual driving. The black dashed line in the middle graph indicates an alpha of 0.01, and the dashed-dotted line indicates a Bonferroni corrected alpha of 0.00083.

Figure 6.

A time-series analysis of the difference in the mean absolute lane position after resuming control from an automated driving system after engaging in a secondary task compared to manual driving. The black dashed line in the middle graph indicates an alpha of 0.01, and the dashed-dotted line indicates a Bonferroni corrected alpha of 0.00083.

Figure 7.

A time-series analysis of the difference in absolute mean lane position after resuming control with a secondary task and without a secondary task. The black dashed line in the middle graph indicates an alpha of 0.01, and the dashed-dotted line indicates a Bonferroni corrected alpha of 0.00083.

Figure 8.

A time-series analysis of the difference in the standard deviation of steering angle after resuming control from an automated driving system after engaging in passive monitoring compared to manual driving. The black dashed line in the middle graph indicates an alpha of 0.01, and the dashed-dotted line indicates a Bonferroni corrected alpha of 0.00083.

Figure 9.

A time-series analysis of the difference in the standard deviation of steering angle after resuming control from an automated driving system after engaging in a secondary task compared to manual driving. The black dashed line in the middle graph indicates an alpha of 0.01, and the dashed-dotted line indicates a Bonferroni corrected alpha of 0.00083.

Figure 10.

A time-series analysis of the difference in the standard deviation of steering angle after resuming control from an automated driving system after engaging in a secondary task compared to passive monitoring. The black dashed line in the middle graph indicates an alpha of 0.01, and the dashed-dotted line indicates a Bonferroni corrected alpha of 0.00083.

The authors argue that these metrics could be a good indicator of the level of control exerted by the driver. For example, higher values of SDSA while showing large variability in mean absolute lane position would indicate scrambled control, whereas large SDSA while maintaining low variability in absolute lane position would indicate opportunistic control, and low variability in both metrics would indicate a tactical level of control.

To determine the aftereffects of automated driving with and without a secondary task before a takeover request was issued, we compared the mean absolute lateral position and standard deviation of absolute steering wheel angle between the two conditions and a baseline drive. The performance metrics were sampled at 75 Hz and were aggregated into 60 bins (approximately 333.33 milliseconds of data/bin).

Procedure

Upon arrival, the participants were asked to read a participant information sheet containing information regarding the study, including the right to at any point abort their trial without any questions asked. After reading the information sheet, the participants were asked to sign an informed consent form indicating that they understood any risks associated with participating and their rights while participating in the experiment.

After filling out the consent forms, the participants were asked to fill out a demographics questionnaire before being informed that they were supposed to drive a vehicle equipped with automated driving features. The participants were told how to activate and deactivate the system and that they would be able to override any system inputs via the steering wheel or pedals. Furthermore, the participants were told that they were responsible for the safe driving of the vehicle regardless of the mode and thus needed to be able to safely resume control in case something caused the system to fail (an event that could not occur in the experiment), in accordance with current legislation (United Nations, 1968). Lastly, the participants were told that the system may prompt them to either resume or relinquish control of the system and that when such a prompt was issued, they would adhere to it only when they deemed it safe to do so (i.e., allowing them to take the time they needed to acquire enough information to make the transfer). This instruction was designed to take away any pressure to immediately follow the prompts out of a will to please the researcher and remind them that they were ultimately responsible for the safe operation of the vehicle.

Once a driver had completed a drive, they were asked whether they felt any nausea or discomfort to make sure simulator sickness was avoided, after which they were offered a short break before continuing the study.

Analysis

The performance metrics were tested in a time-series, assessing the difference between conditions every 333.33 milliseconds for the full 20-second testing period through the use of paired t tests. The tests were corrected using the Bonferroni method, resulting in an alpha of 0.00083, which is represented in the graphs as a horizontal dashed-dotted line in the middle graph of each figure. An uncorrected alpha level of 0.01 is shown as a horizontal black line in the same graph.

The unconventional use of paired t tests used in this study is motivated by the increase in temporal resolution compared to using larger bin sizes (which inflates standard deviation, which has an influence on the resulting test results) and fewer t tests. It must be noted that despite conservative corrections of the significance level, the results should only be seen as an indication of an effect rather than direct evidence. However, should the occurrence of a Type 1 error have been present, it should have shown as a sudden spike in the graph. The authors argue that the reliability of the analysis has been increased through the addition of an effect size metric (Cohen’s d) showing the magnitude of the difference between the conditions.

The level of significance is represented through the negative base-10 logarithm of p, where large values represent small p values in a similar fashion to the Manhattan plot (previously used in Gibson, 2010; Petermeijer, Cieler, & de Winter, 2017; Tanikawa et al., 2012). Cohen’s d was calculated to assess effect size between conditions in the following way:

C o h e n^{'} s D = \frac{M_{2} - M_{1}}{S D_{p o o l e d}}

S D_{p o o l e d} = \sqrt{\frac{((n_{1} - 1) S D_{1}^{2} + (n_{2} - 1) S D_{2}^{2})}{n_{1} + n_{2} - 2}} .

Results

Mean Absolute Lane Position

When comparing the mean absolute lane position between manual driving and after resuming control from the automation in the passive monitoring condition, there are little differences to be seen (Figure 5). This implies that drivers were able to successfully control vehicle lane positioning to the same extent as when driving manual post resuming control from the automated driving system when able to pace the transition process themselves.

Moreover, similar results can be found when comparing baseline manual driving with driving performance post resuming control from the automated driving system after being engaged in a secondary task (Figure 6), which again suggests that drivers do not suffer from detrimental performance when able to pace the transition process themselves.

Figure 7 show little difference in absolute mean lane position in the two automated driving conditions, from the moment control was resumed to 20 seconds after the moment of resumption of control, suggesting little differences in driving performance post takeover when drivers are allowed to pace the transition of control themselves. A slight, low magnitude (2–3 cm), nonsignificant difference in lane position can be seen approximately between 1.5 and 7 seconds after resuming control.

Standard Deviation of Steering Angle

When comparing the standard deviation of steering angle between the passive monitoring condition after resuming control with the baseline manual driving condition, there are significant, strong effect differences for the entire time period (Figure 8). This suggests that drivers exerted more effort in maintaining vehicle position through steering corrections than in the manual drive.

Moreover, similar results can be found when comparing the baseline manual driving condition with the secondary task condition (Figure 9), again suggesting that drivers had to exert more effort maintaining vehicle position after resuming control than drivers driving manually for a continuous, longer time period.

Lastly, results from the comparison of the standard deviation of steering angle between the passive monitoring and secondary task condition show slight but nonsignificant increase in SDSA in the passive monitoring condition (Figure 10), indicating little differences in steering control between the two task conditions when drivers were permitted to pace the transition process themselves.

Discussion

The aim of this article was to assess whether drivers exhibit any of the aftereffects in terms of driver control as described by Hollnagel and Woods (2005) through steering behavior and lane positioning. This was motivated by research suggesting that drivers adapt their behavior proactively depending on the situation and time pressure (Cooper et al., 2009; Eriksson et al., 2014; Eriksson, Marcos, Kircher, Västfjäll, & Stanton, 2015; Kircher et al., 2017; Kircher, Larsson, & Hultgren, 2014; Wandtner et al., 2016). If drivers are allowed to self-pace the transition from automated to manual driving, we hypothesize that the aftereffects and the seemingly scrambled levels of control, such as harsh braking, sudden lane changes, and poor lateral control with large heading errors (e.g. Desmond et al., 1998; Gold et al., 2013, 2016; Naujoks, Mai, & Nekum, 2014), observed in contemporary research into system-paced transitions of control would be reduced. Indeed, according to Hollnagel and Woods, levels of control in a system are dependent on the available time and predictability of a situation, where larger temporal resolution is rewarded with higher, safer levels of control and vice versa. Hollnagel (1993) emphasizes that the “essence of control is planning” (p. 168), implying that forced transitions to manual control likely have a detrimental effect on driving performance.

The results of this study indicate that when drivers were able to moderate their transition to manual control by means of the time they took to resume control (between 1.9 and 25.7 seconds as reported in Eriksson & Stanton, 2017b), they maintained lateral positioning after the transition to a level comparable to when driving manually in both the passive monitoring and distracted driving condition (Figures 5, 6), indicating a higher level of control than what is reported in the literature utilizing shorter lead times of 5 to 7 seconds (Gold et al., 2013, 2016), or an immediate failure resulting in a system paced transition to manual (Desmond et al., 1998). However, it was also found that drivers exerted more effort maintaining their position in the lane after control was resumed as shown by the significant increase in standard deviation of steering angle in the two automated conditions compared to the manual driving condition (Figures 8, 9). The stability of vehicle position while exerting more effort maintaining control performance could indicate that drivers were at the opportunistic level of control, approaching the tactical level. Research by Russell et al. (2016) found that drivers could not maintain the same level of steering control as during manual driving. An alternate explanation of the increase in steering control inputs could be that drivers are carrying out an opportunistic recalibration by exploring the vehicle dynamics and whether there has been a change to the lateral control dynamics since manual control was ceded. This is somewhat supported by Russell et al., who hypothesized that drivers would rely on an inaccurate mental model before reaching their previous manual driving performance while adapting to the manual driving task to the detriment of normal “internally automated” performance (Norman, 1976, p. 70). Thus, the increase in standard deviation of steering angle may be attributed to the fact that drivers had to resume a closed loop activity after being removed from the task for some time and therefore showed larger steering inputs while maintaining lane positioning comparable to manual driving (Russell et al., 2016).

When compared to externally paced transitions in the literature, the results obtained in this study show similar effects as reported by Merat et al. (2014), who found that drivers who were asked to resume control in a predictable manner performed better in terms of lateral positioning and steering controllability than drivers who were not expecting a transition to occur, much like in driver-paced versus externally paced transitions of control. The results of this study do indicate lesser lane deviation post-TOR for the two automated driving conditions compared to what is reported in Merat et al. This discrepancy in magnitude between Merat et al. and the results presented in this article could potentially be partially explained by the different bin sizes as a larger bin size will inherently inflate the standard deviation and may skew the mean due to changes over time. Contrary to Desmond et al. (1998), little effect on lateral positioning was found, indicating that drivers were able to attain an opportunistic or tactical level of control rather than the seemingly scrambled level of control in Desmond et al. Similarly, Gold et al. (2013) found that drivers exhibited indications of scrambled control by performing sudden lane changes or unnecessary use of the hard shoulder when a short lead time was given. Evidently, there may be a relationship between the available time to resume manual control and the resulting level of control, where higher levels of control are attained when drivers have more time to resume manual control and vice versa. This relationship is further strengthened by the results of Gold et al. (2016), who found that drivers performed worse and had more crashes when asked to resume control when traffic density increases while subject to a fixed 7-second lead time before a crash would occur. The findings by Gold et al. (2016) serve as an example of a decay to scrambled control as traffic density increased. This is in line with Eriksson et al. (2015), who showed that drivers need more time to assess a situation when complexity is high.

When comparing the aftereffects of a driver-paced transition on lateral positioning between the two automated driving conditions, no significant differences could be identified in lane positioning (Figure 7) and standard deviation of steering angle (Figure 10) despite the significant increase in TOR response time of 1.5 seconds (range, 1.9–25.7 seconds, median = 4.5 seconds in the passive monitoring condition; and range, 3.1–20.9 seconds, median = 6.06 seconds in the secondary task condition) in the distracted driving condition reported in Eriksson and Stanton (2017b), a previous analysis of the experiment reported in this article focusing on the response time distributions and effects of secondary tasks on response time. This lack of difference could be attributed to the drivers being able to self-moderate the transition (Cooper et al., 2009; Eriksson et al., 2014, 2015; Kircher et al., 2014, 2017; Wandtner et al., 2016), thus extending the temporal horizon, which results in maintaining higher levels of control. Eriksson and Stanton argue that part of the additional time it took drivers to resume control in the secondary task condition could be attributed to the repositioning required by virtue of holding the magazine. Indeed, Petermeijer, Abbink, and de Winter (2015) argue that the repositioning phase should be accounted for as part of the control resumption process.

Given that drivers were completely removed from the visual scanning of the road environment by virtue of the secondary task, in addition to the automation of lateral and longitudinal vehicle control, we argue that the additional 1.5 seconds before resuming control could be partially explained by drivers extending the time horizon to ensure a higher level of control. Thus, the additional time allotted to assess the situation may have been crucial in maintaining the same level of opportunistic or tactical control performance as was observed in the passive monitoring for lane positioning and steering behavior as shown in Figure 7 and Figure 10. These findings indicate that drivers who are allowed enough time between a takeover request and the transition to manual control are able to self-pace the transition process, thereby attaining a higher level of control and reducing the aftereffects on control performance observed in contemporary research utilizing externally paced transitions with shorter lead times.

Conclusions

Contemporary research into externally paced transitions to manual control from automated driving have shown indications of reduced driver control levels through detrimental effects on driving performance when manual control was resumed. In light of this, this article assessed the influence of driver-paced, nonurgent transitions to manual control on driving performance post transfer of control as this is deemed to be the most common type of transition in SAE Level 3 and 4 systems (Eriksson et al., 2017; Eriksson & Stanton, 2017b).

It was found that drivers who were able to pace the transition process back to manual control, irrespective of whether they were engaged in a secondary task or not, exhibited significantly more steering corrections than manual driving conditions while maintaining comparable positioning in the lane. These findings contradict some of the contemporary literature showing that drivers exhibit harsh braking, sudden lane changes, and poor lateral control with large heading errors (Desmond et al., 1998; Gold et al., 2013, 2016). This indicates that drivers attained a higher level of control when resuming control in a self-paced setting compared to an externally paced transfer of control. When the control transition aftereffects were assessed between the passive monitoring and the distracted driving condition, no significant differences in either lateral positioning or corrective steering behavior could be found. However, as was reported in a previous analysis of the response time data in this study in Eriksson and Stanton (2017b), it was found that when drivers resumed control in the distracted driving condition, they took an additional 1.5 seconds to do so. This indicates that drivers took more time to complete the transition to extend the time horizon, thus ensuring that a sufficient control level was maintained.

In conclusion, these results indicate that drivers who are able to self-moderate the time required to transfer back to manual control exhibit less of the detrimental effects observed in externally paced conditions (e.g., those found in Desmond et al., 1998; Gold et al., 2013, 2016). This is promising as a higher level of vehicle control could reduce the risk of accidents in situations where lead times between a request to resume manual control and where manual control is required, such as near the edge of the operational design domain (SAE J3016, 2016) of the automated driving system. Such anticipatory behavior is in line with research by Kircher et al. (2014), who found that users of adaptive cruise control anticipated situations where the system would perform poorly and disengaged the system before such a situation could occur, thus strategically pacing their interaction with the system. The authors would like to argue that this type of behavior is needed from automated vehicles in higher levels of automation (SAE Level 4), where the automation recognizes that it is approaching its ODD and triggers a takeover procedure well in advance to avoid an emergency. Examples of such behavior could be to request manual control when approaching a motorway exit, approaching an area where lane markings are faded, or even when the system detects that it is approaching an area with poor visibility, such as fog.

It must be noted that the experimental design in this article was designed to compress experience and normalize the experience of the transition process to avoid novelty effects and thus is not completely representative with regard to the frequency of requests to transition between automated and manual modes that drivers might experience in normal usage of such a system and that more research must be carried out in more naturalistic conditions to further shed light on the control transition process in SAE Level 4 systems. However, the results support the argument of Eriksson & Stanton (2017b) that the 5th to 95th percentiles of the range of driver takeover response time variability must be accounted for in designing the nonurgent takeover process for SAE Level 3 and 4 systems as this would ensure safer transitions to manual vehicle control.

Key Points

Aftereffects of driver-paced control transition between automated and manual driving was assessed.

Drivers who were allowed to self-pace the transition process show lane keeping performance equal to manual driving.

Drivers showed larger standard deviation of steering angle, indicating that an increased control effort was required to maintain performance on the same level as in manual driving.

Footnotes

Acknowledgements

This research has been conducted as a part of the European Marie Curie ITN project HFAuto-Human Factors of Automated driving (PITN-GA-2013-605817).

Alexander Eriksson received his Master of Science degree in cognitive science from Linköping University in 2014 and is currently a Marie Curie Research Fellow in the EU-funded Marie Curie International Training Network on Human Factors in Highly Automated Vehicles (HF-Auto) within the Faculty of Engineering and the Environment at the University of Southampton where he is undertaking his PhD research. His primary research focus is on human-automation interaction, specifically in how automated vehicles hand back control to the driver in terms of information presentation and cues.

Neville A. Stanton is a chartered psychologist, chartered ergonomist, and chartered engineer and holds the chair in human factors engineering in the Faculty of Engineering and the Environment at the University of Southampton. He is leading the EPSRC/JLR- funded project on Human Interaction: Designing Automated Vehicles (HI: DAVe) and is a partner in the EU-funded Marie Curie International Training Network on Human Factors in Highly Automated Vehicles (HF-Auto).

References

Banks

V. A.

Stanton

N. A.

(2015). Contrasting models of driver behaviour in emergencies using retrospective verbalisations and network analysis. Ergonomics, 58, 1337–1346.

Banks

V. A.

Stanton

N. A.

(2016). Keep the driver in control: Automating automobiles of the future. Applied Ergonomics, 53 Pt B, 389–395.

Bye

Hollnagel

Brendeford

T. S.

(1999). Human-machine function allocation: A functional modelling approach. Reliability Engineering & System Safety, 64, 291–300.

Casner

S. M.

Schooler

J. W.

(2015). Vigilance impossible: Diligence, distraction, and daydreaming all lead to failures in a practical monitoring task. Consciousness and Cognition, 35, 33–41.

Christoffersen

Woods

D. D.

(2002). How to make automated systems team players. Advances in Human Performance and Cognitive Engineering Research, 2, 1–12.

Cooper

J. M.

Vladisavljevic

Medeiros-Ward

Martin

P. T.

Strayer

D. L.

(2009). An investigation of driver distraction near the tipping point of traffic flow stability. Human Factors: The Journal of the Human Factors and Ergonomics Society, 51, 261–268.

Cranor

L. F.

(2008). A framework for reasoning about the human in the loop. Paper presented at the Proceedings of the 1st Conference on Usability, Psychology, and Security.

Damböck

Bengler

Farid

Tönert

(2012). Übernahmezeiten beim hochautomatisierten Fahren [Takeover times for highly automated driving]. Tagung Fahrerassistenz. München, 15, 16.

Dekker

S. W. A.

Woods

D. D.

(2002). MABA-MABA or abracadabra? Progress on human-automation co-ordination. Cognition, Technology & Work, 4, 240–244.

10.

Desmond

P. A.

Hancock

P. A.

Monette

J. L.

(1998). Fatigue and automation-induced impairments in simulated driving performance. Human Performance, User Information, and Highway Design, 1628, 8–14.

11.

Eriksson

Banks

V. A.

Stanton

N. A.

(2017). Transition to manual: Comparing simulator with on-road control transitions. Accident Analysis & Prevention, 102C, 227–234.

12.

Eriksson

Lindström

Seward

Kircher

(2014). Can user-paced, menu-free spoken language interfaces improve dual task handling while driving? In Kurosu

(Ed.), Human-computer interaction. Advanced interaction modalities and techniques (Vol. 8511, pp. 394–405). New York: Springer.

13.

Eriksson

Marcos

I. S.

Kircher

Västfjäll

Stanton

N. A.

(2015). The development of a method to assess the effects of traffic situation and time pressure on driver information preferences. In Harris

(Ed.), Engineering psychology and cognitive ergonomics (Vol. 9174, pp. 3–12). New York: Springer.

14.

Eriksson

Petermeijer

S. M.

Zimmerman

De Winter

J. C. F.

Bengler

K. J.

Stanton

N. A.

(in press). Rolling out the red (and green) carpet: Supporting driver decision making in automation to manual transitions. IEEE Transactions on Human Machine Systems.

15.

Eriksson

Spychala

De Winter

J. C. F.

Stanton

N. A.

(2016). A highly automated driving algorithm implementation on STISIM. Paper presented at the 7th International Conference on Applied Human Factors and Ergonomics.

16.

Eriksson

Stanton

N. A.

(2017a). he1y13/TrafficQuery_STISIM3: A traffic querying class for STISIM drive simulation kernel—Build 3.07.04. Retrieved from https://zenodo.org/record/345955#.WZTVvFGGM2w

17.

Eriksson

Stanton

N. A.

(2017b). Take-over time in highly automated vehicles: Non-critical transitions to and from manual control. Human Factors, 59, 689–705.

18.

Eriksson

Stanton

N. A.

(in press). The chatty co-driver: A linguistics approach applying lessons learnt from aviation incidents. Safety Science.

19.

Gibson

(2010). Hints of hidden heritability in GWAS. Nature Genetetics, 42, 558–560.

20.

Gold

Damböck

Lorenz

Bengler

(2013). “Take over!” How long does it take to get the driver back into the loop? Proceedings of the Human Factors and Ergonomics Society 57th Annual Meeting (pp. 1938–1942). Santa Monica, CA: Human Factors and Ergonomics Society.

21.

Gold

Korber

Lechner

Bengler

(2016). Taking over control from highly automated vehicles in complex traffic situations: The role of traffic density. Human Factors, 58, 642–652.

22.

Gordon

M. S.

Kozloski

J. R.

Kundu

Malkin

P. K.

Pickover

C. A.

(2017). U.S. Patent #9,566,986: Controlling driving modes of self-driving vehicles. Retrieved from http://patft.uspto.gov/netacgi/nph-Parser?Sect1=PTO2&Sect2=HITOFF&p=1&u=%2Fnetahtml%2FPTO%2Fsearch-bool.html&r=1&f=G&l=50&co1=AND&d=PTXT&s1=9566986&OS=9566986&RS=9566986

23.

Hollan

Hutchins

Kirch

(2000). Distributed cognition toward a new foundation for human-computer interaction research. ACM Transactions on Computer-Human Interaction, 7, 174–196.

24.

Hollnagel

(1993). Human reliability analysis: Context and control. San Diego, CA: Academic Press.

25.

Hollnagel

Woods

D. D.

(2005). Joint cognitive systems foundations of cognitive systems engineering. Washington, DC: CRC Press.

26.

Hutchins

(1995). Cognition in the wild. Cambridge, MA: MIT Press.

27.

Kalra

Paddock

S. M.

(2016). Driving to safety: How many miles of driving would it take to demonstrate autonomous vehicle reliability? Transportation Research Part A: Policy and Practice, 94, 182–193.

28.

Kircher

Eriksson

Forsman

Å.

Vadeby

Ahlstrom

(2017). Design and analysis of semi-controlled studies. Transportation Research Part F: Traffic Psychology and Behaviour, 46, 404–412.

29.

Kircher

Larsson

Hultgren

J. A.

(2014). Tactical driving behavior with different levels of automation. IEEE Transactions on Intelligent Transportation Systems, 15, 158–167.

30.

Knappe

Keinath

Bengler

Meinecke

(2007). Driving simulator as an evaluation tool—Assessment of the influence of field of view and secondary tasks on lane keeping and steering performance. Paper presented at the 20th International Technical Conference on the Enhanced Safety of Vehicles (ESV).

31.

Liu

Y.-C.

Schreiner

C. S.

Dingus

T. A.

(1999). Development of human factors guidelines for advanced traveler information systems (ATIS) and commercial vehicle operations (CVO): Human factors evaluation of the effectiveness of multi-modality displays in advanced traveler information systems. Retrieved from https://www.fhwa.dot.gov/publications/research/safety/96150/index.cfm

32.

Happee

Cabrall

C. D.

Kyriakidis

de Winter

J. C.

(2016). Human factors of transitions in automated driving: A general framework and literature survey. Transportation Research Part F: Traffic Psychology and Behaviour, 43, 183–198.

33.

Merat

Jamson

A. H.

Lai

F. F. C. H.

Daly

Carsten

O. M. J.

(2014). Transition to manual: Driver behaviour when resuming control from a highly automated vehicle. Transportation Research Part F: Traffic Psychology and Behaviour, 26, 1–9.

34.

Michon

J. A.

(1985). A critical view of driver behavior models: What do we know, what should we do? In Evans

Schwing

R. C.

(Eds.), Human behavior and traffic safety (pp. 458–524). New York, NY: Plenum Press.

35.

Naujoks

Mai

Nekum

(2014, July). The effect of urgency of take-over requests during highly automated driving under distracted conditions. Paper presented at the 5th International Conference on Applied Human Factors and Ergonomics AHFE 2014, Krakow, Poland.

36.

Norman

D. A.

(1976). Memory and attention. New York, NY: John Wiley.

37.

Petermeijer

S. M.

Abbink

D. A.

de Winter

J. C.

(2015). Should drivers be operating within an automation-free bandwidth? Evaluating haptic steering support systems with different levels of authority. Human Factors: The Journal of the Human Factors and Ergonomics Society, 57, 5–20.

38.

Petermeijer

S. M.

Cieler

de Winter

(2017). Comparing spatially static and dynamic vibrotactile take-over requests in the driver seat. Accident Analysis & Prevention, 99, 218–227.

39.

Russell

H. E. B.

Harbott

L. K.

Nisky

Pan

Okamura

A. M.

Gerdes

J. C.

(2016). Motor learning affects car-to-driver handover in automated vehicles. Science Robotics, 1(1).

40.

SAE J3016. (2016). Taxonomy and definitions for terms related to driving automation systems for on-road motor vehicles, J3016_201609. Warrendale, PA: SAE International.

41.

Seppelt

B. D.

Victor

T. W.

(2016). Potential solutions to human factors challenges in road vehicle automation. In Meyer

Beiker

(Eds.), Road vehicle automation 3 (pp. 131–148). New York, NY: Springer.

42.

Stanton

N. A.

(2014). Representing distributed cognition in complex systems: How a submarine returns to periscope depth. Ergonomics, 57, 403–418.

43.

Stanton

N. A.

(2015). Responses to autonomous vehicles. Ingenia, 62, 69.

44.

Stanton

N. A.

Ashleigh

M. J.

Roberts

A. D.

(2001). Testing Hollnagel’s contextual control model: Assessing team behavior in a human supervisory control task. International Journal of Cognitive Ergonomics, 5, 111–123.

45.

Stanton

N. A.

Young

M. S.

(1998). Vehicle automation and driving performance. Ergonomics, 41, 1014–1028.

46.

Summala

(2000). Brake reaction times and driver behavior analysis. Transportation Human Factors, 2, 217–226.

47.

Tanikawa

Urabe

Matsuo

Kubo

Takahashi

Ito

Matsuda

. (2012). A genome-wide association study identifies two susceptibility loci for duodenal ulcer in the Japanese population. Nature Genetics, 44, 430–434.

48.

Tesla Motors. (2016). Upgrading autopilot: Seeing the world in radar. Retrieved from https://www.tesla.com/en_GB/blog/upgrading-autopilot-seeing-world-radar?redirect=no

49.

United Nations. (1968). Convention on road traffic. Retrieved from http://www.unece.org/fileadmin/DAM/trans/conventn/crt1968e.pdf

50.

Wandtner

Schumacher

Schmidt

E. A.

(2016). The role of self-regulation in the context of driver distraction: A simulator study. Traffic Injury Prevention, 17, 472–479.

51.

World Health Organisation. (2015). Global Health Observatory (GHO) data: Number of road traffic deaths. Retrieved from http://www.who.int/gho/road_safety/mortality/number_text/en/

52.

Young

M. S.

Stanton

N. A.

(2007). Back to the future: Brake reaction times for manual and automated vehicles. Ergonomics, 50, 46–58.