Combining Driving Automation Intent and Hazard Information with an Attention Reminder System: Is More Information Always Better?

Abstract

Automation can result in driver inattention and distraction, which has contributed to fatal collisions. One common safeguard implemented by automakers is “attention reminders” (ARs), which issue alerts when drivers are inattentive. Despite some observed benefits, it is unclear whether ARs alone can ensure safe operation of automation. Thus, ARs might be enhanced by combining them with displays conveying additional information, such as automation intent and surrounding hazards. A driving simulator study was conducted to evaluate the effects of combining ARs with additional information on driver visual attention (measured through gaze behavior). Forty-eight participants were assigned to one of three conditions: Baseline AR, AR + automation intent (that the automation would slow down the car), and AR + automation intent + hazard information (location & severity of potential hazard). The findings suggest that combining ARs with both automation intent and hazard information may have diverted attention away from cues in the environment indicating potential traffic conflicts, while combining the AR with automation intent only supported visual attention to the cues. However, the additional information did not show a performance benefit compared to the Baseline AR. Thus, further research should be done to investigate how to enhance ARs with additional supporting information.

Keywords

driving automation in-vehicle displays driver attention gaze behavior driver monitoring system automation transparency

Introduction

Driving automation can be associated with inattention and distraction, which has contributed to fatal road collisions (e.g., National Transportation Safety Board, 2020). One common safeguard implemented by automakers is “attention reminders,” which leverage information from driver monitoring systems to detect when drivers are inattentive and issue alerts to re-orient their attention to the driving task. Although attention reminders can be beneficial and have become mandated in some jurisdictions (Commission Delegated Regulation (EU) 2023/2590, 2023), it is unclear whether attention reminders alone can ensure appropriate engagement and safe operation of driving automation. Some automakers have been ordered to recall their attention reminder systems (Krisher, 2023), and recent studies suggest that drivers may adapt to the reminders by disengaging just long enough to avoid the alerts, but continuing to engage in potentially distracting activities (Mueller et al., 2024). Thus, attention reminders might be enhanced by combining them with in-vehicle displays conveying additional information.

There has been extensive research on displays supporting the operation of driving automation (for reviews see Kanaan et al., 2024; Wang et al., 2024), with previous studies suggesting a benefit to providing information about the automation intent and surrounding traffic. Automation intent information can provide transparency about the automation’s inner workings and projected behavior, while surrounding traffic information can help highlight potential hazards in the environment. A previous remote study focusing only on intersection incursions found that a display providing both Automation Intent and Hazard Information outperformed displays providing that information separately, but the Automation Intent display was most preferred by users (Kanaan et al., 2023). In this driving simulator study, we conducted an expanded investigation using additional traffic scenarios and collecting driver gaze measures, which can provide insights about drivers’ visual attention allocation. The aim of this study was to evaluate the effectiveness of this additional information in the display compared to a baseline attention reminder.

Approach

The simulator study was conducted with 48 participants (21 Female; average age = 36, standard deviation = 10; average years with full license = 14, standard deviation = 11). A head-mounted eye tracker was used to collect gaze data. To understand how drivers regulate their secondary task engagement while operating automation, participants were provided with a self-paced visual-manual secondary task that mimics the operation of in-vehicle infotainment systems (Donmez et al., 2007). The experiment had two independent variables: Display and Event Criticality.

Display (between-subjects): Participants were randomly assigned to one of three groups: Baseline Attention Reminder (AR), Automation Intent Display (referred to as “Braking Alert”), and Automation Intent + Hazard Information Display (referred to as “Combined Display”); see Figure 1. The Baseline AR consisted of a red eye icon that appeared when drivers have been looking away from the road for more than 3 s, and an auditory alert that was issued if drivers continued to look away. The Braking Alert group were provided with the AR and an icon that appeared when the automation was about to brake in response to a potential hazard. The Combined Display group were provided with both of the above with additional information conveying the upcoming hazard’s expected location and severity, designed based on previous research and display design recommendations (e.g., Wang et al., 2024).

Figure 1.

Display icons presented to each group.

Event Criticality (within-subjects): Participants performed four simulated drives, each including two events (traffic conflicts). Each event was preceded by anticipatory cues (e.g., behaviors of other vehicles or road objects in the environment) that indicated the upcoming event. In the example event in Figure 2, the white car in the right lane tries to cut in front of the participant’s vehicle (blue; left lane) due to a slow truck ahead. Anticipatory cues include the truck slowing down and the reduced distance between the truck and the white vehicle. The event begins when the white vehicle indicates and subsequently performs a left lane change. Participants experienced two versions of each event with varying criticality: action-necessary (required driver intervention to avoid a collision) and action-not-necessary (did not require driver intervention).

Figure 2.

Example of an event that occurred during the simulated drives.

Driver glance behaviors (e.g., presence of a glance, average glance duration) to the anticipatory cues, the secondary task, and the in-vehicle display were collected and analyzed for the period between cue onset and event onset. Various performance and secondary task engagement measures were also collected but are not reported in this abstract due to space limitations. Continuous variables were analyzed using linear mixed models with Display and Criticality as fixed factors and participants as a random factor, while categorical variables were analyzed using mixed effects logistic regression.

Outcome

There was a significant effect of Display on the presence of a glance at the anticipatory cues, time to first glance at the cues, and average glance duration at the cues. The odds of exhibiting a glance at the cues were significantly higher for the Baseline group than the Combined Display group; odds ratio (OR) = 11.6, z = 2.8, p = .005; and the Braking Alert group; OR = 7.54, z = 2.3, p = .002. Participants in the Braking Alert group had the longest average glance duration at the cues. There were no significant differences in average glance duration between the Braking Alert group and the Combined Display group, or between the Combined Display group and the Baseline group. However, participants in the Braking Alert group had a significantly longer average glance duration at the cues than those in the Baseline group; t(43.3) = 2.6, p = .01. Participants in the Combined Display group took on average the longest to glance at cues. The time until the first glance at cues was significantly longer for the Combined Display group than the Baseline group; t(42.9) = 3.7, p = .001, but there were no significant differences between the other groups. Participants in the Combined Display group were also more likely to exhibit glances at the in-vehicle display compared to the other two groups, which may have delayed their glances to the cues. Performance measures such as takeover likelihood and minimum gap time showed a significant effect of event criticality but not the display.

Conclusion

The findings suggest that the Combined Display may have diverted attention away from cues in the environment indicating upcoming traffic conflicts. Although this additional information was meant to alert drivers to upcoming hazards, it attracted more of their attention. This is consistent with previous findings which suggest that providing information about surrounding traffic (compared to simpler displays) was associated with more glances at the display rather than the roadway (e.g., Kraft et al., 2018; Ulahannan et al., 2021). However, similar displays showing Automation Intent and Hazard Information were associated with increased glances to roadway hazards in other studies (e.g., Yang et al., 2018). Thus, there is a need for further examination of display design considerations to identify how information can be communicated to drivers in a timely and appropriate manner to balance informational content and potential for distraction.

Presenting Automation Intent information alone was associated with longer glances at anticipatory cues, which shows some promise. However, performance and secondary task engagement findings did not show a benefit compared to the Baseline attention reminder. Thus, further work is needed to identify how attention reminders can be adapted or augmented to promote safer operation of automation.

Finally, given that the displays always presented perfectly reliable information on Automation intent and Hazard Information, participants may have over-relied on them. Thus, further work should be conducted with more realistic displays with varying degrees of accuracy, and potentially in more realistic contexts (e.g., on-road studies).

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iDs

Dina Kanaan

Birsen Donmez

References

Commission Delegated Regulation (EU) 2023/2590 (2023). http://data.europa.eu/eli/reg_del/2023/2590/oj/eng

Donmez

Boyle

L. N.

Lee

J. D.

(2007). Safety implications of providing real-time feedback to distracted drivers. Accident Analysis and Prevention, 39(3), 581–590.

Kanaan

Laurent

Donmez

(2023). Drivers’ trust and reliance on automated braking at intersections: Effects of automation braking profile and in-vehicle displays [Abstract]. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 67(1), 1033–1037.

Kanaan

Powell

Donmez

(2024). In-vehicle displays for driving automation: A scoping review of display design and evaluation using driver gaze measures. Transport Reviews, 44, 858–888. Advance online publication. https://doi.org/10.1080/01441647.2024.2333152

Kraft

A. K.

Naujoks

Wörle

Neukum

(2018). The impact of an in-vehicle display on glance distribution in partially automated driving in an on-road experiment. Transportation Research Part F: Traffic Psychology and Behaviour, 52, 44–59.

Krisher. (2023, December 13). Tesla recalls nearly all vehicles sold in US to fix system that monitors drivers using Autopilot. AP News. https://apnews.com/article/tesla-autopilot-recall-driver-monitoring-system-8060508627a34e6af889feca46eb3002

Mueller

A. S.

Gershon

Haus

S. H.

Cicchino

J. B.

Mehler

Reimer

(2024). Finding windows of opportunity: How drivers adapt to partial automation safeguards over time. Insurance Institute for Highway Safety.

National Transportation Safety Board. (2020). Collision between a sport utility vehicle operating with partial driving automation and a crash attenuator, Mountain View, CA, (Report No. HAR-20/01). National Transportation Safety Board.

Ulahannan

Thompson

Jennings

Birrell

(2021). Using glance behaviour to inform the design of adaptive HMI for partially automated vehicles. IEEE Transactions on Intelligent Transportation Systems, 23(12), 24157–24172.

10.

Wang

Mehrotra

Wong

Parker

Roberts

S. C.

Kim

Romo

Horrey

W. J.

(2024). Human-machine interfaces and vehicle automation: A review of the literature and recommendations for system design, feedback, and alerts. Transportation Research Part F: Traffic Psychology and Behaviour, 107, 549–561.

11.

Yang

Karakaya

Dominioni

G. C.

Kawabe

Bengler

(2018). An HMI concept to improve driver’s visual behavior and situation awareness in automated vehicle. In 2018 IEEE 21st international conference on intelligent transportation systems (ITSC) (pp. 650–655). IEEE.