Distracted Driver Performance to Multiple Alerts in a Multiple-Conflict Scenario

Introduction

A new generation of technologies is envisioned to rapidly transform the driving experience during the next decade. One technology, known as connected vehicles, could be a viable first step (National Highway Traffic Safety Administration, 2011). Connected-vehicle technology comprises both preinstalled and aftermarket wireless devices that enable fast, reliable, and secure communication between vehicles and the transportation infrastructure. A rapid and widespread deployment of this technology could greatly improve the capabilities of advanced driver assistance systems, particularly, collision avoidance systems (CASs). CASs alert drivers to an impending crash threat so that an appropriate avoidance maneuver can be made (Kiefer et al., 1999). Connected-vehicle technology can vastly improve CASs because of how much their effectiveness depends on alert reliability (Cummings, Kilgore, Wang, Tijerina, & Kochhar, 2007; Lee, McGehee, Brown, & Reyes, 2002). By reducing false and nuisance alerts, CASs may become better at reducing crashes and making roads more predictable, less congested, and ultimately, more sustainable (National Highway Traffic Safety Administration, 2011; RITA, 2013; Toma, Swanson, Smith, & Najm, 2010).

Significant advancements in automated vehicles, which assume partial authority from the driver, are also transforming surface transportation (Lee & Merat, 2012). The market appeal of automated vehicles is that they can relieve drivers of the effort of driving. The technology also stands to greatly improve safety by reducing crashes that result from human error, notably, driver inattention. However, if the automation fails, drivers will need to retake control of the vehicle. As such, consideration should be given to how to direct drivers’ attention back to the road, as drivers will likely withdraw from the driving task when the automation is engaged (Llaneras, Salinger, & Green, 2013). CASs could support drivers during and following their transitions to manual driving. Therefore, having a full understanding of the effects of CASs on driver crash avoidance performance is essential, especially as transportation becomes more connected and automated.

Research has shown that CAS effectiveness depends on drivers’ knowledge of the system and whether the crash threat is visible. Studies performed with drivers unfamiliar with CAS alerts found that the alerts can be effective at directing a distracted driver’s attention back to the road and priming the release of the throttle (Lee et al., 2002; Perez, Neale, & Kiefer, 2009). The alerts can also cue drivers to apply the brakes, but this is more likely to occur when the crash threat is visible, such as when alerted to a rear-end conflict (Lee et al., 2002) or an unfolding intersection violation (Perez et al., 2009). Immediate braking is a less common response made by unfamiliar drivers when the crash threat cannot be seen. For instance, Llaneras et al. (2005) found that a backing alert initiated drivers to visually search for a crash threat but did not evoke an automatic braking response because drivers did not see the obstacle in their rear blind spots. By contrast, drivers who are familiar with CAS alerts have exhibited automatic braking and steering responses. Fitch, Hankey, Kleiner, and Dingus (2011) found that drivers made the correct manual response to seven alerts presented through an integrated haptic seat display, even if they did not fully remember the meaning of the alerts. As such, to accelerate drivers’ familiarity with the CASs, it may be worth making short and compelling multimedia tutorials available in the vehicle prior to driving.

In a single-conflict scenario, drivers do not necessarily perform better with individual distinct alerts compared to a single master alert (Cummings et al., 2007). However, individual alerts provide more detail of the conflict and may accelerate understanding of the CAS operation.

The role of CAS alerts in a multiple-conflict scenario (i.e., distinct conflicts occurring in close temporal proximity to each other) has been debated. On one hand, it is believed that individual alerts that notify drivers of each unfolding conflict could direct drivers’ attention to the correct location in the proper sequence so that an appropriate avoidance maneuver can be performed (Fitch et al., 2011; Fitch, Kiefer, Hankey, & Kleiner, 2007; Green et al., 2008). Conversely, any alert presented subsequently to the first alert could startle, confuse, or interfere with drivers’ execution of the avoidance maneuver (Green et al., 2008). Research about the psychological refractory period has shown that a decision prompt presented within 350 ms before a lead vehicle braking can delay a driver’s avoidance maneuver an average of 176 ms (Levy, Pashler, & Boer, 2006). Wiese and Lee (2004) found that drivers delayed their responses to a CAS alert when the CAS alert followed an e-mail alert by 300 ms. Green et al. (2008) also found that alerts preceding and following a lane-change merge (LCM) alert can delay a driver’s response to the LCM alert by 0.5 s. Drivers may also miss the second alert altogether given that they are limited in their ability to quickly process information when under high stress (Hancock & Warm, 1989).

As such, a single master alert generated at the onset of the first conflict may be more effective than individual alerts if the single master alert succeeds at directing drivers’ attention to the road and does not interfere with the avoidance maneuver. However, if the second unfolding conflict is not immediately visible, a second alert may benefit drivers by making them aware of the conflict even if the alert generates a slight delay in central information processing.

We investigated whether CASs should present individual crash alerts in a multiple-conflict scenario or present only one alert in response to the first conflict and suppress any alert(s) to a subsequent conflict(s). This study was designed under the hypothesis that drivers would have difficulty responding to a second alert because their attention would be consumed by performing an avoidance maneuver to the initial alert. Not only does the response maneuver comprise a series of decisions that need to be made (e.g., whether to look first, brake, or steer and to what degree), but the limited time available can make the response maneuver demanding. As such, investigating drivers’ response performance to a second CAS alert using an on-road study stands to generate insight about information processing under high mental workload.

The multiple-conflict scenario used during this study was modeled after a type of lane-change near-crash event reported in Fitch et al. (2009), which analyzed lane-change crashes and near-crashes recorded during the 100-Car Naturalistic Driving Study (Dingus et al., 2006). During the near-crash events reported in Fitch et al., the driver swerved into an adjacent lane to avoid a crash with a suddenly decelerating lead vehicle. However, in performing this evasive maneuver, the driver nearly crashed with a vehicle traveling in the adjacent lane. As compared to non-evasive lane changes, the drivers in these types of near-crashes were observed to use their turn signals less frequently and to look at their blind spots less frequently prior to swerving into the adjacent lane, suggesting that these drivers did not have sufficient time or ability to perform these driving tasks during the evasive maneuver. Figure 1 shows the multiplexed video data collected from a lane-change near-crash event. The top right video quadrant shows the lead vehicle that the driver is trying to avoid by swerving into the left lane. The adjacent vehicle traveling in the left lane can be seen through the participant’s left window in the top left video quadrant.

OPEN IN VIEWER

Figure 1.

Multiple-conflict near crash recorded in the 100-Car Study.

An integrated CAS could provide an effective countermeasure to this type of scenario. First, a forward collision warning (FCW) alert could direct the distracted driver’s attention to the decelerating lead vehicle. If stopping distance is limited, the driver might swerve left, generating an LCM alert in response to the emerging conflict with an adjacent vehicle. The LCM alert is intended to lessen the severity of the sideswipe conflict by prompting the driver to steer away from the threat and to allow the adjacent vehicle to straddle its lane and the shoulder of the road. It is worth noting that it may be appropriate to generate an LCM alert once a steering maneuver is initiated rather than once the lane markings are crossed, even though a lateral conflict does not emerge until the lane markings are crossed. This is because Fitch et al. (2009) found that sideswipe near-crashes can unfold, on average, within 2.3 s of initial steering input. Therefore, delaying the LCM alert until the lane markings are crossed may not provide sufficient time for the driver to execute an avoidance maneuver.

Method

Procedure

After consenting to participate in a study evaluating in-vehicle warning systems, participants were instructed to drive at 64.4 km/h and follow a truck at a specified distance (i.e., five car lengths). Participants were also informed that another vehicle would be traveling in the adjacent lane. The driver of the adjacent vehicle adjusted the vehicle’s speed to remain five car lengths behind the participant vehicle. Participants were asked to maintain an awareness of both the lead vehicle and the adjacent vehicle. Participants completed one half lap (2.2 miles) on the Virginia Smart Road to become familiar with the test vehicle and the scenario. Participants then performed two training sessions: one that focused on responding to an FCW alert and another that focused on responding to an LCM alert. The order of the two training sessions was counterbalanced.

The FCW alert training session consisted of the lead pickup truck slowly decelerating and the in-vehicle experimenter discretely using a remote control to generate an FCW alert. This Wizard-of-Oz technique (Salber & Coutaz, 1993) enabled drivers to believe that the CAS was generating the alert, not the experimenter. Participants were instructed to change into the left lane and pass the decelerating lead vehicle in response to the FCW alert. This maneuver was performed four times spanning 2.5 laps of the road. Between trials, participants performed various nondriving tasks, such as searching for a song on an iPod, interacting with a Dell tablet PC mounted to the vehicle’s dash, and setting the vehicle’s temperature using the heating, ventilating, and air-conditioning controls. Participants were also asked at various points if there was a vehicle in their blind spots (as noted, the adjacent vehicle always remained five car lengths back).

The LCM alert training session involved the adjacent vehicle traveling in the participant’s blind spot. Participants were asked to accelerate and safely merge in front of the adjacent vehicle. Participants performed this maneuver six times. However, during four of the trials, participants received an LCM alert signifying a rapidly approaching vehicle. Participants were instructed to quickly return to the original lane. The adjacent vehicle would then accelerate past the participant’s vehicle so that the relationship between the alert and the simulated threat was understood. It should be noted that the lead pickup truck also accelerated during each lane-change maneuver to maintain a safe distance from the participant’s vehicle. Between lane-change trials, the adjacent vehicle would fall back five car lengths, and participants would perform various nondriving tasks. Also when the adjacent vehicle was traveling five car lengths back, participants were asked at various points if there was a vehicle in their blind spots.

Participants then followed the lead truck while the adjacent vehicle trailed 10 car lengths behind the participant vehicle. Participants were asked if there was a vehicle in their blind spots, forcing them to notice that the adjacent vehicle was farther back than usual. Participants were then asked to locate an application that did not exist on the tablet PC. As the experimenter instructed the participant to perform the task, the adjacent vehicle accelerated into the participant’s blind spot (unknown to the participant) but hugged the left shoulder of the road, thus creating one lane of room. Once the participant lowered his or her eyes to begin the visual search task, the experimenter remotely triggered the lead pickup truck to silently lower the tailgate and drop a cardboard box on the road (Figure 2). An FCW alert was programmed to activate as the box landed on the road. In reacting to the surprise event, it was anticipated that participants would swerve left. Therefore, half of the participants were assigned to receive an LCM alert once they initiated a left swerve steering maneuver.

OPEN IN VIEWER

Figure 2.

Multiple-conflict scenario used.

Once the box was passed or struck or the vehicle came to a stop in front of the box, participants were debriefed about the additional purpose of the study. All participants consented to continue with the study, which involved completing a questionnaire and performing the same braking maneuver to the box. This time, participants received the alternative alerting approach. After executing the second braking maneuver, another questionnaire was completed. Participants then drove the test vehicle back to the parking lot, were thanked for their participation, and were paid $20 per hour for their time. The protocol was approved by the Virginia Tech Institutional Review Board. Pilot tests demonstrated that the box could be struck with minimal energy transferred to the driver. A low-profile sand bag was attached to the bottom of the box to deter it from bouncing out of the lane.

Results

Of the 51 drivers who participated in this experiment, the box drop failed for 2, 1 withdrew from the study prior to the box drop event due to an illness, 5 swerved right to avoid the box, 13 stopped in front of the box, and 3 hit the box. A total of 27 drivers swerved left to avoid the box, although one driver swerved left after the confederate vehicle passed. Therefore, 26 drivers were exposed to a multiple-conflict scenario.

Thirteen of the 26 drivers received just the FCW alert, whereas the other 13 drivers received an FCW alert and then an LCM alert after the steering input was initiated (the LCM alert was generated an average of 2.76 s after the FCW alert was generated, SE = 0.18 s, min = 1.46 s, max = 3.92 s). Eleven of the 13 drivers who received only the FCW alert steered right after swerving left to avoid the box, whereas all 13 drivers who received both the FCW and LCM alerts steered right after swerving left to avoid the box. These frequency counts were not significantly different, χ²(1) = 2.667, p > .05, suggesting that both groups became equally aware of the adjacent threat.

The mean amount of time drivers took to steer right after swerving left was 2.76 s for the drivers who received only the FCW alert (n = 11, SE = 0.41 s, min = 0.86 s, max = 5.36 s) and 1.70 s for drivers who received both alerts (n = 13, SE = 0.31 s, min = 0.35 s, max = 4.64 s; Figure 3). This 1.06-s difference in mean time to steer was statistically significant, F(1, 22) = 4.4, p < .05, suggesting that drivers who received the LCM alert were faster to correct their initial responses relative to their counterparts. Note that it was a coincidence that the mean time drivers took to steer right after swerving left in the FCW-only alert scenario was the same as the mean time that elapsed between the FCW and LCM alerts.

OPEN IN VIEWER

Figure 3.

Mean amount of time drivers took to steer right after swerving left.

Using a lane-tracking tool, the maximum distance traveled into the left lane by the left side of the participant vehicle was found to be 83 cm for the drivers who received only the FCW alert (SE = 28 cm, n = 13, min = −18 cm, max = 292 cm), whereas it was 75 cm for the drivers who received both alerts (SE = 21 cm, n = 13, min = −58 cm, max = 225 cm). This 8-cm difference in mean lane departure was not statistically significant, F(1, 24) = 0.05, p > .05. Note that the test vehicle was 183 cm wide and that a negative value represents a vehicle that swerved left but did not pass over the lane markings.

Ten of the 13 drivers who received both alerts looked left (i.e., either to the left side-view mirror, to their left blind spot, or to the rearview mirror) when swerving left to avoid the box. Nine of the 13 drivers who received only the FCW alert looked left when swerving left to avoid the box. A chi-square test did not show these frequency counts to be significantly different, χ²(1) = 0.1955, p > .05. Although this result suggests that the LCM alert did not compel drivers to look left any more than baseline, it is important to note that the LCM alert was activated only when a steering input was made. Therefore, the number of drivers who looked left after a steering input was made was also analyzed. Of those, 8 of the 10 drivers who received both alerts looked left after swerving left to avoid the box, and 8 of the 9 drivers who received only the FCW alert looked left after swerving left to avoid the box. Overall, these findings suggest (a) that drivers are more likely to steer left first, rather than to look left first, when avoiding a hazard on the road and (b) that the LCM alert does not compel drivers to look left any more than baseline.

Discussion

In this study, we investigated drivers’ performance in response to multiple CAS alerts presented in a multiple-conflict scenario. Drivers who received both the FCW and LCM alerts were found to be significantly quicker at steering away from the lateral crash threat than the drivers who received only the FCW alert. Most drivers found the multiple alerts appropriate, with a few drivers indicating that it was confusing to receive multiple alerts close in time. The results suggest that some drivers benefit from, and prefer to receive, multiple alerts when encountering a multiple-conflict scenario.

The multiple-alert approach, however, was not found to affect drivers’ visual scanning behavior of the adjacent lane when making an evasive lane change. The majority of drivers were able to look left (i.e., at the left side-view mirror, their left blind spot, or the rearview mirror) during the execution of the evasive lane change regardless of whether they received the LCM alert. The visual scanning patterns observed may indicate that drivers were mindful of the adjacent vehicle on the road and that they exerted effort to verify they would not collide with the adjacent vehicle. The multiple-alert approach was also not found to affect the distance traveled into the left lane. Both groups of drivers generally straddled the origin and adjacent lane when avoiding the box. This finding further suggests that participants were mindful of the adjacent vehicle during the avoidance maneuver and exerted the least amount of effort to avoid the box.

The shorter time to steer away from the crash threat exhibited by drivers who received the LCM alert may have occurred because of their familiarity with the LCM alert and experience in responding to it. This familiarity may have offset any delay in information processing that arises from processing the alert and the avoidance maneuver at the same time. As such, an LCM alert may be beneficial only for drivers who perform frequent and aggressive lane changes or for drivers who have sufficiently experienced its occurrence over an extended period of time.

The alerting approach tested was based on the approach developed and used in the IVBSS study but lacked a haptic brake pulse and a blind-spot visual alert. Drivers’ response performance in the current study may have been different had participants received multimodal alerts or, possibly, haptic seat alerts, given the ability of the latter method to convey direction more reliably than sound alone (Fitch et al., 2007).

The multiple-alert approach tested can inform not only the development of connected-vehicle system requirements but how to design a takeover request when an automated vehicle is no longer able to perform a specified driving task. Gold, Damböck, Lorenz, and Bengler (2013) investigated driver performance subsequent to an automated vehicle signaling a takeover request to avoid an unexpectedly stopped vehicle on the road. They found that drivers were unlikely to scan the adjacent lane prior to swerving left to avoid the obstacle. It is foreseeable that the multiple-alert approach could support drivers when retaking control of an automated vehicle by notifying them of any adjacent vehicles that may be present. The use of two individual alerts in a multiple-conflict scenario is thus worth investigating further.

Finally, it is worth mentioning that 5 drivers did not remember hearing any alert. Curry, Blommer, Greenberg, and Tijerina (2009) also found that drivers had difficulty recalling that they heard a CAS alert after experiencing an imminent collision situation. Perhaps some drivers subconsciously responded to the alerts or simply forgot about them by the time they were surveyed after being debriefed about the surprise event.

In summary, this study suggests that drivers benefit from, and feel it is appropriate to generate, multiple unique auditory alerts in a multiple-conflict scenario (in which the alerts are spaced more than 2 s apart). Generalizing the results beyond driving, this study supplies evidence that humans can process and respond to an alert generated during a state of high mental workload, provided that the alert and response are intuitively mapped and rehearsed. These findings stand to inform the design of connected and automated vehicles and to move surface transportation one step closer to collision-proof vehicles.

Key Points