The Development and Evaluation of Countermeasures to Tactile Change Blindness

Abstract

Objective:

The goal of the present study was to develop and empirically evaluate three countermeasures to tactile change blindness (where a tactile signal is missed in the presence of a tactile transient). Each of these countermeasures relates to a different cognitive step involved in successful change detection.

Background:

To date, change blindness has been studied primarily in vision, but there is limited empirical evidence that the tactile modality may also be subject to this phenomenon. Change blindness raises concerns regarding the robustness of tactile and multimodal interfaces.

Method:

Three countermeasures to tactile change blindness were evaluated in the context of a highly demanding monitoring task. One countermeasure was proactive (alerting the participant to a possible change before it occurred) whereas the other two were adaptive (triggered after the change upon an observed miss). Performance and subjective data were collected.

Results:

Compared to the baseline condition, all countermeasures improved intramodal tactile change detection. Adaptive measures resulted in the highest detection rates, specifically when signal gradation was employed (i.e., when the intensity of the tactile signal was increased after a miss was observed).

Conclusion:

Adaptive displays can be used to counter the effects of change blindness and ensure that tactile information is reliably detected. Increasing the tactile intensity after a missed change appears most promising and was the preferred countermeasure.

Application:

The findings from this study can inform the design of interfaces employing the tactile modality to support monitoring and attention management in data-rich domains.

Keywords

change blindness tactile information presentation multimodal interfaces adaptive displays signal gradation

Introduction

Data overload, especially in the visual channel, represents a major challenge in complex, data-rich domains (e.g., Sarter, 2000, 2007b; Woods, 1995). One promising means of addressing the problem is the introduction of multimodal interfaces that distribute information across multiple sensory channels, primarily vision, audition, and touch. This approach has been shown to be effective in offloading the visual channel, which is increasingly overburdened in several domains (e.g., aviation or the operating room; Ferris & Sarter, 2011; Sarter, 2006a, 2006b, 2007b). Presenting information via this channel helps support functions such as spatial orienting, navigation tasks, and the communication of complex concepts and messages (Jones & Sarter, 2008; Oviatt, 2003; Sarter, 2002). However, the effectiveness of tactile interfaces may be compromised if their design does not take into consideration limitations of human perception and cognition. One such limitation is a phenomenon called change blindness, whereby a person misses a signal when it coincides with another event or disruption (i.e., a transient).

To date, the phenomenon has been studied primarily in vision, but there is now also empirical evidence that the tactile modality can be subject to change blindness. For example, a significant performance decrement was observed when subjects had to distinguish between simple vibrotactile patterns from two to three tactors (devices that present vibration stimuli to the skin) located across the body in the presence of a vibrotactile mask (vibrotactile stimulation from tactors not related to the pattern of interest; Ferris, Stringfield, & Sarter, 2010; Gallace, Auvray, Tan, & Spence, 2006; Gallace, Tan, & Spence, 2007). Even when the tactile stimuli were presented to the highly touch-sensitive fingertip region, change blindness was elicited in the presence of a tactile transient (Malika, Gallace, Hartcher-O’Brien, Tan, & Spence, 2008). Finally, manual control actions, such as pressing a button or turning a steering wheel, have also been shown to elicit tactile change blindness (Gallace, Zeeden, Röder, & Spence, 2010). These findings raise concerns regarding the effectiveness and robustness of tactile and multimodal interfaces. To address this concern, the present study focuses on the development and evaluation of three design-based countermeasures to tactile change blindness, which relate to the cognitive steps involved in successful change detection (Figure 1).

Figure 1.

Cognitive steps required for successful change detection (Jensen, Yao, Street, & Simons, 2011).

The three countermeasures were intended to support various combinations of the first four steps. Since attention is necessary for change detection (Simons, 2000), all three countermeasures were designed to direct attention to the change (Step 1). The first countermeasure, attention guidance, is proactive; it consists of increasing the frequency (i.e., pulse rate) of the tactile cue right before a potential change, with the goal to support encoding of the prechange cue intensity (Step 2). The second and third countermeasures are reactive; that is, they are triggered by an observed failure to notice a change. Countermeasure 2, signal gradation, involves a further increase in the intensity of the tactile signal following a missed change. The third countermeasure, direct comparison, presents the participant with a tactile signal first at the low (prechange) and then at the high (postchange) intensity, with no interval separating the two, if a change was missed. This approach is expected to improve detection rates by supporting relative (as opposed to absolute) judgments and comparisons of cue intensities before and after a change.

The second and third countermeasures represent examples of adaptive information presentation, whereby the timing and/or nature of a signal or display are adjusted automatically in a context-sensitive fashion (Sarter, 2007a; Scerbo, 1996; Trumbly, Arnett, & Johnson, 1994). The need for context-sensitive information presentation has been widely acknowledged (Bennet & Bennet, 2003; Dorneich, Whitlow, Ververs, & Rogers, 2003; Schmorrow & Kruse, 2002). However, no consensus has been reached on the most appropriate and effective implementation of such flexible information presentation.

Gradation, the type of adaptation used for the second countermeasure, consists of varying over time the salience or intensity of a signal to reflect changes in the urgency of the associated task or event. This approach has been shown to be highly effective in alarm design. For instance, Lee, Hoffman, and Hayes (2004) contrasted graded and single-stage tactile warnings in the context of a driving task as part of a collision warning system. The intensity of three tactile warnings (vibrations of the seat) changed over time and corresponded to the severity of the required braking action, that is, high, medium, or negligible. The authors found that graded warnings led to an increased time to collision, indicating a greater margin of safety compared to the single-stage warnings. A similar approach was adopted here by increasing the intensity of the tactile signal following a missed change. Findings from earlier experiments showed that simply repeating the tactile signals (after a change) at the same intensity is not sufficient to ensure change detection (Lu & Sarter, 2014).

Countermeasure 3 is adaptive in that it is triggered by a failure of the participant to notice a change in signal intensity. In this case, the tactile signal following the change is applied first at the low (prechange) and then at the high (postchange) intensity, with no interval separating the two. The goal here is to support Step 4 (in addition to Step 1)—the comparison of the information before and after the change—and to do so without requiring a prolonged retention of a mental representation of the initial signal. Instead, the participant can make a relative judgment of the two signals presented side by side.

These three countermeasures to tactile change blindness were evaluated in the context of monitoring multiple sources of simultaneous information in an event-driven setting. Real-world examples of this task include monitoring of sensor operations, security systems, and video feeds from multiple remote vehicles. For the purposes of this study, we focused on the latter task. Specifically, participants in this study were required to detect two types of targets while monitoring video feeds from nine unmanned aerial vehicles (UAVs).

Hypotheses

We expected the three countermeasures to have the following effects on participants’ performance and preferences:

All three countermeasures were expected to lead to improved performance, compared to no countermeasure, in terms of higher change detection rates (referred to as “hit rates” from this point on) and better multitasking performance.

Correct rejection rates will be higher with attention guidance because this countermeasure is triggered in advance of a change and prepares the participant for making a decision.

Of the three countermeasures, the two adaptive displays—signal gradation and direct comparison—will be preferred over attention guidance because direct comparison will result in some false alarms.

Method

Participants

Twenty undergraduate and graduate students from the University of Michigan participated in this study (13 males and seven females; mean age = 22.8, SD = 2.8). Prior to data collection, all participants signed an informed consent form approved by the university’s institutional review board (Protocol No. HUM00072207). Participants were required to possess normal or corrected-to-normal vision, have no known disorders or injuries that may impair their sense of touch, and have no history of epilepsy (flickering displays may trigger epileptic seizures).

Experimental Setup

Each participant monitored video feeds from nine UAVs and was responsible for responding to long-range radar indications in a simulated combat scenario. Long-range indications were potential targets that could not be seen in a UAV’s field of view but were detected by the UAV’s radar system. The simulation ran on a 20-inch monitor positioned 30 inches from the participant. It displayed nine dynamic UAV feeds (Figure 2).

Figure 2.

Screenshot of the nine unmanned aerial vehicle video feeds on the monitor.

Tactile Display

Twelve C-2 tactors (electromagnetic devices; diameter 2.97 cm; thickness 0.76 cm; Engineering Acoustics, Inc.) applied vibrations to the participant’s back to communicate the (potential) presence of a long-range target. The tactors were attached to a vest (a medical compression garment designed to maintain a consistent pressure on the torso) and belt in a 3 × 3 array (Figure 3). The location of the tactors mapped to the location of the nine video feeds on the monitor. Note that three pairs of tactors (as opposed to individual tactors) were placed in the central column to avoid direct contact with the spine (based on Prinet, Terhune, & Sarter, 2012). Also, the tactors in the top row were not aligned vertically to account for the fact that the participants’ shoulder blades would raise the vest slightly, resulting in tactors at the top of the middle column not being in contact with the person’s back.

Figure 3.

Tactor vest and belt divided into nine sectors to map to the nine unmanned aerial vehicle video feeds on the monitor.

Change Blindness Transient Paradigms: Flicker and Mudsplash

The flicker and mudsplash paradigms used in this study were modeled after earlier visual and tactile change blindness studies (Ferris et al., 2010; Rensink, O’Regan, & Clark, 1997). In the case of a tactile flicker, a vibration (250 Hz, 14 dB) was presented at the same time as the tactile target signal using all nine tactor sectors. The tactile mudsplash employed a random selection of four to six of the tactor sectors at the same intensity (250 Hz, 14 dB; Figure 4). Participants were told that the transient stimuli represented “bugs” or “interference” from the UAVs’ environment and to ignore them to the best of their ability. The tactile transients occurred 2, 4, or 5 s after the start of a trial and lasted 500 ms.

Figure 4.

Depiction of the implementation of tactile flickers and tactile mudsplashes.

Tasks

Participants had to time-share between a long-range and a short-range target detection task. They were instructed that both tasks were of equal importance.

Short-range target indications

Participants were required to search for short-range targets that appeared in the highlighted UAV feed. Short-range targets included armed enemy soldiers, tanks, and other military vehicles (see Figure 5 for examples).

Figure 5.

Examples of short-range targets that would require participants to press the “Tank” button (see Figure 7).

Long-range target indications

A trial started when a UAV radar system detected a potential long-range target. The background of the corresponding UAV feed on the monitor was highlighted, and the tactor corresponding to the relevant video feed began pulsing at a low intensity (250 Hz, 5.4 dB) at a rate of one pulse per 750 ms to alert the participant to a possible threat (an example of visual highlighting is depicted in Figure 6).

Figure 6.

Visual highlighting of screen to direct attention to the unmanned aerial vehicle video feed of interest. Highlighting extinguished at the completion of each trial.

If the potential target was benign (no-change trial), the tactor continued to pulse at the low intensity for the entire 8.5-s trial. Otherwise, if the system ultimately deemed the potential target a threat (change trial), the vibration intensity would increase to a higher intensity (250 Hz, 10.8 dB) 2, 4, or 5 s after the start of a trial and remained at this higher intensity for the remainder of the 8.5-s trial. For change trials, the transient onset occurred simultaneously with the change in vibration intensity.

Participants used the buttons next to the respective UAV feed on the screen to indicate whether or not an intensity change had occurred. Pilot testing confirmed that (a) visual highlighting of the relevant video feed did not result in any performance differences for tactile change detection and (b) tactile changes could reliably be perceived, when presented in isolation, for all tactor locations.

To date, very few studies have involved tactile change detection in the context of multitasking (aside from Ferris et al., 2010), but doing so is important because it replicates the demands experienced by operators in most real-world domains.

Participant Response

When changes in tactile intensity were detected, participants were instructed to press the “Target” button on the top right-hand corner of the respective UAV feed to indicate to Command Central the presence of a long-range target. If there was no change in brightness/intensity, participants were instructed to press the “No Target” button. If participants were unsure whether or not there was a change, they were instructed to press the “?/Unsure” button (see Figure 7 for each of the buttons). Participants could make and change their selection any time during the 8.5-s trial, with the final response being used for data analysis.

Figure 7.

One of the nine unmanned aerial vehicle video feeds with associated response buttons labeled.

Because the attention guidance countermeasure was proactive, participants were instructed to press the Target button after a change in tactile cue intensity was actually detected. In the signal gradation trials, participants were instructed to respond by pressing the Target button if they noticed either the initial change from low to medium tactile intensity or the further increase from medium to high intensity in case of a miss. Similarly, in the direct comparison trials, participants were told to press the Target button if they noticed either the original tactile change or the subsequent direct comparison in case of a miss.

For short-range targets, participants had to press the “Tank” button on the lower right-hand side of the video feed.

Countermeasures to Tactile Change Blindness

The tactile cues indicating the presence of a long-range target either could take the form of the previously described baseline version, or they were augmented by one of three countermeasures. These countermeasures were pilot tested to ensure that they could be reliably detected and accurately interpreted by participants.

Countermeasure 1: Attention guidance

Attention guidance was a proactive measure that was employed during both change and no-change trials (i.e., independent of whether the trial actually involved a change in tactile intensity or not). The cue consisted of pulses at a rate of 1 pulse per 150 ms for 2 s to prepare the participant for a potential change. Figure 8 provides a depiction of the attention guidance cue and its relation to when an intensity change occurred (see Figure 9 for no-change trials).

Figure 8.

Attention guidance for one change trial (dashed lines represent onset and duration of the tactile vibrations).

Figure 9.

Attention guidance for one no-change trial (dashed lines represent onset and duration of the tactile vibrations).

Countermeasure 2: Signal gradation

The signal gradation countermeasure was adaptive in nature; that is, it was triggered 1.5 s after a change if the participant responded incorrectly by indicating that there was no change or by not responding yet. The increased intensity cue was presented at 250 Hz, 16.2 dB, which was close to the maximum gain possible for the C-2 tactors. Figure 10 provides a depiction of the signal gradation and its relation to the three tactile intensity levels.

Figure 10.

Signal gradation for a change trial (dashed lines represent onset and duration of the tactile vibrations).

Countermeasure 3: Direct comparison

Like signal gradation, the direct comparison countermeasure was adaptive and occurred 1.5 s after a change if the participant responded incorrectly by indicating that there was no change or by not responding at all. Figure 11 provides a depiction of the direct comparison and its relation to the two tactile intensity levels.

Figure 11.

Direct comparison for one change trial (dashed lines represent onset and duration of the tactile vibrations).

Procedure

Upon arrival, participants read and signed the consent form. After a brief explanation describing the reason for conducting the study, the experimenter described the visual and tactile cues, described the participants’ tasks, and guided the participants through an interactive demonstration. Next, participants completed two 8-min training sessions that allowed them to practice the tactile baseline combinations, that is, the long-range target change detection task without the presence of transients and countermeasures. Upon completion of the training sessions, participants were given a demonstration of the tactile transients. Participants were asked to wear headphones playing white noise to mask the sound of tactor activation. The participants then completed four blocks of seventy 8.5-s trials: (1) no countermeasure, (2) attention guidance, (3) signal gradation, and (4) direct comparison.

For each countermeasure block, participants were given a demonstration of the countermeasure before the start of the respective block. The order of the four blocks was counterbalanced across subjects. After completing all four blocks, participants filled out a debriefing questionnaire that asked participants to rank in order each of the four tactile displays in terms of the following cue attributes: supporting change detection, annoyance, comfort, and overall preference. In total, the experiment lasted 2 hr.

Experimental Design

This study employed an unbalanced nested design. The two main factors were cue–transient combination (tactile cue only, tactile cue with tactile flicker, tactile cue with tactile mudsplash) and countermeasure type (no countermeasure, attention guidance, signal gradation, and direct comparison). The design was unbalanced in that there was an unequal number of trials for each cue–transient combination. Since we were interested in overcoming change blindness, there were more change trials (56% of all trials) compared to no-change trials (44% trials) in each scenario. A short-range target appeared in 22% of the trials.

Dependent Measures

The dependent measures were accuracy in detection of a change (i.e., target present and Target button selected) and correct rejection when there was no change (i.e., target not present and No Target button selected), multitasking performance, and participant preferences.

Results

Repeated-measures linear models (GLM formulation in SPSS 16.0) were used to identify main and interaction effects. For significant effects, two-tailed Fisher’s LSD post hoc tests were performed to determine differences between means. Display preference was analyzed using a nonparametric Friedman test; a Bonferroni correction was performed for multiple pairwise comparisons.

Hit Rate

There was a significant effect of cue–transient combination, F(2, 18) = 8.62, p = .002, with hit rates being significantly higher in the tactile–baseline combination compared to other intramodal tactile combinations (Figure 12). There was also a significant effect of countermeasure type, F(3, 17) = 18.26, p < .001, with post hoc tests showing that hit rates were the highest for signal gradation (hit rate = 95%) and direct comparison (91%), followed by attention guidance (81%, p < .029 for both pairwise comparisons). Performance was worst with no countermeasure (66%, p = .004). Finally, there was a Cue–Transient Combination × Countermeasure Type interaction, F(6, 14) = 3.27, p = .032, such that in the presence of tactile transients, signal gradation and direct comparison had the highest hit rates, followed by attention guidance and, last, no countermeasure.

Figure 12.

Hit rate for the intramodal tactile cue–transient combinations for change trials (error bars represent standard error; asterisks represent significant countermeasures; countermeasures grouped by brackets are not significantly different from each other).

Correct Rejection Rate

There was a significant effect of cue–transient combination, F(2, 18) = 8.02, p = .003, with correct rejection rates for the tactile mudsplash cases being significantly lower than for the tactile flicker trials (94%, p = .001; Figure 13). There was also a significant effect of countermeasure type, F(3, 17) = 4.30, p < .001, with correct rejection rates being the highest for signal gradation (correct rejection = 97%) and significantly higher than direct comparison and no countermeasure but not attention guidance (92%, p < .029 for both pairwise comparisons). Finally, there was a Cue–Transient Combination × Countermeasure Type interaction, F(6, 14) = 10.25, p < .001. Post hoc tests showed that for the tactile cue–baseline combination, the rejection rates for signal gradation (98%) were significantly higher than no countermeasure (91%, p = .021) and that for the tactile cue–tactile mudsplash combination, rejection rates were significantly lower for the direct comparison, compared to all other countermeasures (p < .001 for all pairwise comparisons).

Figure 13.

Correct rejection rates for the intramodal tactile cue–transient combinations for no-change trials (error bars represent standard error; asterisks represent significant and/or significance between countermeasures).

Multitasking Performance

Figure 14 shows that attention guidance and signal gradation resulted in significantly higher short-range detection rates compared to when there was no countermeasure and with the direct comparison countermeasure (p < .05 for both pairwise comparisons).

Figure 14.

Short-range target detection rates for each tactile display type (bars represent standard error; asterisks represent significant countermeasures; countermeasures grouped by brackets are not significantly different from each other).

Subjective Rankings of Tactile Displays

Rankings differed significantly with respect to how well the countermeasures supported change detection, χ²(3) = 29.70, p < .001. Signal gradation was ranked the highest (mean ranking = 1.35), followed by the direct comparison (2.20), attention guidance (3.15), and no countermeasure (3.30; Figure 15). Signal gradation was also ranked significantly higher than the attention guidance (p < .001), and both the signal gradation and direct comparison were ranked significantly higher than the baseline condition (p < .001 and p = .042, respectively). There was also a significant difference in overall countermeasure preference, χ²(3) = 26.52, p < .001. Again, signal gradation was ranked the highest (mean ranking = 1.35), followed by the direct comparison (2.35), no countermeasure (3.05), and direct comparison (3.25). The signal gradation was also ranked significantly higher than the attention guidance and baseline (p < .001 for both pairwise comparisons). Rankings did not differ with respect to annoyance, χ²(3) = 5.940, p = .115, and comfort, χ²(3) = 7.737, p = .052.

Figure 15.

Mean rankings of each countermeasure for each attribute (with a ranking of 1 being the most favored and 4 being the least favored; asterisks represent significant and/or significance between countermeasures; countermeasures grouped by brackets are not significantly different from each other).

Discussion

The goal of this study was to design and evaluate three countermeasures to intramodal tactile change blindness. The phenomenon has been demonstrated in earlier studies (e.g., Auvray, Gallace, Hartcher-O’Brien, Tan, & Spence, 2008; Ferris et al., 2010; Gallace et al., 2006) and raises concerns about the robustness of tactile and multimodal displays (e.g., Lu, 2014; Lu & Sarter, 2014), which are increasingly introduced in a wide range of domains.

The experiment confirmed the occurrence of intramodal tactile change blindness (Gallace et al., 2006). All three countermeasures significantly improved change detection performance, compared to the baseline condition. Overall, the two adaptive countermeasures, that is, signal gradation and direct comparison, were most beneficial in terms of aiding change detection. Both attention guidance and signal gradation resulted in an increase in correct rejections but likely for different reasons. In the former case, the countermeasure provided support in no-change trials. With signal gradation, on the other hand, a number of participants indicated that the strategy they adopted was to press the No Target button for every trial and then switch their selection when they detected a change or one of the countermeasures. That way, participants could leave their initial response until the extremely salient signal gradation countermeasure was triggered. Direct comparisons were not as effective for supporting correct rejections, especially in the case of tactile mudsplashes. In the debrief, feedback from participants revealed that only half of them perceived this countermeasure as intended (i.e., equal duration at low and high intensity), which may have led to an increased number of false alarms.

When considering both hit rates and correct rejections, overall accuracy was highest with signal gradation, followed by direct comparison. One likely reason for the success of these adaptive measures is that they served as error feedback to participants. Attention guidance prepared participants for a potential change but still required participants to play an active role in monitoring and making a decision. Signal gradation and direct comparison, on the other hand, indicated that an actual change had been missed and thus served as a safe recovery mechanism. This recovery mechanism may have allowed participants to disengage, to an extent, from the task and rely on the countermeasure. This interpretation is in line with earlier observations that the greater the reliability of a system, the more likely operators are to rely and potentially “overrely” on the technology (Parasuraman, Barnes, & Cosenzo, 2007). In this study, the participants’ reliance on the system may be attributed to the fact the automation was designed to appear “trustworthy” based on the design guidelines suggested by Lee and See (2004). The adaptive measures used (a) were based on a simple algorithm invoked in response to an overt operator response and (b) provided immediate results to the operators. For less reliable systems, the participants’ strategy could result in problems, and different countermeasures would need to be developed based on real-time assessments of workload or visual attention to mitigate potential overreliance (Giang et al., 2010; Parasuraman & Wickens, 2008).

The superior performance benefits associated with the two adaptive measures suggest that the various cognitive steps involved in change detection may not be of equal importance or may not be equally likely to break down. Specifically, the early step of encoding the target, which was addressed by the attention guidance measure, may be less critical. Another reason why attention guidance was less effective than the other countermeasures may be its particular implementation. The effectiveness of this approach drastically decreased in the presence of tactile flicker and mudsplash, compared to no transient. This finding suggests that attention guidance may have overwhelmed the participants with the occurrence of three tactile signals—attention guidance, tactile change, and tactile transient—all in close temporal proximity to one another.

The direct comparison was successful in preventing tactile change blindness, which can be explained by the fact that it supported relative judgments. Previous work has shown better performance with relative judgments, which involve a reduced load on memory compared to absolute judgments (Perreault & Cao, 2006). However, it is important to note that correct rejection rates for direct comparisons was lowest when compared to all other countermeasures, including no countermeasure. This finding may be due to the fact that the direct comparison was similar to a gradual change paradigm, that is, a continuous rather than discrete change over time (Simons, 2000). The inability to detect gradual changes has been demonstrated mainly in vision, but there is evidence that people also have difficulty detecting gradual tactile changes (Ferris et al., 2010). The gradual attribute of the direct comparison may have made it difficult to discern when there was a change.

The superior performance observed with signal gradation, followed by direct comparison, is also reflected in participants’ overall preference for these two countermeasures, which was based almost exclusively on how much each measure supported change detection. Comfort and annoyance seem to have had little impact on which countermeasure participants liked, although the rankings were similar to the overall preference. Subjective rankings of the adaptive measures showed that attention guidance was the least preferred method. This result may be due to 30% of the participants feeling that it was more difficult to attend to the tactile cues because the tactile attention guidance created data overload in that channel. Another 15% of participants noted that although attention guidance provided better support for change detection than the baseline, its benefit was nullified in the presence of tactile transients. Thus, attention guidance may have contributed to tactile clutter, which, according to previous literature, can reduce the effectiveness of tactile signals (van Erp, 2002; van Erp, Veltman, van Veen, & Oving, 2003).

In the absence of transients, the findings support using the sense of touch for communication and guiding attention (Geldard, 1957, 1960; Jones & Sarter, 2008). Potential applications include supporting monitoring/surveillance/vigilance tasks, such as military operations (e.g., as part of UAV control), health care (e.g., tele–intensive care units), process and quality control, and in the future, automated driving. The majority of the literature to date has shown the benefits of the tactile channel, but little work has focused on the limitations of this channel. The findings here highlight a major limitation: the inability to reliably attend to concurrent streams of tactile stimuli. This finding may suggest that tactile stimuli are best presented serially, as detection accuracy was best without transients and with the adaptive countermeasures when transients were present. This finding also may suggest that unexpected vibrations could potentially interfere with the ability to detect, differentiate, and appropriately respond to tactile signals.

Authors of further work need to explore more effective design and implementation of these countermeasures to improve accuracy under various contexts and situations. One open question and potential concern with this approach is whether people will experience cue fatigue from the 8.5-s tactile signals and the highest signal intensities over longer periods, especially if they are used to support continuous monitoring tasks (Ferris et al., 2010; McLanders, Santomauro, Tran, & Sanderson, 2014). It will also be critical to ensure that people perceive the signal the same way it was intended by the designer and to minimize the risk of creating tactile clutter. Overall, the results show the promise of adaptive displays, but more work needs to address how to best integrate adaptive design principles, particularly when the system is less reliable.

Key Points

Three types of tactile displays—one proactive and two adaptive measures—were developed to counter the effects of tactile change blindness and were evaluated in the context of monitoring multiple unmanned aerial vehicle video feeds.

All tactile countermeasures improved performance compared to the baseline in terms of hit rate, with the two adaptive measures (signal gradation and direct comparison) resulting in the highest tactile change detection. The success of these adaptive countermeasures may be due to the fact they served as error feedback to participants and were context sensitive, triggering only when the participants responded incorrectly.

Increasing the salience of the tactile signal after a miss with the signal gradation countermeasure was best when considering hit rates, correct rejections, and participant preference. It is recommended that increasing the saliency of the tactile signal be used as a means to counter tactile change blindness.

The attention guidance countermeasure, which increased tactile pulse rate before any potential change, resulted in the highest correct rejection rates. However, this countermeasure was the least preferred display, as participants indicated it caused an overload of tactile information.

Footnotes

Sara Lu Riggs is an assistant professor in the Department of Industrial Engineering at Clemson University. She earned her PhD in industrial and operations engineering from the University of Michigan in 2014.

Nadine Sarter is a professor in the Department of Industrial and Operations Engineering–Center for Ergonomics at the University of Michigan. She received her PhD in industrial and systems engineering from The Ohio State University in 1994.

References

Auvray

Gallace

Hartcher-O’Brien

Tan

H. Z.

Spence

(2008). Tactile and visual distractors induce change blindness for tactile stimuli presented on the fingertips. Brain Research, 1213, 111–119.

Bennet

(2003). Designing the knowledge organization of the future: The intelligent complex adaptive system. In Holsapple

(Ed.), Handbook on knowledge management (pp. 623–638). Heidelberg, Germany: Springer.

Dorneich

M. C.

Whitlow

S. D.

Ververs

P. M.

Rogers

W. H.

(2003). Mitigating cognitive bottlenecks via an augmented cognition adaptive system. In Proceedings of the IEEE Systems, Man, and Cybernetics International Conference (pp. 937–944). Piscataway, NJ: IEEE.

Ferris

T. K.

Sarter

(2011). Continuously informing vibrotactile displays in support of attention management and multitasking in anesthesiology. Human Factors, 53, 600–611.

Ferris

Stringfield

Sarter

(2010). Tactile “change blindness” in the detection of vibration intensity. In Proceedings of the Human Factors and Ergonomics Society 54th Annual Meeting (pp. 1316–1320). Santa Monica, CA: Human Factors and Ergonomics Society.

Gallace

Auvray

Tan

H. Z.

Spence

(2006). When visual transients impair tactile change detection: A novel case of crossmodal change blindness? Neuroscience Letters, 398, 280–285.

Gallace

Tan

H. Z.

Spence

(2007). Do “mudsplashes” induce tactile change blindness? Perception & Psychophysics, 69, 477–486.

Gallace

Zeeden

Röder

Spence

(2010). Lost in the move? Secondary task performance impairs tactile change detection on the body. Consciousness and Cognition, 19, 215–229.

Geldard

F. A.

(1957). Adventures in tactile literacy. American Psychologist, 12, 115–124.

10.

Geldard

F. A.

(1960). Some neglected possibilities of communication. Science, 131, 1583–1588.

11.

Giang

Santhakumaran

Masnavi

Glussich

Kline

Chui

. . . Zelek

(2010). Multimodal interfaces: Literature review of ecological interface design, multimodal perception and attention, and intelligent adaptive multimodal interfaces (Report No. CR 2010–051). Toronto, ON: Defence Research and Development Canada–Toronto.

12.

Jensen

M. S.

Yao

Street

W. N.

Simons

D. J.

(2011). Change blindness and inattentional blindness. Cognitive Science, 2, 529–546.

13.

Jones

L. A.

Sarter

N. B.

(2008). Tactile displays: Guidance for their design and application. Human Factors, 50, 90–111.

14.

Lee

J. D.

Hoffman

J. D.

Hayes

(2004). Collision warning design to mitigate driver distraction. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 65–72). New York, NY: ACM.

15.

Lee

J. D.

See

K. A.

(2004). Trust in automation: Designing for appropriate reliance. Human Factors, 46, 50–80.

16.

S. A.

(2014). Tactile and crossmodal change blindness and its implications for display design (PhD dissertation). University of Michigan, Ann Arbor.

17.

S. A.

Sarter

(2014). Tactile change blindness in an unmanned aerial vehicle control task. In Proceedings of the Human Factors and Ergonomics Society 58th Annual Meeting (pp. 1706–1710). Santa Monica, CA: Human Factors and Ergonomics Society.

18.

Malika

Gallace

Hartcher-O’Brien

Tan

Spence

(2008). Tactile and visual distractors induce change blindness for tactile stimuli presented on the fingertips. Brain Research, 1213, 111–119.

19.

McLanders

Santomauro

Tran

Sanderson

(2014). Tactile displays of pulse oximetry in integrated and separated configurations. In Proceedings of the Human Factors and Ergonomics Society 58th Annual Meeting (pp. 674–678). Santa Monica, CA: Human Factors and Ergonomics Society.

20.

Oviatt

(2003). Multimodal interfaces. In Sears

Jacko

(Eds.), The human–computer interaction handbook: Fundamentals, evolving technologies and emerging applications (2nd ed., pp. 413–432). Boca Raton, FL: CRC Press.

21.

Parasuraman

Barnes

Cosenzo

(2007). Adaptive automation for human–robot teaming in future command and control systems. International C2 Journal, 1(2), 43–68.

22.

Parasuraman

Wickens

C. D.

(2008). Humans: Still vital after all these years of automation. Human Factors, 50, 511–520.

23.

Perreault

J. O.

Cao

C. G.

(2006). Effects of vision and friction on haptic perception. Human Factors, 48, 574–586.

24.

Prinet

J. C.

Terhune

Sarter

N. B.

(2012). Supporting dynamic re-planning in multiple UAV control: A comparison of 3 levels of automation. In Proceedings of the Human Factors and Ergonomics Society 56th Annual Meeting (pp. 423–427). Santa Monica, CA: Human Factors and Ergonomics Society.

25.

Rensink

R. A.

O’Regan

J. K.

Clark

J. J.

(1997). To see or not to see: The need for attention to perceive changes in scenes. Psychological Science, 8, 368–373.

26.

Sarter

N. B.

(2000). The need for multisensory interfaces in support of effective attention allocation in highly dynamic event-driven domains: The case of cockpit automation. International Journal of Aviation Psychology, 10, 231–245.

27.

Sarter

N. B.

(2002). Multimodal information presentation in support of human-automation communication and coordination. Advances in Human Performance and Cognitive Engineering Research, 2, 13–35.

28.

Sarter

N. B.

(2006a). Multimodal information presentation: Design guidance and research challenges. International Journal of Industrial Ergonomics, 36, 439–445.

29.

Sarter

N.B.

(2006b). Visual, tactile, and multimodal information processing. In Marras

W. S.

Karwowski

(Eds.), The occupation ergonomics handbook: Fundamentals and assessment tools for occupational ergonomics (2nd ed., pp. 23.1–23.25). Boca Raton, FL: CRC Press/Taylor & Francis.

30.

Sarter

(2007a). Coping with complexity through adaptive interface design. In Jacko

(Ed.), Human–computer interaction: HCI intelligent multimodal interaction environments (pp. 493–498). Heidelberg, Germany: Springer.

31.

Sarter

N. B.

(2007b). Multiple resource theory as a basis for multimodal interface design: Success stories and qualifications, and research needs. In Kramer

Wiegmann

Kirlik

(Eds.), Attention: From theory to practice (pp. 187–195). New York, NY: Oxford University Press.

32.

Scerbo

M. W.

(1996). Theoretical perspectives on adaptive automation. In Parasuraman

Mouloua

(Eds.), Automation and human performance: Theory and applications (pp. 37–63). Hillsdale, NJ: Lawrence Erlbaum.

33.

Schmorrow

D. D.

Kruse

A. A.

(2002). Improving human performance through advanced cognitive system technology. In Proceedings of the Interservice/Industry Training, Simulation & Education Conference [CD-ROM]. Arlington, VA: I/ITSEC.

34.

Simons

D. J.

(2000). Current approaches to change blindness. Visual Cognition, 7, 1–15.

35.

Trumbly

J. E.

Arnett

K. P.

Johnson

P. C.

(1994). Productivity gains via an adaptive user interface: An empirical analysis. International Journal of Human–Computer Studies, 40, 63–81.

36.

van Erp

J. B.

(2002). Guidelines for the use of vibro-tactile displays in human computer interaction. In Proceedings of EuroHaptics, 2002, 18–22.

37.

van Erp

J. B. F.

Veltman

J. A.

Van Veen

H. A. H. C.

Oving

A. B.

(2003). Tactile torso display as countermeasure to reduce night vision goggles induced drift. In Spatial disorientation in military vehicles: Causes, consequences and cures. RTO Meeting Proceedings (Vol. 86, pp. 49.1–49.8). Neuilly-sur-Seine, France: NATO Research and Technology Organization.

38.

Woods

D. D.

(1995). The alarm problem and directed attention in dynamic fault management. Ergonomics, 38, 2371–2393.