Performance effects of imperfect multi-modal sensory cueing in a target detection simulation

Abstract

Past research has shown that multi-modal sensory cues can reduce the workload of the user while simultaneously increasing performance capacity. This study looks to examine how performance is impacted in a multi-modal sensory cueing target detection task in which the cueing automation is imperfect. Twenty-seven undergraduate participants volunteered to take part in the present multi-modal sensory automation target detection task. The independent variables were trial (i.e., three five-minute trial blocks) and the cueing method (i.e., tactile, auditory or a combination of tactile and auditory cueing) used to assist visual search for target detection across three screens. Dependent variables included each participant’s response time and rate of accuracy. Results illustrate a significant decrease in response in the final trial when compared with the first trial. Results also illustrated a decrease in response time in each successive trial compared with the previous, each reflective of learning effects. A one-trial block exposure (5 minutes) to imperfect automation resulted in a response time decrease of 24%, while a two-trial block exposure (10 minutes) resulted in a response time decrease of 38%. Errors of omission results showed significantly lower miss rates in the final trial block when compared with the first trial block. In addition, errors of omission were lower in each successive trial compared with the previous. A one-trial block exposure (5 minutes) to imperfect automation resulted in a decrease in misses of 45%, while a two-trial block exposure (10 minutes) to imperfect automation resulted in a decrease of 65% in such misses. Our results suggest that interchanging multi-modal cues create stronger learning trends in a human–automation system than uni-modal cues. Results also showed that in spite of the automation used, automation failure resulted in a significant performance decrement. Auditory automation cueing failure produced a sevenfold increase in response time, while tactile automation cueing failure and a combination of auditory and tactile automation cueing failure produced a fourfold increase in response time. A speed–accuracy trade-off is not the cause of these results, because auditory automation cueing failure produces a twofold decrease in accuracy, and a combination of auditory and tactile automation cueing failure produced a threefold decrease in accuracy.

Keywords

Auditory cueing tactile cueing imperfect automation target detection visual search

Get full access to this article

View all access options for this article.

References

Lee

See

. Trust in automation: designing for appropriate reliance. Hum Factors 2004; 46: 50–80.

Wickens

. Automation trust, reliability, and attention. Tech. Rep. AHFD-02–14/MAAD-02–2, 2002. Savoy, IL: University of Illinois, Aviation Research Lab.

Parasuraman

Riley

. Humans and automation: Use, misuse, disuse, abuse. Hum Factors 1997; 39: 230–253.

Wickens

. Imperfect and unreliable automation and its implications for attention allocation, information access and situation awareness. Final Technical Report ARL-00–10/NASA-00–2, 2000. Savoy, IL: Aviation Research Lab., University of Illinois.

Rovira

McGarry

Parasuraman

. Effects of imperfect automation on decision making in a simulated command and control task. Hum Factors 2007; 49: 76–87.

Merlo

Wickens

Yeh

. Effect of reliability on cue effectiveness and display signaling. In: proceedings of the 4th annual army federated laboratory symposium, 2000, pp.27–31. College Park, MD: Army Research Laboratory.

Hancock

Warm

. A dynamic model of stress and sustained attention. Hum Factors 1989; 31: 519–537.

Dixon

Wickens

. Automation reliability in unmanned aerial vehicle control: a reliance-compliance model of automation dependence in high workload. Hum Factors 2006; 48: 474.

Wickens

. The structure of attentional resources. In: Nickerson

(ed.) Attention and performance VIII. Hillsdale, NJ: Erlbaum, 1980, pp.39–257.

10.

Wickens

Christopher D

. “Processing resources and attention.” Multiple-task performance (1991): 3–34.

11.

Gellevij

Van Der Meij

De Jong

. Multimodal versus unimodal instruction in a complex learning context. J Exp Educ 2002; 70: 215.

12.

Whitehead

. Parallels between dual coding theory and quantum physics. Encounter 2001; 14: 42.

13.

White

Kehring

Glumm

. Effects of unimodal and multimodal cues about threat locations on target acquisition and workload. Mil Psychol 2009; 21:497–512.

14.

Hancock

Mercado

Merlo

. Improving target detection in visual search through the augmenting multi-sensory cues. Ergonomics 2013; 56: 729–738.

15.

Van Erp

Van Veen

. Vibrotactile invehicle navigation system. Transport Res F Traffic Psychol Behav 2004; 7: 247–256.

16.

Brock

McClimens

McCurry

. Virtual auditory cueing revisited. In: proceedings of the 16th international conference on auditory display, Washington, DC, 2010.

17.

Wickens

Dixon

. Workload demands of remotely piloted vehicle supervision and control: (1) single vehicle performance. Technical report AHFD-02–10/MAD-02–1, 2002. Savoy, IL: University of Illinois, Institute of Aviation, Aviation Human Factors Division.

18.

Calhoun

Fontejon

Draper

. Tactile versus aural redundant alert cues for UAV control applications. In: proceedings of the human factors and ergonomics society 48th annual meeting, 2004, pp.137–141.

19.

Merlo

Duley

Hancock

. Crossmodal congruency benefits for combined tactile and visual signaling. Am J Psychol 2010; 123: 413–424.

20.

Merlo

Hancock

. Quantification of tactile cueing for enhanced target search capacity. Mil Psychol 2011; 23: 137–153.

21.

Poulton

. Influential companions: effects of one strategy on another in the within-subjects designs of cognitive psychology. Psychol Bull 1982; 91: 673.

22.

Stanton

Young

McCaulder

. Drive-by-wire: the case of mental workload and the ability of the driver to reclaim control. Saf Sci 1997; 27: 149–159.

23.

Wickens

Sarter

. Informing the design of multimodal displays: a meta-analysis of empirical studies comparing auditory and tactile interruptions. In: proceedings of the 55th annual human factors and ergonomics society meeting, 2011, pp.1170–1174.