Imperfect Diagnostic Automation: An Experimental Examination of Priorities and Threshold Setting

Abstract

Diagnostic automation, such as alarms, alerts, or automatic target recognition systems can vary in their reliability and threshold setting, the latter influencing the balance between misses and false alarms. This experiment examined the implications of both of these, when the system detected military targets in parallel with the human operator in an unmanned air vehicle simulator. Unlike previous investigations of this paradigm in a dual task setting, the automation diagnosis task was very difficult, and priority between this task and a concurrent task was explicitly varied, along with the threshold setting. The results revealed that: (a) reliability as low as 0.6 aided human performance. (b) the effects of the automation threshold shift could be partially modeled in terms of the classic reliance-compliance dimensions of automation dependence, at both task priority settings. (c) The priority effects were more pronounced with miss-prone automation. (d) directing attention to false-alarm prone automation actually degrades human-system performance.

Get full access to this article

View all access options for this article.

References

Dixon

S.R.

McCarley

Wickens

C.D.

(2006). How do automation false alarms and misses affect operator compliance and reliance. HFES Proceedings..

Dixon

S.R.

Wickens

C.D.

(in press). Automation reliability in unmanned aerial vehicle flight control. Human Factors.

Goh

Wiegmann

Madhavan

(2005).

Maltz

Shinar

(2003). New alternative methods in analyzing human behavior in cued target acquisition. Human Factors, 45(2), 281–295.

Meyer

(2001). Effects of warning validity and proximity on responses to warnings. Human Factors, 43(4), 563–572.

Meyer

(2004). Conceptual issues in the study of dynamic hazard warnings. Human Factors, 46(2), 196–204.

Navon

Gopher

(1979). On the economy of the human processing system. Psychological Review 86, 254–285.

Pritchett

(2001). Reviewing the role of cockpit alerting systems. Human Factors and Aerospace Safety, 1, 5–38.

Sorkin

R. D.

Woods

D. D.

(1985). Systems with human monitors, a signal detection analysis. Human-Computer Interaction, 1, 49–75.

10.

Wickens

C.D.

Dixon

S.R.

(in press). Is there a magic number 7 (to the minus 1)? The benefits of imperfect diagnostic automation. Theoretical Issues in Ergonomics Science.

11.

Wickens

C.D.

Dixon

S.R.

Ambinder

M.S.

(2006). Workload and automation reliability in unmanned air vehicles. In Cooke

N. J.

Pringle

Pedersen

Connor

(Eds.), Advances in human performance and cognitive engineering research, Vol. 7, Human factors of remotely operated vehicles (pp. 209–222). Elsevier Ltd.

12.

Wickens

C.D.

Dixon

S.R.

Goh

Hammer

(2005). Pilot dependence on imperfect diagnostic automation in simulated UAV flights. Proceedings of the 13^th Annual International Symposium of Aviation Psychology..

13.

Wickens

C.D.

Rantanen

E.M.

(in press). Effects of conflict alerting system reliability and task difficulty on pilots' conflict detection with cockpit display of traffic information. Ergonomics.

14.

Yeh

Wickens

C. D.

(2001). Attentional filtering in the design of electronic map displays: A comparison of color-coding, intensity coding, and decluttering techniques. Human Factors, 43(4). 543–562.