Objective: The objective of the study was to examine pilots' automation monitoring strategies and performance on highly automated commercial flight decks. Background: A considerable body of research and operational experience has documented breakdowns in pilot-automation coordination on modern flight decks. These breakdowns are often considered symptoms of monitoring failures even though, to date, only limited and mostly anecdotal data exist concerning pilots' monitoring strategies and performance. Method: Twenty experienced B-747-400 airline pilots flew a 1-hr scenario involving challenging automation-related events on a full-mission simulator. Behavioral, mental model, and eye-tracking data were collected. Results: The findings from this study confirm that pilots monitor basic flight parameters to a much greater extent than visual indications of the automation configuration. More specifically, they frequently fail to verify manual mode selections or notice automatic mode changes. In other cases, they do not process mode annunciations in sufficient depth to understand their implications for aircraft behavior. Low system observability and gaps in pilots' understanding of complex automation modes were shown to contribute to these problems. Conclusion: Our findings describe and explain shortcomings in pilot's automation monitoring strategies and performance based on converging behavioral, eye-tracking, and mental model data. They confirm that monitoring failures are one major contributor to breakdowns in pilot-automation interaction. Application: The findings from this research can inform the design of improved training programs and automation interfaces that support more effective system monitoring.
Get full access to this article
View all access options for this article.
References
1.
Abbott, K., Slotte, S., Stimson, D., Bollin, E., Hecht, S., Imrich, T., et al. (1996). The interfaces between flight crews and modern flight deck systems (FAA Human Factors Team Report). Washington, DC: Federal Aviation Administration.
2.
Bellenkes, A.H., Wickens, C.D., & Kramer, A.F. (1997). Visual scanning and pilot expertise: The role of attentional flexibility and mental model development. Aviation, Space, and Environmental Medicine, 68, 569-579.
3.
Boorman, D.J., & Mumaw, R.J. (2004). A new autoflight/FMS interface: Guiding design principles. InA. Pritchett &A. Jackson (Eds.), Proceedings of the International Conference on Human-Computer Interaction in Aeronautics [CD-ROM]. Toulouse, France: EURISCO International.
4.
Casner, S.M. (2003). Teaching cockpit automation in the classroom (NASA/TM-2003-211865). Moffett Field, CA: NASA Ames Research Center.
5.
Chappell, A.R., Crowther, E.G., Mitchell, C.M., & Govindaraj, T. (1997). The VNAV tutor: Addressing a mode awareness difficulty for pilots on glass cockpit aircraft. IEEE Transactions on Systems, Man, and Cybernetics, 27, 372-385.
6.
Diez, M., Boehm-Davis, D.A., Holt, R.W., Pinney, M.E., Hansberger, J.T., & Schoppek, W. (2001). Tracking pilot interactions with flight management systems through eye movements. In Proceedings of the 11th International Symposium on Aviation Psychology [CD-ROM]. Columbus: Ohio State University.
7.
Dornheim, M.A. (1995, January 30). Dramatic incidents highlight mode problems in cockpits. Aviation Week and Space Technology, 6-8.
8.
Eldredge, D., Dodd, R.S., & Mangold, S.J. (1991). A review and discussion of flight management system incidents reported to the Aviation Safety Reporting System (Battelle Report prepared for the Department of Transportation). Columbus, OH: Volpe National Transportation Systems Center.
9.
Funk, K., Lyall, E., & Riley, V. (1996). Perceived human factors problems of flight deck automation (Final Tech. Rep. prepared for the FAA). Washington, DC: Federal Aviation Administration.
10.
Harris, R.L., & Christhilf, D. (1980). What do pilots see in displays? In Proceedings of the Human Factors Society 24th Annual Meeting (pp. 22-26). Santa Monica, CA: Human Factors and Ergonomics Society .
11.
Helleberg, J., & Wickens, C.D. (2003). Effects of data-link modality and display redundancy on pilot performance: An attentional perspective. International Journal of Aviation Psychology, 13, 189-210.
12.
Huettig, G., Anders, G., &Tautz, A. (1999). Mode awareness in a modern glass cockpit: Attention allocation to mode information. In Proceedings of the 10th International Symposium on Aviation Psychology [CD-ROM]. Columbus: Ohio State University.
13.
Moray, N. (1986). Monitoring behavior and supervisory control. In K. Boff, L. Kaufman, & J. Thomas (Eds.), Handbook of perception and human performance (Vol. 2, pp. 40-1-40-51). New York: Wiley.
14.
Mumaw, R.J., Boorman, D.J., & Griffin, J.C. (2001). Constructing a conceptual framework for "glass cockpit" training. In Proceedings of the 11th International Symposium on Aviation Psychology [CD-ROM]. Columbus: Ohio State University.
15.
Mumaw, R.J., Boorman, D., Griffin, J., Moodi, M., & Xu, W. (2000). Training and design approaches for enhancing automation awareness (Boeing Document D6-82577). Seattle, WA: Boeing Commercial Aviation.
16.
Mumaw, R.J., Sarter, N., Wickens, C., Kimball, S., Nikolic, M., Marsh, R., et al. (2000). Analysis of pilots'monitoring and performance on highly automated flight decks (Final Project Rep.: NASA Ames Contract NAS2-99074) . Seattle, WA: Boeing CommercialAviation .
17.
Nikolic, M.I., Orr, J.M., & Sarter, N.B. (2004). Why pilots miss the green box: How display context undermines attention capture. International Journal of Aviation , 14(1), 39-52.
18.
Sarter, N.B. (2000). The need for multisensory feedback in support of effective attention allocation in highly dynamic event-driven environments: The case of cockpit automation. International Journal of Aviation Psychology, 10, 231-245.
19.
Sarter, N.B., & Woods, D.D. (1992). Pilot interaction with cockpit automation: Operational experiences with the flight management system (FMS). International Journal of Aviation Psychology, 2, 303-321.
20.
Sarter, N.B., & Woods, D.D. (1994). Pilot interaction with cockpit automation: II. An experimental study of pilots' model and awareness of the flight management system. International Journal of Aviation Psychology, 4, 1-28.
21.
Sarter, N.B., & Woods, D.D. (1995). "How in the world did we ever get into that mode?" Mode error and awareness in supervisory control. Human Factors , 37, 5-19.
22.
Sarter, N.B., & Woods, D.D. (1997). Team play with a powerful and independent agent: Operational experiences and automation surprises on the Airbus A-320. Human Factors, 39, 553-569.
23.
Sarter, N.B., & Woods, D.D. (2000). Team play with a powerful and independent agent: A full-mission simulation study. Human Factors, 42, 390-402.
24.
Sarter, N.B., Woods, D.D., & Billings, C.E. (1997). Automation surprises. In G. Salvendy (Ed.), Handbook of human factors and ergonomics (2nd ed., pp. 1926-1943). New York: Wiley.
25.
Sherry, L., & Polson, P.G. (1999). Shared models of flight management system vertical guidance. International Journal of Aviation Psychology, 9, 139-153.
26.
Sklar, A.E., & Sarter, N.B. (1999). "Good vibrations": The use of tactile feedback in support of mode awareness on advanced technology aircraft. Human Factors, 41, 543-552.
27.
Sparaco, P. (1994, January 3). Human factors cited in French A320 crash. Aviation Week and Space Technology, 30.
28.
Wickens, C.D., Goh, J., Helleberg, J., Horrey, W.J., & Talleur, D.A. (2003). Attentional models of multitask pilot performance using advanced display technology. Human Factors, 45, 360-380.
29.
Wiener, E.L. (1989). Human factors of advanced technology ("glass cockpit") transport aircraft (NASA Contractor Rep. No. 177528). Moffett Field, CA: NASA-Ames Research Center.
