Objective: This study adopts J. Rasmussen's (1985) abstraction hierarchy (AH) framework as an analytical tool to identify problems and pinpoint opportunities to enhance complex systems. Background: The process of identifying problems and generating recommendations for complex systems using conventional methods is usually conducted based on incompletely defined work requirements. As the complexity of systems rises, the sheer mass of data generated from these methods becomes unwieldy to manage in a coherent, systematic form for analysis. There is little known work on adopting a broader perspective to fill these gaps. Method: AH was used to analyze an aircraft-automation system in order to further identify breakdowns in pilot-automation interactions. Four steps follow: developing an AH model for the system, mapping the data generated by various methods onto the AH, identifying problems based on the mapped data, and presenting recommendations. Results: The breakdowns lay primarily with automation operations that were more goal directed. Identified root causes include incomplete knowledge content and ineffective knowledge structure in pilots' mental models, lack of effective higher-order functional domain information displayed in the interface, and lack of sufficient automation procedures for pilots to effectively cope with unfamiliar situations. Conclusion: The AH is a valuable analytical tool to systematically identify problems and suggest opportunities for enhancing complex systems. It helps further examine the automation awareness problems and identify improvement areas from a work domain perspective. Application: Applications include the identification of problems and generation of recommendations for complex systems as well as specific recommendations regarding pilot training, flight deck interfaces, and automation procedures.
Get full access to this article
View all access options for this article.
References
1.
Annett, J. (2005). Hierarchical tasks analysis (HTA). In N. Stanton, A. Hedge , K. Brookhuis, & E. Salas (Eds.), Handbook of human factors and ergonomics methods (pp. 58-75). Boca Raton, FL: CRC Press.
2.
Bureau of Air Safety Investigation. (1998). Advanced technology aircraft safety survey report. Canberra , Australia: Author.
Degani, A., & Heymann, M. (2002). Formal verification of human-automation interaction . Human Factors, 44, 28-43.
5.
Dinadis, N., & Vicente, K. (1999). Designing functional visualizations for aircraft systems status displays. International Journal of Aviation Psychology, 9, 271-290.
6.
Gras, A., Moricot, C., Poirot-Delpech, S.L., & Scardigli, V. (1994). Faced with automation: The pilot, the controller, and the engineer. Paris: Publications of the Sorbonne.
7.
Hajdukiewicz, J.R., Burns, C.M., Vicente, K.J., & Eggleston, R.G. (1999). Work domain analysis for intentional systems. In Proceedings of the Human Factors and Ergonomics Society 43rd Annual Meeting (pp. 333-337). Santa Monica, CA: Human Factors and Ergonomics Society.
8.
Hajdukiewicz, J.R., & Vicente, K. (2004). A theoretical note on the relationship between work domain analysis and task analysis. Theoretical Issues in Ergonomics Sciences, 5, 527-538.
9.
Ho, D., & Burns, C.M. (2003). Ecological interface design in aviation domain: Work domain analysis of automated collision detection and avoidance. In Proceedings of the Human Factors and Ergonomics Society 47th Annual Meeting (pp. 119-123). Santa Monica, CA: Human Factors and Ergonomics Society.
10.
Hutchins, E.L. (1996). The integrated mode management interface (NASAAmes NCC 92-578, Final Rep.). Moffett Field, CA: NASA Ames Research Center.
11.
Infield, S.E., King, W., Palen, L., Possollo, A., Bates, M., & Pfaff, T. (1995). International flight management system usage survey: Airline report. Seattle, WA: Boeing Commercial Airplane Group.
12.
Itoh, J., Yoshimura, S., Ohtsuka, T., & Masuda, F. (1990). Cognitive task analysis of nuclear power plant operators for man-machine interface design. In Proceedings of the ANS Topics Meeting and advances in human factors research on man-computer interactions: Nuclear and beyond (pp. 96-102). La Grange Park, IL: American Nuclear Society.
13.
Jamieson, G.A., & Vicente, K.J. (2005). Designing effective human-automation-plant interfaces: A control-theoretic perspective. Human Factors, 47, 12-24.
14.
Kirwan, B., &Ainsworth, L. (1992). A guide to task analysis. London: Taylor & Francis.
15.
Leveson, N.G. (2000). Intent specifications: An approach to building human-centered specifications. IEEE Transactions on Software Engineering, 26, 15-35.
16.
Mazaeva, N., & Bisantz, A.M. (2003). Modeling automation within an abstraction hierarchy . In Proceedings of the Human Factors and Ergonomics Society 47th Annual Meeting (pp. 649-652). Santa Monica, CA: Human Factors and Ergonomics Society.
17.
McClumpha, A.J., James, M., Green, R.G., & Belyavin, A.J. (1991). Pilots' attitudes to cockpit automation. In Proceedings of the Human Factors Society 35th Annual Meeting (pp. 102-106). Santa Monica, CA: Human Factors and Ergonomics Society.
18.
Morters, K. (1988). B767 flight deck automation research ( NASA Research Paper32:420). Moffett Field, CA: NASAAmes Research Center.
19.
Mumaw, R.J., Boorman, D., & Griffin, J. (2001). Constructing a conceptual framework for "glass cockpit" training. In Proceedings of the 11th International Symposium on Aviation Psychology (pp. 106-112). Columbus, OH : Ohio State University.
20.
Mumaw, R.J., Boorman, D., Griffin, J., & Xu, W. (2000). Training and design approaches for enhancing automation awareness (Boeing Doc. D6-82577). Seattle, WA: Boeing.
21.
Mumaw, R.J., Sarter, N.B., Wickens, C.D., Kimball, S., Xu, W., & Xu, X. (2000). Analysis of pilots' monitoring and performance on highly automated flight decks (NASA Ames Contract NAS2-99074, Final Rep.) . Seattle, WA: Boeing.
22.
Naikar, N., & Sanderson, P.M. (1999). Work domain analysis for training-system definition and acquisition. The International Journal of Aviation Psychology, 9, 241-269.
23.
Naikar, N., & Sanderson, P.M. (2001). Evaluating design proposals for complex systems with work domain analysis. Human Factors , 43, 529-542.
24.
Naikar, N., Pearce, B., Drumm, D., & Sanderson, P.M. (2003). Designing teams for first-of-a kind, complex systems using the initial phases of cognitive work analysis: Case study. Human Factors, 45, 202-217.
25.
Rasmussen, J. (1985). The role of hierarchical knowledge representation in decision making and system management. IEEE Transactions on Systems, Man, and Cybernetics, 15, 234-243.
26.
Rasmussen, J., Pejtersen, A., & Goodstein, L.P. (1994). Cognitive systems engineering . New York: Wiley.
27.
Reising, D.V.C., & Sanderson, P.M. (2000a). Testing the impact of instrumentation location and reliability on ecological interface design: Control performance. In Proceedings of the XIVth Triennial Congress of the International Ergonomics Association and 44th Annual Meeting of the Human Factors and Ergonomics Society (pp. 1.124-1.127). Santa Monica, CA: Human Factors and Ergonomics Society .
28.
Reising, D.V.C., & Sanderson, P.M. (2000b). Testing the impact of instrumentation location and reliability on ecological interface design: Fault diagnosis performance. In Proceedings of the XIVth Triennial Congress of the International Ergonomics Association and 44th Annual Meeting of the Human Factors and Ergonomics Society (pp. 3.591-3.594). Santa Monica, CA: Human Factors and Ergonomics Society .
29.
Sarter, N.B., Mumaw, R.J., & Wickens, D.C. (2007). Pilots' monitoring strategies and performance on automated flight decks: An empirical study combining behavioral and eye-tracking data. Human Factors, 49, 347-357.
30.
Sarter, N.B., Wickens, C., Mumaw, R., Kimball, S., Nikolic, M., & Xu, W. (2003). Modern flight deck automation: Pilots' mental model and monitoring patterns and performance. In Proceedings of the 12th International Symposium on Aviation Psychology (pp. 1031- 1036). Dayton, OH: Wright State University.
31.
Sarter, N.B., & Woods, D.D. (1992). Pilot interaction with cockpit automation: Operational experiences with the flight management system. International Journal of Aviation Psychology, 2, 303-321.
32.
Sarter, N.B., & Woods, D.D. (1994). Pilot interaction with cockpit automation: II. An experimental study of pilots' model and awareness of the flight management system. International Journal of Aviation Psychology, 4, 1-28.
33.
Sarter, N.B., & Woods, D.D. (1995). "How in the world did we ever get into that mode?" Mode error and awareness in supervisory control. Human Factors , 37, 5-19.
34.
Sarter, N.B., & Woods, D.D. (1997). Team play with a powerful and independent agent: Automation surprises on the Airbus A-320. Human Factors, 39, 553-569.
35.
Sarter, N.B., & Woods, D.D. (2000). Team play with a powerful and independent agent: A full-mission simulation study. Human Factors, 42, 390-402.
36.
Scott, W.B. (1995, January). United training stresses cockpit discipline . Aviation Week and Space Technology, 15-18.
37.
Sheridan, T.B., & Parasuraman, R. (2006). Human-automation interaction. In Reviews of human factors and ergonomics (Vol. 1, pp. 89-129). Santa Monica, CA: Human Factors and Ergonomics Society.
38.
Stamatis, D.H. (2003). Failure mode and effect analysis: FMEA from theory to execution. Milwaukee, WI: American Society for Quality Press.
39.
St.-Cyr , O., & Burns, C.M. (2002). Mental models and ecological interface design: An experimental investigation . In Proceedings of the Human Factors and Ergonomics Society 46th Annual Meeting (pp. 270-274). Santa Monica, CA: Human Factors and Ergonomics Society.
40.
Vicente, K.J. (1996). Improving dynamic decision making in complex systems through ecological interface design: Research overview. System Dynamic Reviews, 12, 251-279.
41.
Vicente, K.J. (1999). Cognitive work analysis: Towards safe, productive, and healthy computer based work. Hillsdale, NJ: Erlbaum.
42.
Vicente, K.J. (2002). Ecological interface design: Progress and challenges . Human Factors, 44, 72-78.
43.
Vicente, K.J., Christoffersen, K., & Pereklita, A. (1995). Supporting operator problem solving through ecological interface design. IEEE Transactions on Systems, Man, and Cybernetics, SMC-25, 529-545.
44.
Vicente, K.J., & Rasmussen, J. (1992). Ecological interface design: Theoretical foundations. IEEE Transactions on Systems, Man, and Cybernetics, SMC-22, 589-606.
45.
Wiener, E.L. (1989). Human factors and advanced technology (glass cockpit) transport aircraft (TR 177528). Moffett Field, CA: NASAAmes Research Center.
46.
Wong, W.B.L., Sallis, P.J., & O'Hare, D. (1998). The ecological approach to interface design: Applying the abstraction hierarchy to intentional domains . In Proceedings of the Eighth Australian Conference on Computer-Human Interaction, OzCHI '98 (pp. 144-150). Perth, Australia: Ergonomics Society of Australia.
47.
Xu, W., Dainoff, M.J., & Mark, L.S. (1999). Facilitating complex search tasks by externalizing functional properties of a work domain. International Journal of Human-Computer Interaction, 11, 201-230.
