Objective: This investigation sought to develop a reliable and valid classification system for identifying and classifying the underlying causes of pilot errors reported under the Aviation Safety Action Program (ASAP). Background: ASAP is a voluntary safety program that air carriers may establish to study pilot and crew performance on the line. InASAP programs, similar to the Aviation Safety Reporting System, pilots self-report incidents by filing a short text description of the event. The identification of contributors to errors is critical if organizations are to improve human performance, yet it is difficult for analysts to extract this information from text narratives. A taxonomy was needed that could be used by pilots to classify the causes of errors. Method: After completing a thorough literature review, pilot interviews and a card-sorting task were conducted in Studies 1 and 2 to develop the initial structure of the Aviation Causal Contributors for Event Reporting Systems (ACCERS) taxonomy. The reliability and utility of ACCERS was then tested in studies 3a and 3b by having pilots independently classify the primary and secondary causes of ASAP reports. Results: The results provided initial evidence for the internal and external validity of ACCERS. Pilots were found to demonstrate adequate levels of agreement with respect to their category classifications. Conclusions: ACCERS appears to be a useful system for studying human error captured under pilot ASAP reports. Future work should focus on how ACCERS is organized and whether it can be used or modified to classify human error in ASAP programs for other aviation-related job categories such as dispatchers. Application: Potential applications of this research include systems in which individuals self-report errors and that attempt to extract and classify the causes of those events.
Get full access to this article
View all access options for this article.
References
1.
Beaubien, J.M., & Baker, D.P. (2002). A review of selected aviation human factors taxonomies, accident/incident reporting systems, and data reporting tools. International Journal of Applied Aviation Studies, 2(2), 11-36.
2.
CHIRP Charitable Trust (1999). CHIRP Survey - 1999, Feedback, 51. Retrieved January 2, 2007 , from http://www.chirp.co.uk/main/downloads/html/FB51_Supplement.htm
3.
Cohen, J. (1960). A coefficient of agreement for nominal scales . Educational and Psychological Measurement, 20, 37-46.
4.
Ewing, G., Logie, R., Hunter, J., McIntosh, N., Rudkin, S., Freer, Y., & Ferguson, L. (2002). A new measure summarising "information" conveyed in cluster analysis of cart sort data: Application to a neonatal intensive care environment. In P. Lucas, L. Asker, & S. Miksch (Eds.), Intelligent data analysis in medicine and pharmacology IDAMAP 2002 (pp. 25-29). Brussels, Belgium: European Coordinating Committee on Artificial Intelligence (ECCAI).
Fleishman, E.A., & Mumford, M.D. (1991). Evaluating classifications of job behavior: A construct validation of the ability requirement scales. Personnel Psychology, 44, 523-575.
7.
Fleishman, E.A., Quaintance, M.K., & Broedling, L.A. (1984). Methodological issues in developing and evaluating classificatory systems. In Taxonomies of human performance: The description of human tasks (pp. 64-89). San Diego, CA: Academic Press.
8.
Fleiss, J.L. (1981). Statistical methods for rates and proportions (2nd ed.). New York: Wiley.
9.
Gwet, K. (2002). Computing inter-rater reliability with the SAS® system. Statistical Methods for Inter-Rater Reliability Assessment , 3, 1-16.
10.
Helmreich, R.L., Butler, R.E., Taggart, W.R., & Wilhelm, J.A. (1995). The NASA/University of Texas/FAA Line/LOS checklist: A behavioral marker-based checklist for CRM skills assessment (Tech. Paper 94-02). Houston: University of Texas Aerospace Crew Research Project.
11.
Johnson, W. (1997). Classifying pilot human factors causes in A-10 Class A mishaps. Unpublished graduate research project, Embry-Riddle Aeronautical University , Daytona Beach, FL.
12.
Reason, J. (1990). Human error. New York: Cambridge University Press.
13.
Shappell, S.A., & Wiegmann, D.A. (1997). A human error approach to accident investigation: The taxonomy of unsafe operations. International Journal of Aviation Psychology, 7, 269-291.
14.
Shappell, S.A., & Wiegmann, D.A. (1998, April). Failure analysis classification system: A human factors approach to accident investigation . Presented at SAE: Advances in Aviation Safety Conference and Exposition, Daytona Beach, FL.
15.
Shappell, S.A., & Wiegmann, D.A. (1999). Human factors analysis of aviation accident data: Developing a needs-based, data-driven, safety program . In Proceedings of the 3rd International Workshop on Human Error, Safety, and Systems Development [CD-ROM]. Vienna , Austria: International Federation for Information Processing.
16.
Shappell, S.A., & Wiegmann, D.A. (2003). A human error approach to aviation accident analysis. Burlington, VT: Ashgate.
17.
Smith, P.C., & Kendall, L.M. (1963). Retranslation of expectations: An approach to construction of unambiguous anchors for rating scales. Journal of Applied Psychology, 47, 149-155.
18.
Toms, M., Cummings-Hill, M., Curry, D., & Cone, S. (2001). Using cluster analysis for deriving menu structures for automotive mobile multimedia applications. In R. Balci, S. Cook, M. A. Cummings-Hill, C. L. Pattie, M. D. Vala, &A. M. Vertiz (Eds.), Human Factors in Automotive Design (pp. 105-112). Warrendale, PA: SAE International.
19.
Wiegmann, D.A., Rich, A., & Shappell, S.A. (2000). Human error and accident causation theories, frameworks and analytical techniques: An annotated bibliography (Tech. Rep. ARL-00-12/FAA-00-7). Savoy, IL: University of Illinois, Aviation Research Lab.
20.
Wiegmann, D.A., & Shappell, S.A. (2003). Assessing the reliability of the human factors analysis and classification system (HFACS). Aviation, Space, and Environmental Medicine, 72, 1006-1016.
