An empirical study was conducted to evaluate human factors issues related to the automation visibility and information quality of an information automation system. Information automation is responsible for the collection, processing, analysis, and presentation of information to the flightcrew. Previous analysis had identified a set of characteristics particularly applicable to aircraft flight deck information automation and associated human factors issues. Air transport pilots interacted with an example information automation system in ways that allowed investigation of the information automation characteristics of automation visibility and information quality. The evaluation found that poor information quality appeared to be difficult for participants to detect, even when they were presented with the highest automation visibility level. In the times that they did not successfully compensate, participants tended to over-trust the automation, so when information was missing and they were under high workload, they chose the top plan suggested by the automation even though it was not the truly best plan. Trust in automation was reduced by low information quality, but compensated for by increased automation visibility.
References
1.
AbbottK.McKenneyD.RailsbackP. (2013). “Operational Use of Flight Path Management Systems”, Final Report, Performance-based operations Aviation Rulemaking Committee/Commercial Aviation Safety Team, Flight Deck Automation Working Group.
2.
AndreA. D.CutlerH. A. (1998). Displaying uncertainty in advanced navigation systems. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 42(1), 31-35. SAGE Publications.
3.
BassE. J.PritchettA. R. (2008). Human-automated judge learning: A methodology for examining human interaction with information analysis automation. IEEE Systems, Man and Cybernetics, Part A: 38(4), 759-776.
4.
BatiniC.CappielloC.FrancalanciC.MaurinoA. (2009). Methodologies for data quality assessment and improvement. ACM Computing Surveys (CSUR), 41(3), 16.
5.
BillingsC. E. (1991). Toward a human-centered aircraft automation philosophy. International Journal of Aviation Psychology, 1(4), 261-270.
6.
BillingsC. E. (1997). Aviation automation: The search for a human-centered approach. Lawrence Erlbaum Associates, Mahwah, NJ.
7.
BillingsC. E.WoodsD. D. (1994). Concerns about adaptive automation in aviation systems. Human performance in automated systems: Current research and trends, 264-269.
8.
BisantzA.MMarsiglioS.S.MunchJ. (2005). Displaying uncertainty: Investigating the effects of display format and specificity. The Journal of the Human Factors and Ergonomics Society, 47(4), 777-796.
9.
DeganiA.BarshiI.ShaftoM. G. (2013). Information Organization in the Airline Cockpit Lessons From Flight 236. Journal of Cognitive Engineering and Decision Making, 7(4), 330-352.
10.
DeveansT.KewleyR. (2009). Overcoming information overload in the cockpit (ORCEN Tech. Rep. No. DSE-TR-0904). West Point, NY: U.S. Army.
11.
DorneichM.C.WhitlowS.D.MillerC.A.AllenJ.A (2004). “A Superior Tool for Airline Operations“, Ergonomics in Design. Vol. 12, No. 2, pp. 18-23.
12.
DudleyR.DorneichM.C.Letsu-DakeE.RogersW.WhitlowS.D.DillardM.NelsonE. (2014). “Characterization of Information Automation on the Flight Deck,” Proceedings of the Human Factors and Ergonomics Society Annual Meeting. Chicago, IL, October 27–31.
13.
EndsleyM. R. (1996). Automation and situation awareness. Automation and human performance: Theory and applications, 163-181.
FunkK.LyallB.WilsonJ.VintR.NiemczykM.SuroteguhC.OwenG. (1999). Flight Deck Automation Issues, International Journal of Aviation Psychology, Vol. 9, No. 2, pp. 109-123.
16.
JPDO (2009). Concept of Operations for the Next Generation Air Transportation System. Version 3.0, Washington, DC, 153.
17.
KaberD. B.RileyJ.TanK.EndsleyM. R. (2001). On the design of adaptive automation for complex systems. International Journal of Cognitive Ergonomics, 5, 37–57.
18.
LandryS.J. (2009). “Flight Deck Automation”, Springer Handbook of Aviation (Nof, ed.), Dordrecht: Springer. Chapter 68, pp. 1213-1240.
19.
Letsu-DakeE.RogersW.WhitlowS.D.NelsonE.DillardM.DorneichM.C.DudleyR. (2015). “light Deck Information Automation: A Human-In-The Loop In-Trail Procedure Simulation Study,” 34th Digital Avionics Systems Conference. Prague, Czech Republic, September 13-17.
20.
MyersB. L.KappelmanL. A.PrybutokV. R. (1997). A comprehensive model for assessing the quality and productivity of the information systems function: toward a theory for information systems assessment. Information Resources Management Journal (IRMJ), 10(1), 6-26.
21.
MumawR. J.SarterN. B.WickensC. D. (2001). Analysis of pilots’ monitoring and performance on an automated flight deck. In Proceedings of the Eleventh International Symposium in Aviation Psychology (p. 6). Columbus: Ohio State University.
22.
OckermanJ. J.PritchettA. R. (2002, September). Impact of Contextual Information on Automation Brittleness. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting (Vol. 46, No. 3, pp. 382-386). SAGE Publications.
23.
ParasuramanR.SheridanT.B.WickensC.D. (2000). A model for types and levels of human interaction with automation. IEEE Systems, Man and Cybernetics, Part A, 30(3), 286-297.
24.
PipinoL. L.LeeY. W.WangR. Y. (2002). Data quality assessment.Communications of the ACM, 45(4), 211-218.
25.
ReevesC. A.BednarD. A. (1994). Defining quality: alternatives and implications. Academy of management Review, 19(3), 419-445.
26.
RogersW.Letsu-DakeE.DorneichM.DudleyR.NelsonE.DillardM.WhitlowS.AlmquistJ.HamblinC* OttJ. (2015). “Flight Deck Information Automation: Analysis and Recommendations”, Final Report to Federal Aviation Administration.
27.
RoviraE.ZinniM.ParasuramanR. (2002, September). Effects of information and decision automation on multi-task performance. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting (Vol. 46, No. 3, pp. 327-331).
28.
SarterN. B.WoodsD. D. (1994, March). Decomposing automation: Autonomy, authority, observability, and perceived animacy. Paper presented at the First Automation Technology and Human Performance Conference, Washington, DC.
29.
SarterN. B.WoodsD. D. (1995). How in the world did we ever get into that mode? Mode error and awareness in supervisory control. Human Factors, 37, 5–19.
30.
SarterN. B.WoodsD. D.BillingsC. E. (1997). Automation surprises. In SalvendyG. (Ed.), Handbook of human factors and ergonomics (2nd ed., pp. 1926-1943). New York: Wiley.
31.
ScerboM. W. (1996). Theoretical perspectives on adaptive automation. In ParasuramanR.MoulouaM. (Eds.), Automation and human performance: Theory and applications (pp. 37–63). Mahwah, NJ: Lawrence Erlbaum.
32.
StviliaB.GasserL.TwidaleM. B.SmithL. C. (2007). A framework for information quality assessment. Journal of the American Society for Information Science and Technology, 58(12), 1720-1733.
33.
TenneyY. J.RogersW. H.PewR.W. (1998). Pilot Opinions on Cockpit Automation Issues. International Journal of Aviation Psychology, Vol. 8, No. 2, pp. 103-120.
34.
WandY.WangR. (1996). Anchoring data quality dimensions in ontological foundations. Comm. ACM39, 11.
35.
WangR.StrongD. (1996). “Beyond Accuracy: What Data Quality Means to Data Consumers.” Journal of Management Information Systems, 12(4), p. 5-34.
36.
WhitlowS. D.DorneichM. C.FunkH. B.MillerC. A. (2002). Providing appropriate situation awareness within a mixed-initiative control system. IEEE Systems, Man and Cybernetics Conference (Vol. 5, pp. 5).
37.
WoodsD. D. (1996). Decomposing automation: Apparent simplicity, real complexity. Automation and human performance: Theory and applications, 3-17.
