Minding the Gap: Tools for Trust Engineering of Artificial Intelligence

Abstract

There is growing consensus and appreciation for the importance of trust in the development of Artificial Intelligence (AI) technologies; however, there is a reliance on principles-based frameworks. Recent research has highlighted the principles/practice gap, where principles alone are not actionable, and may not be wholly effective in developing more trustworthy AI. We argue for complementary, evidence-based tools to close the principles/practice gap, and present ELATE (Evidence-Based List of Exploratory Questions for AI Trust Engineering) as one such resource. We discuss several tools or approaches for making ELATE actionable within the context of systems development.

Keywords

trust Artificial Intelligence trust engineering foresight naturalistic decision making

Get full access to this article

View all access options for this article.

References

Bettin

Steelman

Wallace

Pontious

Veinott

(2022). Identifying and addressing risks in the early design of a sociotechnical system. Proceedings of the 2022 HFES 66th Annual Meeting, 66(1), 1514–1518. https://doi.org/10.1177/1071181322661307

Blasch

Sung

Nguyen

(2020, November 9). Multisource AI scorecard table for system evaluation. AAAI FSS-20: Artificial intelligence in government and the public sector, Washington, DC, USA.

Bossen

Dindler

Iversen

O. S.

(2016). Evaluation in participatory design: A literature survey. Proceedings of the 14th Participatory Design Conference, 1, 151–160. https://doi.org/10.1145/2940299.2940303.

Chiou

E. K.

Lee

J. D.

(2023). Trusting automation: Designing for responsivity and resilience. Human Factors, 65(1), 137–165. https://doi.org/10.1177/00187208211009995

Danks

(2022). Digital ethics as translational ethics. In Vasiliu-Feltes

Thomason

(Eds.), Applied ethics in a digital world (pp. 1–15). IGI Global. https://doi.org/10.4018/978-1-7998-8467-5.ch001

Delbeq

A. L.

Van de Ven

A. H.

(1975). A group process model for problem identification and program planning. Journal of Applied Behavioral Sciences, 7(4), 466–491. https://doi.org/10.1177/002188637100700404

Dorton

S. L.

(2022). Supradyadic trust in artificial intelligence. Artificial Intelligence and Social Computing, 28, 92–100. https://doi.org/10.54941/ahfe1001451.

Dorton

S. L.

Harper

(2021). Trustable AI: A critical challenge for naval intelligence. Center for International Maritime Security (CIMSEC). Retrieved from. https://cimsec.org/trustable-ai-a-critical-challenge-for-naval-intelligence/

Dorton

S. L.

Harper

S. B.

(2022a). A naturalistic investigation of trust, AI, and intelligence work. Journal of Cognitive Engineering and Decision Making, 16(4), 222–236. https://doi.org/10.1177/15553434221103718

10.

Dorton

S. L.

Harper

S. B.

(2022b). Self-repairing and/or buoyant trust in artificial intelligence. Proceedings of the HFES 66th International Annual Meeting, 66(1), 162–166. https://doi.org/10.1177/1071181322661098

11.

Dorton

S. L.

Harper

S. B.

Neville

K. J.

(2022). Adaptations to trust incidents with artificial intelligence. Proceedings of the HFES 66th International Annual Meeting, 66(1), 95–99. https://doi.org/10.1177/1071181322661146

12.

Dorton

S. L.

Ministero

L. M.

Alaybek

Bryant

D. J.

(2023). Foresight for ethical AI. Frontiers in Artificial Intelligence, 6, Article 1143907. https://doi.org/10.3389/frai.2023.1143907

13.

Ezer

Bruni

Cai

Hepenstal

S. J.

Miller

C. A.

Schmorrow

D. D.

(2019). Trust engineering for human-AI teams. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 63(1), 322–326. https://doi.org/10.1177/1071181319631264

14.

Flanagan

J. C.

(1954). The critical incident technique. Psychological Bulletin, 51(4), 327–358. https://doi.org/10.1037/h0061470

15.

Helmer

(1967). Analysis of the future: The Delphi method. The RAND Corporation. https://apps.dtic.mil/sti/pdfs/AD0649640.pdf

16.

Hoff

K. A.

Bashir

(2015). Trust in automation: Integrating empirical evidence on factors that influence trust. Human Factors, 57(3), 407–434. https://doi.org/10.1177/0018720814547570

17.

Hussein

Shahin

Hoda

Whittle

Perera

Nurwidyantoro

Shams

R. A.

Oliver

(2022). How can human values be addresses in agile methods? A case study on SAFe. IEEE Transactions on Software Engineering, 48(12), 5158–5175. https://doi.org/10.1109/TSE.2022.3140230

18.

Klein

(2022). Snapshots of the mind. MIT Press.

19.

Kurt

(2017). Improving design understandings and skills through enhanced metacognition: Reflective design journals. International Journal of Art and Design Education, 36(2), 226–238. https://doi.org/10.1111/jade.12094

20.

Lee

J. D.

See

K. A.

(2004). Trust in automation: Designing for appropriate reliance. Human Factors, 46(1), 50–80. https://doi.org/10.1518/hfes.46.1.50_30392

21.

Lyons

Highland

Bos

Lyons

Skinner

Schnell

Hefron

(2022). Measuring perceived agent appropriateness in a live-flight human-autonomy teaming scenario. Ergonomics in Design: The Quarterly of Human Factors Applications, Article 106480462211293. https://doi.org/10.1177/10648046221129393

22.

McDermott

P. L.

ten Brink

R. N.

(2019). Practical guidance for evaluating calibrated trust. Proceedings of the Human Factors and Ergonomics Society 2019 Annual Meeting, 63(1), 362–366. https://doi.org/10.1177/1071181319631379

23.

McGregor

(2021). Preventing repeated real world AI failures by cataloging incidents: The AI incident database. Proceedings of the AAAI Conference on Artificial Intelligence, 35(17), 15458–15463. https://doi.org/10.1609/aaai.v35i17.17817.

24.

McNamara

Smith

Murphy-Hill

(2018). Does ACM’s code of ethics change ethical decision making in software development? Proceedings of the 26th ACM Joint Meeting on European Software Engineering Conference on the Foundations of Software Engineering, 729–733. https://doi.org/10.1145/3236024.3264833

25.

Mittelstadt

(2019). Principles alone cannot guarantee ethical AI. Nature Machine Intelligence, 1(11), 501–507. https://doi.org/10.1038/s42256-019-0114-4

26.

Munn

(2022). The uselessness of AI ethics. AI and Ethics, 3(3), 869–877. https://doi.org/10.1007/s43681-022-00209-w

27.

O’Hear

E.H.

Atchley

Tenhundfeld

N.L.

(2022). System-wide trust: The impact of an error in a multi-component system. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 66(10), 1777–1781. https://doi.org/10.1177/1071181322661119.

28.

O’Shaughnessy

(2022). One of the biggest problems in regulating AI is agreeing on a definition. Carnegie Endowment for International Peace. https://carnegieendowment.org/2022/10/06/one-of-biggest-problems-in-regulating-ai-is-agreeing-on-definition-pub-88100

29.

Papautsky

E. L.

Strouse

Dominguez

(2020). Combining cognitive task analysis and participatory design methods to elicit and represent task flows. Journal of Cognitive Engineering and Decision Making, 14(4), 288–301. https://doi.org/10.1177/1555343420976014

30.

Parasuraman

Riley

(1997). Humans and automation: Use, misuse, disuse, abuse. Human Factors, 39(2), 230–253. https://doi.org/10.1518/001872097778543886

31.

Patterson

E. S.

Woods

D. D.

Tinapple

Roth

E. M.

(2001). Using cognitive task analysis (CTA) to seed design concepts for intelligence analysts under data overload. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 45(4), 439–443. https://doi.org/10.1177/154193120104500437

32.

Probasco

E. S.

Toney

A. S.

Curlee

K. T.

(2023). The Inigo Montoya problem for trustworthy AI. Center for Security and Emerging Technology. https://doi.org/10.51593/20230014a

33.

Sawyer

Flanders

Wixon

(1996). Making a difference – the impact of inspections. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 376–382. https://doi.org/10.1145/238386.238579.

34.

Schaefer

K.E.

Chen

J.Y.C.

Szalma

J.L.

Hancock

P.A.

(2016). A meta-analysis of factors influencing the development of trust in automation: Implications for understanding autonomy in future systems. Human Factors, 58(3), 377–400. https://doi.org/10.1177/0018720816634228.

35.

Schneeberger

Rottger

Cabitza

Campagner

Plass

Muller

Holzinger

(2023). The tower of babel in explainable artificial intelligence (XAI). Machine Learning and Knowledge Extraction, 14065, 65–81. https://doi.org/10.1007/978-3-031-40837-3_5.

36.

Shen

Deng

W. H.

Chattopadhyay

Z. S.

Wang

Zhu

(2021). Value cards: An educational toolkit for teaching social impacts of machine learning through deliberation. FAccT, 21, 850–861. https://doi.org/10.1145.3442188.3445971.

37.

Stanley

J. C.

Dorton

S. L.

(2023). Exploring trust with the AI incident database. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 67(1), 489–494. https://doi.org/10.1177/21695067231198084

38.

Steen

Manschot

DeKoning

(2011). Benefits of co-design in service design projects. International Journal of Design, 5(2), 53–60. https://ijdesign.org/index.php/IJDesign/article/viewFile/890/339.

39.

Verberne

F. M.

Ham

Midden

C. J.

(2012). Trust in smart systems: Sharing driving goals and giving information to increase trustworthiness and acceptability of smart systems in cars. Human Factors, 54(5), 799–810. https://doi.org/10.1177/0018720812443825

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.27 MB