Abstract
We provide a transdisciplinary viewpoint on creating artificial social intelligence for human-agent teaming. We discuss theoretical, methodological, and technological insights, drawn from different disciplines, to more fully illuminate how cross-disciplinary research can inform research design and development. We unite ideas spanning human factors, cognitive and computer science, and organizational behavior. Grounding our ideas in real world challenges for human-AI teaming, and via a series of questions designed to facilitate synthesis across disciplines, we illustrate how transdisciplinary team science more effectively asks and answers complex questions on human-agent teaming. Our objective is to contribute to research and development in the field of human-AI and human-robot teaming by emphasizing a more human-centered perspective on AI.
Keywords
Introduction
As artificial intelligence (AI) is increasingly incorporated into the modern workplace (O’Neill, McNeese, Barron, & Schelble, 2022), researchers must better understand how cognition and collaboration are affected by AI. Towards this end, transdisciplinary research communities need to form to more fully examine human-AI, or human-agent teaming (HAT) and its impact on socio-technical systems more broadly (McNeese, Demir, Cooke, & Myers, 2018; Wiltshire, Warta, Barber, & Fiore, 2017). These systems vary in complexity, ranging from dyadic interactions between a human and AI, collaborative interactions, where AI is a member of the team, to more facilitating or coaching interactions where AI is guiding teams. Across these, researchers must examine, not simply how AI alters completion of tasks, but also how AI affects human team members and their interactions (Bendell, Williams, Fiore, & Jentsch, 2021a; Kohn, de Visser, Wiese, Lee, & Shaw, 2021; Schelble, Flathmann, McNeese, Freeman, & Mallick, 2022; Terblanche, Molyn, de Haan, & Nilsson, 2022).
Importantly, there is a solid foundation of research on which to build such studies. In the broader study of teamwork, through most of the 20th century, team research emphasized the study of attitudinal, cognitive, and behavioral processes that emerge from dynamic interactions between team members. In the past few decades there has also been an incorporation of machines and how interactions between humans and various technologies influence collaboration. With significant research coming from the social and organizational sciences, as well as neuroscience and computer science, the study of teams has been developing into a truly transdisciplinary research area spanning levels of analysis (Fiore & Kapalo, 2017).
Unfortunately, rapidly changing technology too often outpaces our ability to study its impact, thus limiting our ability to understand its implications as well as appropriately develop applications. This is certainly an issue with the infusion of AI into collaborative environments where applications are quickly developed and implemented to do more than be a tool for a team. As such, we must transcend current perspectives and view intelligent technology as some form of teammate (Fiore & Wiltshire, 2016). For example, recent research considers the transformation of machines from tools to teammates in their own right, with studies coming out of human-robot interaction research (Phillips et al., 2011; Warta et al., 2016) and humanagent teaming research (McNeese et al., 2018; McNeese, Demir, Cooke, & She, 2021; O’Neill et al., 2022).
Along with this acknowledgement comes recognition of the need to study the attitudinal, behavioral, and cognitive facets of teams and how these change with AI’s involvement. For example, not only might these more intelligent technologies alter behaviors like communication, but they are likely affecting attitudes like trust and cohesion, and cognitive processes like problem solving (Kohn et al., 2021; Schelble et al. 2022; Terblanche et al., 2022; Zhao et al., 2023).
Within this larger research space, we discuss the study of “artificial social intelligence” (ASI) via the integration of theoretical and methodological concepts from across disciplines. ASI consists of social cognitive elements that facilitate group collaboration and broader interpersonal processes. We focus on one of the more foundational features of social cognitive processing in the context of ASI, that of machine theory of mind (e.g., Bendell, Williams, Fiore, & Jentsch, 2021b; Williams, Fiore, & Jentsch, 2022). This is the capacity of an artificial intelligence to attribute mental states and infer the intentions of human members of a team. The study of social intelligence in human-AI teaming is still in its infancy, and largely not understood (Wagner & Briscoe, 2017; Williams et al., 2022; Zhao et al., 2023). To redress this gap, we must transcend disciplines to better study the development and application of ASI in human-agent teams. From this, concepts and methods from varied disciplines can be integrated for the study of such complex systems. This enables a richer examination of how people and technology interact in challenging, high-stakes operational settings. As such, we can improve the understanding and use of human-AI teaming.
Cross-disciplinary research can take many different forms and vary in complexity (Klein, 2004, 2008; Wagner et al., 2011). At its most simple, there is multidisciplinary research, where scientists from different fields pursue a common objective. Here, scientists have complementary expertise and typically work separately or consecutively and come together to share their findings. In the context of human-AI teaming, this might involve a psychologist simply providing conceptual grounding to computer scientists developing AI architectures. Next is interdisciplinary research, which requires more than mere complementarity. This requires teams systematically integrate information, data, techniques, tools, views, concepts, and/or theories from two or more disciplines or bodies of specialized knowledge (National Academy of Sciences, 2004). In the context of human-AI teaming, this would involve computer scientists and social and organizational scientists codeveloping a model of human-machine teaming that draws from an understanding of cognition and coordination to faithfully engineer machine capabilities for collaboration.
But our emphasis is on transdisciplinarity. Transdisciplinary research (TDR) requires more than either complementarity or integrative approaches. It blends and expands upon theories, concepts, and techniques from differing fields and ways of knowing (Lang, Wiek, Bergmann, Stauffacher, Martens, Moll, Swilling, & Thomas, 2011). From this, researchers can pursue development of “a common axiom that transcends individual disciplinary perspectives, typified as an overall synthesis of generic systems” (Klein, 2008, p. S117). Relevant to the human factors community, TDR is problemoriented, crossing not only disciplines but also social sectors to involve a wide range of stakeholders (Stokols et al., 2008). It entails pursuing a research collaboration that crosses analytical levels (e.g., individual, team, and organizational). Further, it often requires translational partners as active participants. In the case of ASI, this would include representatives of the sector where technology like AI would be used (e.g., industry, academia, military). Overall, the goal of TDR is to acquire a richer understanding of the problem being studied in order to more efficaciously develop solutions (Hadorn, Pohl, & Bammer, 2010).
We propose that ASI, in particular, and human-AI teaming in general, are well suited to transdisciplinary research. We discuss how this strategy offers a solid foundation for building teams of scientists interested in fundamental and applied topics for ASI. Specifically, human-agent teaming research places a significant emphasis on multi-level consideration, a focus on complex problems, and the engagement of collaborators from outside science. In the language of cross-disciplinary research, we suggest that the study of ASI be pursued as a particular form of transdisciplinary action research. Here, the aim is to increase the efficiency of teams composed of members crossing, science, engineering, industry, and government, so that they are better able to comprehend and address complex problems (Stokols, 2006). We argue that TDR approaches—where researchers and practitioners can contribute their specialized perspective on human behavior, design, efficacy, and motivation to help other fields develop complex socio-technical systems - represent a significant step forward for the human factors community in developing human-AI teaming. We next discuss the real-world context for our research, a project being conducted by the US Department of Defense that aims to create artificial social intelligence for teamwork.
Artificial Social Intelligence for Successful Teams (ASIST)
The Artificial Social Intelligence for Successful Teams (ASIST) program was created by the Defense Advanced Research Projects Agency (DARPA, 2016) to build fundamental AI theory and create advanced systems showing social skills. This form of machine social competence should demonstrate a capability of inferring goals and needs of teammates and offer suggestions to improve performance on the overall team, while maintaining situational awareness (Williams et al., 2022). To this end, members of the ASIST research teams have been collaborating to develop agents that exhibit machine ToM and the capacity to contribute to the success of a team.
The program created an evaluation testbed for these agents utilizing standardized interfaces and adaptable openworld settings. This consists of a collection of common communication/action channels as well as information streams from virtual sensors accessible to ASIST agents (e.g., mechanisms for agents to convey information to human teammates in the testbed and options they can use to engage with the team). This game-based experimental task was modeled after urban search and rescue (USAR) operations. It required a team to work together while taking on different roles (medic, transporter, and engineer) to find and save victims of a building collapse. The task was carried out in a gamified virtual setting constructed on the framework made available by Minecraft (Mojang, 2015). Teams were made up of three people, and throughout missions could play different roles. To coordinate their efforts, teams could talk via text or voice chat. ASI acted as advisors, supporting teams as they worked to complete their missions by keeping tabs on their objectives and course of action, and sending text messages to intervene when the agent deemed necessary. A description of the testbed as an experimental platform can be found in Carrol et al. (2021) and full description of the testbed environment can be found in Huang et al. (2022).
The program was designed with transdisciplinary research as an overarching objective. That is, in order to develop ASI capable of complex social cognition, the program required collaboration between social and computer scientists and members of the operational community. As noted, the goal was to develop ASI capable of the social-cognitive mechanism known as theory of mind and an understanding of how social circumstances influence the mental state attributions that intelligent agents make. From this, agents should be responsive to rapid changes in the mission or team. And a longer-term goal is ASI capable of building mental models of their environment, including integration of prior knowledge with current observations, contextual cues, and information from teammates. Overall, this requires integration of theory and concepts from the study of teams in the cognitive, computational, and organizational sciences.
This project provided the contextual grounding that helps us provide richer transdisciplinary insights on developing AI capable of social intelligence. We next discuss a set of important ideas coming out of this research and how it informs our general understanding of human-agent teaming. We also identify lessons learned from research involving diverse disciplines. We conclude with a set of research questions meant to move the field forward and help guide human-agent teaming and the development of artificial social intelligence. Our overarching objective is to demonstrate how transdisciplinary research can assist in overcoming this important 21st-century challenge. We next provide a sampling of the findings and representative research questions that can guide the developing field of human-agent teaming.
Disciplinary Perspectives and Findings From Across Disciplines
The ASIST research teams acted as a multi-team system, that is, a team of teams, collaborating across disciplines to study artificial social intelligence. For this form of transdisciplinary research to effectively contribute to creating ASI, it must draw from the cognitive, computational, social, and organizational sciences. From the cognitive sciences, concepts ranging from working memory and workload, to narrative and storytelling, contributed to the research. From cognitive engineering, we focus on understanding, implementing, and measuring teamwork in human-machine systems to improve team performance. From computer science came adaptive software architectures and technology along with physically embedded intelligent autonomous systems. From the organizational sciences, in conjunction with computational thinking, we consider automated algorithms to measure and improve team inputs, processes and outputs. Overall, we conceptualized tools that combine human and artificial intelligence to improve performance of sociotechnical systems. With this overview of the ASIST team’s expertise and disciplinary contributions, we next summarize some of our major research findings.
Research Findings
We next introduce the primary approaches implemented along with findings from the contributing research teams. These include understanding features of individual and team interaction between humans and agents, how knowledge and behavior related to coordination, and how this can be measured.
First, findings suggest that a key component for studying teamwork in human-agent teams is understanding interdependence in joint activity. In this context, artificial social intelligence (ASI) requires understanding task interdependencies, and how team roles manage these. This, then, can be used to guide how ASI manage interactions as joint activity and provide guidance to improve collaboration.
Second, considering cognition in these teams, findings suggest that ASI requires several kinds of knowledge: knowledge of human competencies, knowledge of the tasks, knowledge of what the individuals know about the current situation, social qualities of the individuals, and knowledge of the individual’s current emotional state. Further, these kinds of knowledge can be treated differently by the ASI. For example, theory of mind representations can support what the individuals know, and stories can represent how individuals behave and their effects on team performance. Putting this in the context of collaboration, humans share knowledge about tasks and team interactions by telling stories. In light of ASI, stories can capture essential elements without the details and are reusable in circumstances that share essential details. Because humans are adept at describing their “lessons learned” as short stories, an ASI system that uses short stories can be easily primed to give intelligent advice. Further, the more stories that are added about the task and individuals, the more helpful ASI can be.
Third, a more general model building approach, based upon existing team theory and indicators of team effectiveness, can also inform ASI. Here, research focuses on modelling strategy selection under bounded rationality while teams engaged in the USAR context. Target prioritization and navigation strategies are important drivers of overall team performance. These strategies could be predicted based on a number of factors that an ASI can monitor and use. This includes past experience and utility, strategy shifts based on temporally dynamic mental states (e.g., experience, strategy performance, time constraints, and current utility), individual differences, and environmental changes such as perturbations to the navigation environment. These factors can feed an individual level model that could be integrated with an InputsMediators-Outputs-Inputs (IMOI) model of team effectiveness. From this, it is possible to develop metrics such as emergent leadership, planning, use of transactive memory systems, etc. These could be deployed as analytical components of use to the ASI by integrating static and dynamic inputs such as individual behaviors and conversations between teammates, to produce minute-by-minute metrics of team effectiveness.
Finally, the above requires taking the perspective of environment and evaluation, focusing on how to create a testbed environment that can effectively support ASI research, and how to design studies to best make use of that testbed. We focused on two functions of a testbed, providing a concrete combination of task, instrumentation, and software development environment to drive development of ASI, and enabling collection of data to train and evaluate those ASI. From this approach, the research suggests that ASI has different but complementary strengths as compared to human social intelligence. This applies both to the ability to perceive team behavior and assess teamwork, and the ability to identify interventions that may improve teamwork. These differences pose interesting testbed design challenges that can advance research in human-agent teaming with ASI as collaborator.
Future Directions For ASI Research
In this section we provide a sample of the important issues and questions for future research in ASI. Continuing with our point about ASI research requiring transdisciplinarity, these questions transcend disciplines, bringing together theory, concepts, and methods to more fully develop an understanding of the social and cognitive processes needed for AI to operate as a member of a team.
Building and Testing Competent ASI
Although focusing on social intelligence, what features of humans and/or task, must be incorporated to consider an ASI as competent?
What are the appropriate measures and metrics for assessing whether an ASI is effective?
Can we develop an ASI competency/aptitude rubric?
Can we better assess what makes meaningful improvement to the teamwork/taskwork?
How can we create a performance review framework for ASI/hybrid teams that can be compared across tasks and studies to allow for better evaluation and comparison of applications or improvements?
What is the role of human motivation in ASI and HAT?
How individuals and teams perform is undoubtedly linked to their motivation. How can motivation be measured or inferred from observed behavior with ASI?
How important is the understanding of motivation to predicting team success?
How can an ASI infer motivation, for example, is inverse planning a viable solution?
How much do agents need to know about their team members to support collaboration?
How could data from members playing different team games support reasoning about how they would perform in other games?
Can we recognize traits of a player from prior performance in different games that will predict how they will perform in another, how they will respond to interventions, where they will be strong, and where they will be weak?
Can ASI draw from prior data about team players, such as from after-action reporting, to model and categorize players and to build interventions tailored to the individuals?
What are the design requirements and challenges for studying ASI in complex collaborative testbed environments?
How can we design team activity so that it is visible to ASI within the testbed?
How can the team communicate with each other in ways that enable teamwork but are also perceivable to ASI?
What assumptions about taskwork are embedded in human perception of team activity
What does that mean for what aspects of task definition need to be made machine-readable to ASI?
Is it necessary that an agent be “social” to improve team process? If so, in what ways? How does the addition of an ASI necessarily or implicitly change teamwork dynamics?
How do these dynamics change when considering individual cognition?
What processes do teams/members actually offload to ASI or accommodate ASI in their team?
What are the differences in teamwork/taskwork when ASI is a true teammate vs. an advisor? Is there evidence to support further investment in ASI for hybrid teams?
How can we accurately describe the differing strengths between human advisors and ASI, and how can those strengths be combined?
Some findings suggest that human and artificial social intelligences have different strengths that may be complementary. Can we develop more accurate identification and quantification of those differences will help not only to better evaluate ASI, but also to increase the impact of ASI as a team advisor.
What types of interaction with ASI are most useful for a team and for the ASI?
ASI research to date has involved limited interaction with human team members, with interventions presented with little or no response and feedback mechanism. What types of interaction will be most useful for ASI to develop better models of teams?
What types of interactions will be best for human team members, and increase understanding and trust of ASI?
What are the most impactful aspects of ASI interventions and how can they be measured to demonstrate the value of ASI.
How can the assessment of human-agent team processes be better measured without confounds coming from ASI inputs and/or outcomes.
How does an understanding of interdependence facilitate teamwork and social intelligence.
Generalizability of Research Findings
A continuing challenge is understanding how findings in human-agent teaming research may generalize. This ranges from generalization in other domains in the same testbed, or to different testbeds with multiple domains.
In humans, social intelligence is thought to be generalizable across task domains. How does that apply to ASI, and how much task knowledge is needed before ASI can be effective in its interactions with a team?
Can ASI that is trained on one task domain be brought to another, as long as the interactions between team members have similar characteristics?
More broadly, how generalizable is ASI when considering task knowledge?
Associated with this, we must consider the generalizability/performance trade-off. Specifically, the greater the generalizability, the more likely ASIs have to rely on high-level models with lower accuracy. In this context, then, is there a path to a general ASI?.
Conclusions
It is increasingly recognized that human-centered artificial intelligence is a crucial area for enhancing collaborations between humans and machines (Garibay et al., 2023; Wiltshire et al., 2017). We have discussed a significant aspect of humancentered AI, that of the social-cognitive aspects of human intelligence. This pertains to our capacity to comprehend and engage with others by inferring their mental states, encompassing their thoughts, emotions, and actions (Williams et al., 2022). We emphasize a fundamental aspect of social cognition, that of theory of mind, which involves attributing mental states to others while observing their actions or during interactions. Despite considerable progress in artificial intelligence, they still lack the kind of social awareness necessary for effective collaboration. We argued that, for intelligent systems to genuinely collaborate with humans, they must acquire proficiency in theory of mind. Only then can human-agent teams collaborate effectively and exhibit the same cognitive and coordinative capabilities as human-human teams.
Footnotes
Acknowledgements
This material is based upon work supported by the Defense Advanced Research Projects Agency (DARPA) under Contract No. W911NF-20-1-0008. Any opinions, findings and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of DARPA or the authors’ affiliated University.
