Abstract
Objective
The research examined how humans attribute blame to humans, nonautonomous robots, autonomous robots, or environmental factors for scenarios in which errors occur.
Background
When robots and humans serve on teams, human perception of their technological team members can be a critical component of successful cooperation, especially when task completion fails.
Methods
Participants read a set of scenarios that described human–robot team task failures. Separate scenarios were written to emphasize the role of the human, the robot, or environmental factors in producing the task failure. After reading each scenario, the participants allocated blame for the failure among the human, robot, and environmental factors.
Results
In general, the order of amount of blame was humans, robots, and environmental factors. If the scenario described the robot as nonautonomous, the participants attributed almost as little blame to them as to the environmental factors; in contrast, if the scenario described the robot as autonomous, the participants attributed almost as much blame to them as to the human.
Conclusion
We suggest that humans use a hierarchy of blame in which robots are seen as partial social actors, with the degree to which people view them as social actors depending on the degree of autonomy.
Application
The acceptance of robots by human co-workers will be a function of the attribution of blame when errors occur in the workplace. The present research suggests that greater autonomy for the robot will result in greater attribution of blame in work tasks.
Introduction
As social animals, humans spend much of their time and effort interacting with one another. Consequently, social interaction plays a central role in routine human activities. A number of psychological processes direct human behavior in social situations, thereby guiding human social interaction (e.g., Fazio & Olson, 2003; Heider, 1958; Sieverding, Decker, & Zimmermann, 2010). Research in social psychology has focused on identifying those psychological processes and their contribution to how people interact across a multitude of contexts, including attribution theory (Heider, 1958), theories of prejudice (Fazio & Olson, 2003), and various “social norms” (Sieverding et al., 2010). These processes, which provide benefits to humans, appear to be shaped by evolution to a degree (Neuberg, Kenrick, & Schaller, 2010). In addition, social processes may lead to successful behaviors which are reinforced in social situations. To the extent that these processes are successful early in life, they are likely to be acquired by each individual during childhood. However social processes are acquired, they assure that humans are well suited to interact with other humans.
What occurs when humans interact with another entity that might activate social processes, but which is not human? Specifically, what occurs when humans interact with robots? Humans have interacted with robots or robot-like machines for only a few decades. Accordingly, humans are likely to transfer the social processes typically reserved for human–human interaction to those occasions when they interact with robots. Similarly, Nass, Steuer, and Tauber (1994) have suggested that people using computers view those devices as social actors. Their argument was that computer users are not mistaking computers for humans, but that computers evoke social processes in their users. Or, in our terms, computers, robots, and other technological devices activate social processes in human users because those are the processes that we have to interact with entities in the world that appear to us to have some degree of sentience or autonomy.
The Robot Institute of America (1980) defined a robot as “a reprogrammable multifunctional manipulator designed to move material, parts, tools, or specialized devices, through variable programmed motions for the performance of a variety of tasks.” A more condensed definition provided by Bekey (2005) is “a machine that senses, thinks, and acts” (p. 2). Robots in this sense have been increasing in influence exponentially (Acemoglu & Restrepo, 2017). They have moved into a number of places in daily life so that people have little choice but to engage with them in a collaborative context (Breazeal, 2011; Fortunati, Esposito, & Lugano, 2015; Pieska, Luimula, Jauhiainen, & Spiz, 2012).
Robots can further be sub-defined by their level of autonomy. Autonomous robots are “intelligent machines capable of performing tasks in the world by themselves, without explicit human control over their movements” (Bekey, 2005, p. 1). Not only is robotic influence increasing as a whole, but robots are becoming increasingly more autonomous (Acemoglu & Restrepo, 2017; Fortunati et al., 2015). Autonomy seems to be a defining trait in humans (Ryan & Deci, 2000). Accordingly, an increase in robot autonomy is likely to be followed by an increase in the use of social processes in human–robot interaction (HRI; Breazeal, 2004; Fong, Nourbakhsh, & Dautenhahn, 2003).
Research has shown that humans import social processes to their interactions with computers (Friedman, 1995; Nass & Moon, 2000; Nass et al., 1994). Nass and Moon (2000) performed a series of studies which showed that humans automatically apply social processes such as politeness and reciprocity toward computers, even if they are unaware of doing so. Nass et al. (1994) found similar results, reporting that, although humans have social responses to computers, they are not always conscious of such actions nor do they indicate a belief that computers are human. Social responses include, but are not limited to, gender stereotyping, differentiating between “self” and “other,” and politeness. The authors suggest that their results should not be interpreted as a form of social deficiency but rather as organic social interaction to a perceived social situation. Friedman (1995) examined the process of moral reasoning and accountability in the context of human–computer interaction (HCI). The study involved a pair of scenarios in which a computer system was charged with making a complex decision. Post-scenario interviews showed that participants attributed to computers social processes such as intentions and decision-making capabilities, though all participants indicated that the decision making of the computer was different in one or more ways from that of humans. Results also indicated that participants attributed blame to the computer consistently only 21% of the time when a computer-based error occurred. Reasons for blaming the computer were typically centered around the computer’s role in a mistake which caused human harm, whereas human blame typically fell under the category of negligence in failing to meet an expected performance criteria. This study not only reinforces the notion that humans treat computers as social actors but also that they consider them accountable for errors in at least some situations.
In a study designed to examine attribution of blame in HRI, Kim and Hinds (2006) used a Wizard of Oz technique that involved a remotely controlled robot that varied in perceived degree of autonomy. In a Lego assembly task in which a robot moved between four assembly stations at which participants worked, the high autonomy version of the robot appeared to accept correct assemblies and reject incorrect ones, then moved to the next station when the interaction was complete. In contrast, the low-autonomy version did not judge the acceptability of the assembly and waited for the participant to tell it to move to the next station. After the task was finished, participants completed a questionnaire in which they rated the robot, themselves, and the other human workers on blame for errors and problems in assembly and on credit for success and accomplishments. In general, participants attributed more blame to the high autonomy robot than the low autonomy version, as well as less blame to themselves and their human co-workers when they worked with the high-autonomy robot than the low-autonomy version. Perceived autonomy of the robot also influenced participants’ attribution of credit, but only in that co-workers were given less credit when they worked with the high-autonomy robot than with the low autonomy one. Attribution of credit did not differ for ratings of the robot or the participant’s rating of herself. Although this study had high face validity in that the participants actually worked with a robot and performed the assembly task themselves, the reliability appears to be lower. For example, the research report does not describe the rate of errors in assembling the Legos, its distribution across participants, or its relation to the ratings of blame and credit. In addition, the high autonomy group received feedback about their errors—their robot rejected incorrect assemblies and accepted correct ones—whereas the low autonomy group received no such feedback—their robot accepted all of their assemblies. As a consequence, participants in the two conditions may have had very different ideas about the number of errors.
The research discussed above suggests that social processes are not uniquely applied to human-human interaction; humans repurpose social processes to interactions with technology. As the use of robots increases in everyday tasks, the frequency of interactions between humans and robots in collaborative situations will become more common, perhaps even necessary. Therefore, understanding the social components of HRI, and how our human-to-human processes transfer into these situations becomes essential to understanding human and robot co-operation (e.g., Kim & Hinds, 2006). Attribution of blame for errors may be especially important, given the inevitability of error occurrence. The current study proposes an experimental method similar to that used by Friedman (1995) to investigate how humans attribute blame when an error occurs across a variety of different scenarios in which robots are integrated. One advantage to this design is that it permits a high degree of experimental control so that participants receive scenarios in which only the variables of interest differ between conditions. In addition, the scenarios used in the present experiment included conditions in which aspects of the environment, such as a loose table leg, might serve as the focus of blame for an error. If autonomy plays a key role in the application of social processes to interacting with robots, then people should attribute less blame to robots when they have low autonomy—much like the blame attributed to the environment—but more blame to robots when they have high autonomy—similar to the blame attributed to humans.
Method
Design
An experiment was designed to answer this research question: will autonomy influence the degree to which humans attribute blame to robots for a task-related error? The experiment used a mixed-model design with a between-participants variable of the described level of autonomy for the robot involved in the task and within-participant variables of (1) the entity that served as the focus of fault for the error in the scenario and (2) the setting of the scenario. This research complied with the American Psychological Association Code of Ethics and was approved by the Institutional Review Board at North Carolina State University. Informed consent was obtained from each participant.
Participants
A total of 164 participants volunteered for the experiment. Participants were recruited from Amazon’s Mechanical Turk and were compensated $0.50 for reading the narratives and responding to the questions. Of the 164 participants, 64 read the autonomous scenarios and 100 read the nonautonomous scenarios. For further demographic information, see Table 1.
Demographic Information for the Two Groups
Instruments
To examine the attribution of blame for errors occurring in human–robot team efforts, the experimenters developed 18 scenarios. The experimenters chose to create scenarios rather than use real-life accounts of error on human–robot teams so that specific features could be manipulated and others controlled. Scenarios were written so that either a human operator/team member, a robot, or an environmental factor served as the focus of blame for the task failure. Scenarios were also written across three different contexts—military, surgical, and warehouse setting. The purpose of writing symmetrical scenarios across multiple contexts was to reduce the effects of situation-specific contexts on blame attribution. Finally, one group of participants received scenarios in which the robot was described as being fully autonomous, whereas the other group of participants received scenarios in which the robot was described as being fully dependent on human input. In total, this made for 18 scenarios (two levels of autonomy x three settings x three entities—human, robot, and environment—described as being at fault). Except for the level of autonomy, the nine scenarios (three settings x three entities) for the two groups were identical. The Appendix contains three examples of the 18 scenarios, representing the three settings, the three entities, and both levels of autonomy. All of the scenarios can be read at https://djgillan7.wixsite.com/website/research-resources
Procedure
All parts of the study procedure were executed by means of an online survey. Participants were randomly assigned to a group that read scenarios that described either autonomous or nonautonomous robots. Participants in each autonomy group read all nine scenarios (three settings x each of the three entities at fault) in random order. Following each scenario, participants were asked to determine the amount of blame that they attributed to each entity—human, robot, or environment—for the task failure. They were instructed to give their responses as a percent totaling 100 across the three entities. In addition, participants responded to a comprehensive question in which they chose from a list of descriptors concerning the scenario that they had just read. An incorrect response to this question resulted in that trial being excluded from the analysis.
Results
In general, humans tended to receive a greater share of blame (52.1%), with robots receiving less (28.4%) and the environment receiving the least (19.4%), main effect of Entity Receiving Blame, F(2, 162) = 155.9, p < .0001. A Newman-Keuls test (p = .01) showed that the proportion of blame attributed to the human operator or monitor significantly exceeded that attributed to the robot and the environment, and the blame attributed to the robot significantly exceeded that attributed to the environment.
Combining the data for both groups, the attribution of blame to humans, robots, and the environment was a function of the descriptions in the scenarios, as can be seen in Figure 1. As the left three bars in Figure 1 show, following the scenarios written to focus on the human operator or monitor as responsible for error in the tasks, participants strongly attributed blame to the human. The middle bars show that the robot and human shared the blame in the scenarios written to focus on the robot as responsible for error in the tasks. And the right three bars show that the environment and human shared the blame in the scenarios written to focus on the environment as responsible for error. This pattern of the means for attribution of blame resulted in a significant Focus of Blame x Entity Receiving Blame interaction, F(4, 648) = 127.2, p < .0001. The distribution of blame as a function of the focus of blame in the scenarios indicates that the scenarios worked as intended—on average, each entity received markedly more blame in the scenarios which were written to focus blame on that entity.
Blame for an error in a task given to humans (white bars labeled H), robots (black bars labeled R), and environmental features (striped bars labeled E) as a function of the focus of blame in the task description scenarios.
Perhaps most important relative to the hypothesis concerning autonomy, the attribution of blame across humans, robots, and the environment was influenced by the level of robot autonomy, Entity Receiving Blame x Autonomy interaction, F(2, 162) = 18.6, p < .0001. Figure 2 shows the changes in attribution of blame as a function of autonomy. In the nonautonomous condition, humans received a very high level of blame, whereas the robot and environment received a much lower and similar share of blame. In contrast, in the robot autonomy condition, the robot’s blame was much higher and close to the blame attributed to humans. The blame attributed to the environment remained about the same as in the nonautonomous condition.
Blame for an error in a task given to humans, robots, and environmental features as a function of the described level of autonomy of the robot in the task scenario—nonautonomous (white bar labeled Non) and autonomous (striped bar labeled Auto).
Figure 3 shows the interactions between the blame as attributed to the three entities, human, robot, or environment as a function of the blame described in the scenarios averaged across the three types of scenarios. First, the figure shows a general trend for the robot described to be autonomous to receive more blame in all conditions than the robot described to be nonautonomous—the overall means were 35.9 and 23.6, respectively, t(df = 1,472) = 7.7, p < .0001. Second, the figure shows in both autonomy conditions, the scenario that described the greatest degree of blame for an entity resulted in the highest attribution of blame for that entity. However, in the nonautonomous condition, even though blame for the robot when it was the focus of blame in the scenario was relatively high, and blame for the environment when it was the focus of blame was also relatively high, the human entity received a greater amount of blame. Of greatest interest to the hypothesis that motivated this study, Figure 3b shows that when the robot was not autonomous and was the focus of blame in the scenario, humans were allocated a greater share of the blame for errors than were robots, t(df = 299) = 56.4, p < .0001; in contrast, Figure 3a shows that when the robot was autonomous and was the focus of blame in the scenario, robots were allocated a greater share of the blame for errors than were humans, t(df = 299) = 91.0, p < .0001.
a. Blame for an error in a task given to humans (black bars labeled Hum), robots (white bars labeled Rob), and environmental features (striped bars labeled Env) as a function of the focus of blame in the task description scenarios with robots described as autonomous. b. Blame for an error in a task given to humans (black bars labeled Hum), robots (white bars labeled Rob), and environmental features (striped bars labeled Env) as a function of the focus of blame in the task description scenarios with robots described as nonautonomous.
Discussion
The results support the existence of a social blame hierarchy in which humans tend to be highly likely to be blamed for an error, whereas robots in general receive less overall blame than humans, but more than the surrounding environment. In this hierarchy, humans may be considered a full social actor and the environment a nonactor, with robots falling somewhere between these two poles, with the characteristics of the robot determining its specific position. As a partial social actor, robots would assume some accountability for the outcome of a task, but typically would not be held as accountable as humans. This partial actor hypothesis explains why humans received slightly more blame than robots. The next step is to determine why robots would be perceived in this way.
First, perceived free choice may play a role in robots being perceived as partial social actors. The choices of humans may be seen as more independent and truly free than that of robots. Though robot decision-making is deterministic, it may be that environmental causes are seen as more random and chaotic in comparison to the systematic logic of robot actions. Second, the appearance of robots is typically more humanlike than that of other features in a warehouse, hospital, or military environment. Though our study did not use visualizations of the robots described in the narratives, robots are often conveyed in light of their anthropomorphism both in terms of appearance and motion. Both of these explanations place the perception of robots squarely between that of being human-like and nonhuman-like, consistent with the social blame hierarchy.
Consistent with the previous research by Kim and Hinds (2006), our participants rated autonomous robots as more blameworthy than non-autonomous robots. Accordingly, studies with very different methods—one involving humans and robots in a co-working environment and the other providing a number of different work scenarios—show similar findings that humans will assign more blame to autonomous robots than to nonautonomous ones. A similar line of reasoning as above accounts for differences based on robot autonomy. Autonomous robots may appear to have more freedom of choice than nonautonomous robots due to their independence of action and mimicking of a decision-making process. In addition, nonautonomous robots may be seen as merely an extension of the human operator as opposed to an independent actor. If nonautonomous robots are perceived to be extensions of a human operator, then accountability for the outcome of an action would fall to the human. Thus, the present study is consistent with the hypothesis of a blame hierarchy containing humans, technological devices, and environmental factors in which freedom of action determines the place of the technological devices in the hierarchy.
A substantial amount of recent research has focused on trust in automation (see French, Duenser, & Heathcote, 2018; Lee & See, 2004, for reviews) and in robots (see Hancock, Billings, & Schaefer, 2011; Kuipers, 2018, for reviews). Research on the factors related to trust in human–human, human–automation, and HRIs indicate that ability and reliability are important for trust (e.g., Hancock, Billings, Schaefer, Chen, de Visser, & Parasuraman, 2011; Lee & See, 2004; Mayer, Davis, & Schoorman, 1995; Yagoda & Gillan, 2012). Frequent errors would be likely to reduce the judgment of ability and reliability by humans working with a robot, thereby reducing the trust invested in that robot. Based on the present research and that of Kim and Hinds (2006), humans will more likely blame autonomous robots for errors in a task than they will blame nonautonomous robots. Thus, errors might lead to a greater reduction in trust for autonomous robots than nonautonomous robots.
Applications
The industrialized world may be in a transition period during the next few decades, in which autonomous and semi-autonomous robots will become a much greater part of our work lives and even our daily lives (e.g., Simon, 2018). The acceptance of those robots by human co-workers will be a function of the attribution of blame when errors occur in the workplace. The present research suggests that greater autonomy for the robot will result in greater attribution of blame in work tasks. Given primacy effects in the formation of an impression (Asch, 1946), people who work with autonomous robots for the first time may blame the robots and form negative impressions of the robots, which will make acceptance much more difficult. Such lower levels of acceptance of robots by co-workers and managers could lead to disuse of robots, that is, the failure to use robots when their use would be beneficial for safety and productivity (e.g., Lee, 2008; Parasuraman & Riley, 1997). Accordingly, introduction of robots into the workplace should be done with great care and only in situations in which the robot is most likely to be highly successful.
Limitations of the Study
Research on the actor–observer effect has shown that the perspective a person takes in a scenario can greatly affect that person’s perception of the situation, including how causes are determined and accountability is attributed (Jones & Nisbett, 1971). Future research should also examine blame distribution in HRI as a function of observer perspective. As robots increase in influence, as well as autonomy, understanding how social processes are applied can increase both understanding of the HRI as well as provide useful information to designers of robots.
The present study used written scenarios, an approach that has been useful in studying blame attribution in human–human interaction (e.g., Lagnado & Channon, 2008). However, most participants in a HRI study will have had much less experience with robots than with other humans. Accordingly, future research should examine human attribution of blame to robots in more realistic situations and should use participants with extensive experience interacting with robots.
The research described in this paper used participants recruited from Amazon’s Mechanical Turk (MTurk). Research has shown that MTurk participants are more diverse than undergraduate student samples and are more representative of the United States’ population in age and geographical location, as well as being reliable in their responses (Buhrmester, Kwang, & Gosling, 2011). However, because MTurk participants take part in research via the Internet, they may be more technologically oriented than the general U.S. population. As a consequence, they may have greater knowledge about robots and automation than and attitudes that differ from the general population. Further research should examine the responses to scenarios like the ones in this research as a function of individual differences in knowledge and attitudes about robotics.
One final limitation of the study centers on the use of a mixed-model design. A major independent variable, autonomy of the robot, was a between-participants variable. Having different groups of participants read scenarios in which the robot had the same level of autonomy made it unlikely that participants would realize that robot autonomy was being varied. However, a second major independent variable, the focus of blame in a scenario, was a within-participant variable, so that each participant read some scenarios written with the human operator as the cause of an error, some in which the robot was the cause, and some in which the environment was the cause. Accordingly, participants may have responded with their attribution of blame to the specific entity because of a perception of an implicit demand to respond in a certain way rather than to provide their genuine attributions. A solution for this limitation would be to have both robot autonomy and focus of blame be between-participant variables.
Future Research
In addition to the future research described above to address limitations of the present experiment, other research should expand on the experiment described here and the Kim and Hinds (2006) experiment. Specifically, both studies treated autonomy as a binary variable—low (or no) autonomy versus high autonomy. However, autonomy in technological entities, such as automobiles, is typically considered to have multiple possible levels (see Parasuraman, Sheridan, & Wickens, 2000, for a review). For example, in the case of vehicles, the Society of Automotive Engineers has defined five levels, incrementally changing from a human driver receiving assistance from an automated system, which is also responsible for some driving subtasks (Level 1), to an automated system with the capability of doing all of the driving subtasks that the human driver would normally do under all conditions (Level 5) (SAE International, 2018). Future research should include scenarios or experimental situations that vary in the level of autonomy at more than two levels.
Not only are there different levels of autonomy for robots, but humans may have different levels of autonomy, at least as perceived by others (e.g., Ryan & Deci, 2006). For example, a young child might be viewed by adults as having less autonomy than an adult. Likewise, an adult with schizophrenia might be considered as having less autonomy in behavior as another non-schizophrenic adult. Or, citizens in an autocratic state or even prisoners in a democratic state may be perceived as having reduced autonomy. Future research might also vary the levels of autonomy for human operators of robots in different scenarios. An experiment that systematically and independently varied the autonomy of robots and humans might be able to identify a multidimensional surface for attribution of blame based on perceived autonomy.
The present research also treated errors more simplistically than a human observer of a task might. For example, if I slipped on a patch of ice that was clearly evident in my route, an observer of that behavior would probably strongly attribute that error to me. However, if the ice were not visible, for example, black ice on a highway, slipping on it might not be as strongly attributed as my error. Accordingly, future research might vary the degree to which an error could be anticipated or in which the conditions causing the error were perceptible for either a human or a robot.
Key Points
When allocating blame for an error in a humanrobot team, in general, humans receive the greatest blame, robots the next most, and environmental factors the least.
The allocation of blame to robots is a function of the perceived autonomy of the robot. Nonautonomous robots receive an amount of blame similar to environmental factors, whereas autonomous robots receive an amount of blame similar to humans.
The results are consistent with a hypothesis that people assign blame on the basis of a hierarchy that is based on perceived autonomy or freedom of action.
Footnotes
Appendix
Acknowledgments
The authors thank the anonymous reviewers for suggestions concerning future research involving varying autonomy of robots and humans and investigating the possible effects of the knowledge of participants.
Caleb Furlough is a user researcher at Citrix in Raleigh, NC. He received his PhD in Psychology (Human Factors area) in 2017 from North Carolina State University.
Thomas Stokes is a user experience researcher at UserZoom. He received his PhD in Psychology (Human Factors area) in 2018 from North Carolina State University.
Douglas J. Gillan is a professor in the Department of Psychology at North Carolina State University. He received his PhD in psychology in 1978 from the University of Texas at Austin.