Human–Automation Trust to Technologies for Naïve Users Amidst and Following the COVID-19 Pandemic

Development of Trust in Automation

Trust in automation has been widely studied over the last 30 years (Adams et al., 2003; Hoff & Bashir, 2015; Muir, 1989; Muir & Moray, 1996; Schaefer et al., 2016).

A commonly-referenced definition of human–automation trust is “an attitude that an agent will help achieve an individual’s goals in a situation characterized by uncertainty and vulnerability” (Lee & See, 2004, p. 51). One of the early human–automation trust theories is Muir’s three-dimensional theory (Muir, 1994; Muir & Moray, 1996). The theory bases its foundation on theories of interpersonal trust (e.g., Barber, 1983; Rempel et al., 1985) and proposes that automation trust evolves similarly. That is, automation trust is controlled by three distinct factors: predictability, dependability, and faith. The theory further assumes that, as an operator gains more experience interacting with an automated system, automation trust evolves from predictability to dependability and then faith. Predictability refers to the perceived consistency and desirability of behaviors of the machine, dependability refers to the extent to the stability of machine behaviors based on accumulation of behavioral evidence, and faith refers to the expectation that the machine performs beyond the current situation that operators gathered behavioral evidence, generalizing to future situations.

Muir and Moray (1996) tested this model in a simulated supervisory control environment. They asked naïve participants to control an automated system unfamiliar to them, the pasteurizer plant simulation, where they interacted with the system for several days until their performance reached a predefined criterion. Participants reported their subjective ratings on various constructs including predictability, dependability, faith, and overall trust after each interaction. Surprisingly, Muir and Moray’s (1996) results indicated the opposite of the data pattern predicted by their model. Trust was first governed by faith, then dependability, and finally, predictability, as they became accustomed to controlling the automated system. Faith emerged as the best predictor of trust after the first training session, when presumably operators possessed limited system understanding. Although Muir and Moray (1996) attributed this finding to failure in their experiment, we argue that automation trust actually develops from faith, especially for naïve users who may likely do not have full access to underlying processes of the automated system. Muir and Moray’s (1996) results mirror the current situation that naïve users of automation, such as the general public, are interacting with and adopting new technologies, but having little knowledge capabilities and expectation of the technologies. Therefore, their unexpected findings seem to underscore that initial information and early interactions can control how the naïve users develop their trust in automation (e.g., Dzindolet et al., 2002).

Building off Muir’s theory, Lee and Moray (1992), further elaborated by Lee and See (2004), proposed four dimensions of trust in human–machine systems: foundation, performance, process, and purpose. The foundation of trust refers to fundamental assumptions of natural and social order that allow trust to arise. Of current focus are the three informational bases of automation trust: performance, process, and purpose, mapping onto predictability, dependability, and faith in the Muir’s theory, respectively. Briefly, performance describes what the automation does according to past and present operation; process describes how the automation operates in terms of its programmed algorithm; purpose describes why the automation was developed, representing the automation designer’s goal which is not always directly observable (Chancey et al., 2017; Lee & Moray, 1992; Lee & See, 2004). Thus, the three dimensions of trust may have different trajectories based on information that operators receive about the automated systems.

Recently, we conducted a flight simulator study that examined which of the three trust dimensions in Lee and Moray’s (1992) framework are most affected by multitasking load for naïve participants (Karpinsky et al., 2018). In Karpinsky et al. (2018), we asked 40 undergraduate participants to perform two portions of the Multi-Attribute Task Battery-II (MATB-II; Santiago-Espada et al., 2011), a manual tracking task and a system-monitoring task assisted by an imperfect automated aid. The tracking task required participants to track a moving target representing the correct directional path of the aircraft. The system-monitoring task required them to ensure that moving pointers do not go outside of the predefined target levels. An imperfect automated aid with reliability of 70% was active throughout the interaction to support the system-monitoring task, which either correctly (hit) or incorrectly (false alarm) identified engine malfunction events. The results indicated that subjective trust toward the signaling system was reliably lower when the tracking task demanded more attention even though the reliability of the signaling system remained identical. More critically, this effect emerged only for the performance and process dimensions of automation trust but not for the purpose dimension.

The results highlight two points. First, naïve users under high workload scenarios may misperceive behaviors of an automated system due to suboptimal information sampling revealed by their eye movements. Second, naïve users may initially base their trust on their perception of performance of the automated system and further develop trust based on the other two dimensions as they gain more experiences with the system. Within the theoretical frameworks offered in the literature, we have started characterizing the trajectories of human–automation trust development for naïve users.

Additionally, theories of automation trust assume that trust development depends on the perception of risk. Risk can be defined as “the extent to which there is uncertainty about whether potentially significant and/or disappointing outcomes of decisions will be realized” (Sitkin & Pablo, 1992, p. 10). As with other preceding works (Chancey et al., 2017; Lyons et al., 2011; Mayer et al., 1995), our research also showed that increased perceived risk elevates automation trust, especially on the performance dimension (Sato et al., 2020). Sato et al. (2020) asked 40 nonexpert undergraduate participants to concurrently perform the tracking task and the system-monitoring task with the assistance of an imperfect automated system. In this experiment, half the participants were assigned to the high-risk condition and were told that they would have to repeat the experiment if their performance was less than the target level. The other half was assigned to the low-risk condition and did not receive these instructions. The perceived risk questionnaire confirmed that the high-risk condition elicited higher levels of perceived risk than the low-risk condition. The results demonstrated that participants’ trust ratings were reliably higher in the high-risk condition than in the low-risk condition when their task load was high, on the performance dimension only. This implies that, within the framework of Lee and Moray’s (1992) model, individuals under a high-risk low-workload condition will be likely to display higher levels of trust than in other conditions.

Human–Automation Interaction Amidst and Following The COVID-19 Crisis

The wide spread of COVID-19 around the globe has prompted people to take preventive measures, such as avoiding close social contact with others. This, in turn, will likely accelerate the development and implementation of automated technologies to support daily and work activities, including telemedicine, teleconferencing, and telework technology. One particular user population of concern is aging adults who may need to use new or new-to-them technologies to support their lives such as telemedicine (e.g., Barretto et al., 2006; Looije et al., 2010) and online shopping (e.g., Koh & Sundar, 2010). The literature on human–automation trust predicts possible patterns of the development of trust in automated technologies for such naïve users.

According to Muir and Moray’s (1996) findings, trust in a reliable automated system may be initially governed by faith for naïve users. Amidst the COVID-19 crisis, individuals may have no other options but to use automated technologies, such as using teleconferencing tools to communicate with their doctor about their health concerns. In highly uncertain situations, such as those created by COVID-19, naïve users would be forced to make a leap of faith to use unfamiliar technologies. Theoretically, users develop faith based on predictability and dependability by evaluating behavioral evidence of interaction with the automated system. However, most individuals amidst the crisis presumably do not have an adequate amount of interaction to sufficiently develop predictability or dependability. Rather than the true capabilities of an automated system, these users may base their trust on information from the automated system’s manufacturers or distributers (e.g., advertisements, hearsay, etc.). Most underlying processes of modern automated systems are so complex that even professional users often do not completely understand (Sheridan & Hennessy, 1984). Therefore, naïve users of modern automated systems are likely susceptible to developing their trust based on faith developed via superficial information about the system.

Naïve users in the COVID-19 era may develop faith in at least two ways. First, users interacting with unfamiliar technologies may employ the analogical process of trust (Lee & See, 2004). This process modulates levels of trust according to external agent and environment characteristics, such as manufacturer or organization characteristics. The analogical process demands less cognitive resources than the analytic process. Bliss et al. (1995) demonstrated that the “hearsay” condition in a cry-wolf alarm paradigm, where participants were told that false alarm rates were higher than the actual false alarm rate, increased response rates in comparison with the control condition. Naïve users amidst the COVID-19 pandemic may rely on the analogical process to establish a leap of faith, based on superficial information from intermediary sources.

Second, users who have not interacted with novel technologies could develop trust toward the technologies because it was created or endorsed by institutions holding similar salient values with the users (Siegrist et al., 2000). The salient values similarity theory (Earle & Cvetkovich, 1995) states that the more similar salient values between the individual and the institution, specifically an individual’s sense of the goals and/or processes that should be followed in a given situation, the more trust the individual places toward technology. Naïve users’ trust in unfamiliar technologies may vary depending on similarity of values between themselves and manufacturers, when sources of information may be limited due to the COVID-19 pandemic.

How do findings from the previous research extend to issues of automation trust and use for naïve automation users following the crisis? Although no concrete predictions can be made at the moment, the theories of human–automation trust offer insights into the issue. First, as COVID-19 recedes and/or is more controlled and uncertainties in their environment decrease, the users who initially trusted the system based on faith may continue to trust if they accumulate behavioral evidence to sufficiently develop dependability and predictability. However, trust that is based purely on faith could also be easily broken if the users do not have adequate perception or knowledge of the system functions. For example, trust toward teleconferencing systems may persist for users who actively monitor behaviors of the systems and accumulate evidence that the system is, in fact, trustworthy. It is unclear, however, how branding and manufacturer information of automation help naïve users establish trust toward the automation as human−automation interaction unfolds over time with or without environmental uncertainties. This pattern may follow the predicted pattern of swift trust, where users form high levels of trust quickly in an ad hoc team and verify this level of trust over time as the number of interactions increase (McKnight et al., 1998; Meyerson et al., 1996). It is possible that naïve users set their initial trust levels based on their first few interactions with an automated system and continue to trust at the initial level, because the initial information has a strong influence on their motive that human–automation relationship will continue to be “good” and “decent” in the future (Rempel et al., 1985). Our lab is currently investigating this possibility using the flight simulator environment.

Second, our research indicates that users’ workload level is critical to the development of trust. When participants reported higher levels of workload, they displayed lower performance-based trust toward an imperfect automated system (Karpinsky et al., 2018). Lee and Moray’s (1992) model predicts that with more interactions with an automated system, users will have more opportunity to develop not only performance-based but also process- and purpose-based trust in automation. It is an empirical question whether the suppression effect of workload on automation trust emerges on the process and purpose dimensions and how this potentially adverse effect can be mitigated. The literature has identified factors that affect mental workload, including exogenous factors such as multitasking (e.g., Kahneman, 1973; Moray, 1967; Wickens, 2008) and task difficulty (e.g., Karpinsky et al., 2018), endogenous factors such as age-related decline in processing capacity (e.g., Brown & Park, 2003), and individual factors such as expertise and knowledge (see Vidulich & Tsang, 2015; Wierwille & Eggemeier, 1993). The COVID-19 crisis can influence these factors which can directly impact the development of trust toward unfamiliar technologies. This issue can arise more severely for aging adults (Ho et al., 2005) who would be at greater risk of misperceiving capabilities of automated systems and ultimately automation mis- or distrust (e.g., Karpinsky et al., 2018; Parasuraman & Riley, 1997). Some evidence suggests that older adults are more likely to trust automation than young adults (Donmez et al., 2006; Ho et al., 2005; Schaefer et al., 2016) and use technologies as frequently as young adults when the technologies are perceived beneficial in some domains (Mitzner et al., 2010; Olson et al., 2011; Rogers & Fisk, 2010). It remains unclear how age-related decline in processing capacity and increase in knowledge interact with development of trust toward automated technologies.

Future Directions and Conclusion

The COVID-19 crisis has significantly disturbed our daily personal and professional activities. More individuals than ever before are being encouraged or required to use automated technologies. This crisis will likely accelerate the implementation of advanced automated technologies to a wider variety of activities for a broader user population.

For researchers, a critical future research consideration is how naïve users initially construct their trust toward unfamiliar technologies and maintain, or fail to maintain, adequate levels of trust over time while the situation improves and risk decreases. The two theories above suggest that active monitoring and analysis of automated system behaviors are critical for forming appropriate levels of trust on the predictability dimension of Muir and Moray’s (1996) model and the performance and process dimensions of Lee and Moray’s (1992) model. Future research should focus on how naïve users’ perception of automation manufacturers and organizations influence their trust toward and interactions with automated system. Additionally, what factors influence either persistence or discontinuance of automation trust that bases on faith over time should be clarified. Lastly, further research should aim to reveal relationships between automation trust and use (e.g., Chancey et al., 2017) as environmental uncertainty changes such as the COVID-19. Currently, we are conducting a longitudinal study that explores how naïve users’ trust toward technologies that they have recently adopted evolves across different time points as the COVID-19 unfolds.

For practitioners, accurate presentation of information about system capabilities, made available by technology manufacturers and distributers, is critical for naïve users to initiate and maintain their trust toward automated technologies. After “forced” adoption of such technology amidst the COVID-19, human–automation trust research suggests that they may discontinue to trust the technology if they do not understand how it works and why it operates in the way they observe. Additional transparency on functionalities of novel technologies adopted during the COVID-19 crisis may facilitate the development of appropriate human–automation trust. As the pandemic becomes more controlled, manufacturers and distributers should focus on communicating system information effectively to retain users.

Key Points