Does distrust in humans predict greater trust in AI? Role of individual differences in user responses to content moderation

Abstract

When evaluating automated systems, some users apply the “positive machine heuristic” (i.e. machines are more accurate and precise than humans), whereas others apply the “negative machine heuristic” (i.e. machines lack the ability to make nuanced subjective judgments), but we do not know much about the characteristics that predict whether a user would apply the positive or negative machine heuristic. We conducted a study in the context of content moderation and discovered that individual differences relating to trust in humans, fear of artificial intelligence (AI), power usage, and political ideology can predict whether a user will invoke the positive or negative machine heuristic. For example, users who distrust other humans tend to be more positive toward machines. Our findings advance theoretical understanding of user responses to AI systems for content moderation and hold practical implications for the design of interfaces to appeal to users who are differentially predisposed toward trusting machines over humans.

Keywords

Content moderation human–AI interaction individual differences machine heuristic

Introduction

The use of artificial intelligence (AI) for content moderation has met with considerable pushback from users, mainly stemming from different perceptions about AI systems, their functioning, and their role in the curation process.¹ As Sundar (2020) explains, “merely identifying AI as the locus of user interactions can serve as a cue for triggering a variety of heuristics based on stereotypes about the operation of machines” (p. 79). Heuristics are “cognitive rules of thumb” that are employed by human beings to make everyday decisions (Sherman and Corty, 1984). In the context of AI, users apply different heuristics, or mental shortcuts, that influence their perceptions of AI (Sundar, 2020). On the one hand, users hold the perception that AI is more accurate and objective compared to humans—this heuristic is known as the “positive machine heuristic” (Molina and Sundar, 2022). On the other hand, users hold the perception that AI is rigid and lacks the nuances of human subjective judgment—this heuristic is referred to as the “negative machine heuristic.” Depending on the heuristic that users invoke about AI, they will evaluate AI technologies differently (either more positively or more negatively) (Molina and Sundar, 2022). The invocation of the positive or negative machine heuristic may depend on the context of interaction, as well as on individual differences such as the user’s previous knowledge of technology, their dispositional trust, and political ideology. Knowing which heuristics are invoked by which kinds of users could have important implications for the design of interfaces of content moderation systems, as it could shed light on how to frame the role of the machine on the interfaces of AI systems. In the next section, we expand on the different perceptions that users hold about AI in general, with an emphasis on the specific context of content moderation.

Perceptions of AI versus human

User acceptance of AI is context-dependent. When it performs mechanistic tasks, such as counting and scheduling, its decisions are perceived as more reliable than those of humans (Lee, 2018). But when it performs subjective tasks, such as hiring, evaluating work performance, or content moderation (the context of the current study), perceptions of AI are not as straightforward. On the one hand, there is a lot of pushback on the use of AI in domains such as moderation of user-generated content. This stems from doubts about the accuracy of the system, especially given that it might result in taking down legitimate content shared by users. By eliminating content, the system is taking away the control and freedom afforded by new media technologies that users have come to expect, particularly on social media platforms. Therefore, content moderation can trigger reactance, especially if content is taken down or the user is blocked as a consequence (Jhaver et al., 2018; West, 2018). Users fear that moderation performed by AI may lead to a high volume of false positives because “the meaning of language is highly context-sensitive and constantly in flux; a word could radically change its meaning if used at different places over time” (Gollatz et al., 2018: 7). The complexity of the human language makes achieving a high accuracy score difficult for AI, which has to be trained utilizing clearly defined contexts. Feature engineering approaches also necessitate that we provide the machine with potential features or useful signals. These features must be at a very granular level to be implemented for machine detection, which can be challenging in the context of content moderation (Molina et al., 2021).² When users think that AI is unable to detect subtleties of human language, they tend to perceive AI as unfit to classify user-generated content, because incorrect classification can lead to content being taken down, an issue many believe to be a violation of freedom of their speech, tantamount to censorship (Gollatz et al., 2018; West, 2018).

However, research exploring classification of user-generated content performed by humans reveals similar issues. In West’s (2018) study, participants viewed content moderation as a form of censorship, regardless of the source of classification (AI or human). Similarly, Jhaver et al.’s (2018) analysis revealed that one of the biggest challenges of human classification is that moderators have “different viewpoints and tolerance level, and what might offend one person may be perfectly reasonable to another” (p. 22). In other words, although humans do indeed have the capacity to contextualize and empathize with a post (Gollatz et al., 2018), users acknowledge that humans have their own biases and experiences that can also influence their decision-making (Jhaver et al., 2018). In fact, in one set of studies, participants expressed more acceptance of AI compared with humans for content moderation because decisions made by AI are at least based on the same constant rules being applied (Binns et al., 2018; Jhaver et al., 2018), and, thus, are “statistically fair” (Binns et al., 2018: 9). For example, Wang (2021) and Gonçalves et al. (2021) found that uncivil and hate speech comments moderated by a machine were perceived as less biased and more transparent than those moderated by human moderators.

Our research team investigated these mixed findings further (Molina and Sundar, 2022) and attribute them to the different heuristics that users hold of AI. We found that user trust in AI, humans, or a combination of the two for performing content moderation depends on the heuristics they invoke when they are informed about the source responsible for classifying the content. In the next section, we explain cognitive heuristics in general, and those specific to AI in the content moderation context.

Cognitive heuristics and evaluations of AI

Cognitive theories suggest that the human brain has limited capacity when processing information (Sherman and Corty, 1984). When making decisions, we often rely on simple decision rules, known as cognitive heuristics (Chaiken, 1980). For example, source credibility studies have long revealed that characteristics of the speaker influence source evaluation. Behling and Williams (1991) found that models were perceived as more intelligent when wearing formal clothing than when dressed more casually (Hood look). This example represents the operation of the heuristic: “if the speaker is dressed more formally, then he/she must be more intelligent.” In this way, there are several “if-then” statements that help us evaluate our daily interactions, including those that occur online. Online heuristics are formalized by the Modality–Agency–Interactivity–Navigability (MAIN) model (Sundar, 2008), which explains that affordances of technology can act as cues in the interaction context, triggering a variety of heuristics that, in turn, influence users’ credibility evaluations. A prominent heuristic identified by this model is the machine heuristic, which has been widely applied to the AI context and refers to the perception that machines are more objective compared with humans, and thus more credible (e.g. Wang et al., 2020). In the content moderation domain, the machine heuristic translates to: “if the classification was done by AI, then it is objective and accurate.” Nonetheless, users can have varying (and often opposing) perceptions about AI. Lee (2018) found that when AI performs mechanistic tasks, it is indeed perceived as more objective and accurate (positive machine heuristic). However, when the AI performs tasks characterized as being more “human,” such as moderating content, then it is perceived as incapable of subjective judgment, and thus, less trustworthy (negative machine heuristic).

Given the various heuristics that can be invoked by the presence of AI as the decision-maker, we (Molina and Sundar, 2022) asked whether source transparency (by letting the user know that an AI or human performed the classification) would invoke the positive or negative machine heuristic, in a content moderation task. We found that both positive and negative machine heuristics can be invoked by users. When the former is triggered, the user tends to trust an AI content classification system more than a human-based moderation system. However, when the latter is triggered, the opposite occurs, that is, the user tends to trust human more than AI.

This raises an important question: How can we predict whether the positive or the negative machine heuristic will be triggered?

Cognitive heuristics are developed through past experiences and observations (Chaiken, 1980). While several heuristics are based on one’s understanding of sociocultural norms (e.g. if the speaker is dressed more formally, then she or he must be more intelligent), other heuristics (such as the machine heuristic) might depend on many factors, such as context and individual differences among users. As Sherman and Corty (1984) explain, the development of heuristics depends on several factors: the person should be able to abstract relevant dimensions from the problem or event, the person should be aware of the relevance of such a dimension, and the person should have experience with the problem or event to create a rule. Thus, “different heuristics may be applied to the exact same problem or to the same problem type as the time and context change” (Sherman and Corty, 1984: 195). In this article, we explore four individual differences as potential predictors of positive and negative machine heuristics in the content moderation domain. We classify them into two groups: (1) psychological and ideological characteristics and (2) technological characteristics.

Psychological and ideological characteristics

Two important characteristics in the content moderation domain are dispositional trust and political ideology. Dispositional trust is defined as “a general belief in the honest and cooperative intentions of others” (Van Lange et al., 1998: 797). Research suggests that individual differences in dispositional trust influence fairness and credibility perceptions. For example, Bianchi and Brockner (2012) found that people with a higher dispositional trust perceived higher fairness within their organization, which, in turn, increased job satisfaction and organizational commitment. Similarly, in the online domain, Zhang et al. (2019) found that dispositional trust increased message credibility in an eWOM (electronic word-of-mouth) environment.The effects were mediated by the structural assurance of eWOM and eWOM skepticism. In the context of AI, dispositional trust has been theorized as an important predictor of trust in automation (Lee and See, 2004). Studies have focused on testing users’ general tendency to trust automation (or dispositional trust toward AI) (Hoff and Bashir, 2015). The propensity to trust other humans, however, could be a relevant factor that, to our knowledge, has not been studied yet. This factor could be especially relevant in sensitive context domains such as content moderation where users with low dispositional trust might not trust other humans in making an unbiased and correct decision regarding the classification of user-generated content. If this is the case, then users with low dispositional trust will be more accepting of moderation performed by AI (compared with humans) because of their overall distrust of other humans. However, it is also possible that users with low dispositional trust are as hesitant of AI as they are of humans. This is especially true in the content moderation domain because content moderation resembles censorship (West, 2018), which can negatively influence users’ perceptions of a system, regardless of the source of moderation. According to Zhang et al. (2019) “when the communication context is not very trustworthy, instead of following their natural trust tendency, people may choose to rely their judgment more on the impressions of the reliability of the context” (p. 57). As such, users may believe that neither AI nor human should be entrusted with content moderation. Thus, we pose the following research question:

RQ1. What is the relationship between dispositional trust and the invocation of (a) positive and (b) negative machine heuristics, in the context of content moderation?

Yet another user characteristic relevant to content moderation is users’ political orientation. On the one hand, trust in science has declined for political conservatives (Gauchat, 2012), who are less likely to adopt and invest in new technologies such as autonomous vehicles (Mack et al., 2021) and energy-efficient technology (Gromet et al., 2013). By extension, it is possible that political conservatives will be less accepting of AI for content moderation and thus will be more likely to invoke the negative machine heuristic, or the belief that AI is not capable of nuanced subjective judgment. On the other hand, content moderation has, perhaps rightfully, raised concerns over its consequences for freedom of expression (see Llansó et al., 2020, for discussion). Many argue that this sentiment is stronger among those holding more conservative political ideology who claim that companies like Twitter, Facebook, and Google are biased against conservative speech and moderate right-leaning content unfairly (Samples, 2019). This is consistent with years of research on the hostile-media perception suggesting that users who identify as conservatives (Republicans) tend to perceive higher media bias against their viewpoint (Eveland and Shah, 2003). However, research suggests that AI moderation may actually reduce perceived bias. Waddell (2019) found that an AI-written story was perceived as less biased than a human-written story. It is possible that conservatives would prefer AI for content moderation because they believe that humans would likely be ideologically biased, and more so if they are liberal media persons (Lee, 2005), whereas AI would be a more objective entity. In this case, the positive machine heuristic would more likely be invoked. Due to conflicting possibilities and lack of dispositive empirical evidence, we pose the following research question:

RQ2. What is the relationship between political ideology and the invocation of (a) positive and (b) negative machine heuristics in the content moderation domain?

Prior experience with technology

As mentioned earlier, cognitive heuristics are developed from prior experiences and observations (Chaiken, 1980). Therefore, an important predictor of whether a user will invoke a positive or negative heuristic about AI is their perceptions and experiences with technology in general and AI technologies in particular. For example, research reveals that “power users” (or individuals who feel comfortable using information technology in general and use it efficaciously and innovatively) differ in several ways from non-power users. Sundar and Marathe (2010) found that in low privacy situations, power users preferred user-initiated customization over system-initiated personalization, while non-power users showed a preference for system personalization. Given power users’ disdain for system tailoring, it is possible that in the content moderation context, they will invoke the negative machine heuristic because automation would be seen as taking away the human agency expected from social media platforms (Sundar, 2020). In addition, power users might be more aware of the technical difficulties that underlie content classification, especially that which requires high contextual knowledge such as content moderation of user-generated content. As such, they will be more likely to invoke the negative machine heuristic. However, it is possible that power users will invoke the positive machine heuristic because they might be less wary of AI technologies for content moderation and more aware of its benefits (i.e. speed and superiority in classifying large amounts of content). A recent study demonstrated that users with more understanding of AI report more positive attitudes toward AI (Weitz et al., 2021). Thus, we pose the following research question:

RQ3. What is the relationship between power use and the invocation of (a) positive and (b) negative machine heuristics in the content moderation domain?

The invocation of the positive or negative machine heuristic, however, is dependent on users’ perceptions of technology in general (power use) and AI in particular. The advancement of AI technologies has brought mixed feelings about the use of these technologies (Oh et al., 2017). On the one hand, there is a sense of excitement about the future due to the potential of AI surpassing human intelligence and its benefits in several domains. On the other hand, there is a sense of fear of these technologies. In fact, one out of four individuals in the United States has expressed fear of AI (Liang and Lee, 2017). Several factors predict fear of AI, with exposure to science fiction predicting it beyond demographic characteristics (Liang and Lee, 2017). This suggests that fear of AI is largely driven by vicarious experiences with AI, in turn, influencing users’ subsequent interaction with AI-based technologies. In the current media environment, societal challenges, including misinformation and hate speech online, have been attributed to AI and many users have become wary of AI for content moderation (Wojcieszak et al., 2021). Thus, it is likely that people with higher fear of AI will be more likely to invoke the negative machine heuristic (compared with the positive machine heuristic). Indeed, research supports that fear of AI leads to a lack of adoption of AI (e.g. Ochmann et al., 2020). We extend this finding and analyze if fear of AI is more likely to invoke the negative machine heuristic than the positive machine heuristic. Thus, we hypothesize the following:

H1. In the content moderation domain, fear of AI will be (a) a positive predictor of the negative machine heuristic and (b) a negative predictor of the positive machine heuristic.

Role of the machine heuristic in user trust toward AI

The section above described and hypothesized how individual differences can guide the invocation of the positive or the negative machine heuristic. According to the MAIN model (Sundar, 2008), the invocation of the heuristic should guide users’ evaluation of the content moderation system. We found evidence of these effects in our study (Molina and Sundar, 2022) such that users trusted an AI content moderation system just as much as they trusted human moderation. However, it depended on whether the user invoked the negative or the positive machine heuristic. Those who invoked the positive machine heuristic trusted the AI system more (compared with the human system) and those who invoked the negative machine heuristic trusted the human more (compared with the AI system). Thus, we reason that if a particular individual difference predicts the invocation of the positive or negative machine heuristic, then the invoked heuristic should, in turn, influence trust toward the system. We test this empirically in our research and ask if the invocation of the machine heuristic (negative or positive) in the content moderation domain is contingent on users’ individual differences (dispositional trust, political orientation, power usage, and fear of AI), and if that invocation translates into trust toward the AI system. In other words, we test if there is a relationship between users’ individual differences and trust toward the system because of the operation of the machine heuristic. More formally, we ask the following research question:

RQ4. What is the mediating role of the positive and negative machine heuristic on the effect of (a) dispositional trust, (b) political orientation, (c) power usage, and (d) fear of AI on trust in a content moderation system?

Method

This article reports one part of a larger between-subjects experiment exploring the effects of source of content classification (AI, human, combination), type of transparency (no transparency, transparency-only, interactive transparency), and classification decision (flagged vs. not flagged) on trust toward AI systems (Molina and Sundar, 2022). In this article, we focus on the role of individual differences in invoking the positive and the negative machine heuristics, and their effects on trust toward the system. However, we include the experimental factors reported elsewhere (Molina and Sundar, 2022) as control variables in all the analyses reported in this article.

Participants

For this study, we recruited 750 participants from the United States using Amazon Mechanical Turk (M-Turk). The final sample consisted of 676 participants after deleting those who provided incomplete responses or did not pass one of three attention checks. Participants in our study mostly self-identified as Caucasian (78.4%) and were highly educated (60.8% reported a bachelor’s degree or higher). In addition, 52.4% were male, 46.6% female, and 0.9% marked other or preferred not to report.

Procedure

Upon consenting to be part of the study, participants were told that we are studying users’ perceptions of a content classification system under development, and we ask that they interact with it and answer questions about their thoughts and feelings afterward. Participants were randomly assigned to one of 18 conditions varying in the source of moderation, type of transparency, and classification decision (Molina and Sundar, 2022). Upon accessing the site, participants were provided information about the classification system, including the source of moderation (AI, human, or both), as well as a definition of hate speech and suicidal ideation (depending on the context to which they were assigned). This definition was provided to highlight the difficulty in determining whether a post is or is not hate speech/suicidal. Immediately thereafter, participants were presented with a post said to be from another social media user (participants were randomly assigned to see one of four posts. The posts and the method employed for selection are detailed in Appendix A of the Online Supplemental materials). After reading the post, participants were told that the content of the post was either flagged or not flagged as hate speech/suicidal ideation by the AI/human/both. The interaction ended at this point for participants in the no-transparency condition. Participants in the transparency-only and interactive transparency conditions were then provided a list of words that were used to determine the classification. Those in the interactive transparency condition were additionally allowed to interact with the list of words by suggesting words for inclusion or exclusion. Details about the experimental conditions can be found in (Molina and Sundar (2022). After completing the interaction, participants were redirected to a questionnaire asking about their trust toward AI, the invoked heuristics, and other questions of interest detailed in the measures section. The effects of the experimental conditions on trust toward the system are reported in (Molina and Sundar, 2022). In this article, we analyze the role of individual differences in invoking the positive and negative machine heuristics.

Measures

This section describes the measured variables utilized in this study. All questions were asked on a 7-point scale.

Machine heuristic

The machine heuristic was assessed through two dimensions, as conceptualized by Sundar (2020). Invocation of positive stereotypes of machines, known as the “positive machine heuristic,” was assessed through four items that asked participants to rate their agreement with the following descriptors of the classification system: has machine-like precision, is error-free, has machine-like accuracy, and has machine-like objectivity (M = 4.37, SD = 1.31, α = .79). Invocation of negative stereotypes of machines, referred to as the “negative machine heuristic,” was assessed through the following seven descriptors of the classification system: is able to detect human emotion (RC), has human-like subjective judgments (RC), is able to provide contextual background (RC), has human intuition (RC), is unyielding, rigid, and mechanistic (M = 4.19, SD = 1.00, α = .66). A confirmatory factor analysis assessing whether the positive and negative machine heuristic form part of a unidimensional model or are part of a two-factor model confirmed that they represent two distinct factors, χ² difference = 270, p < .001.

Power use

To assess power use, or participants’ perceived expertise with information technology, participants were asked 12 items from Sundar and Marathe (2010). Items included questions such as “I would feel lost without information technology” and “using any technological device comes easy to me” (M = 5.21, SD = 0.95, α = .84).

Fear of AI

Fear of AI was measured through an adapted scale from Sundar et al. (2016). Items include the following: “I don’t know why, but artificial intelligence scares me” and “I would hate the idea of artificial intelligence making judgments about things” (M = 3.59, SD = 1.07, α = .81).

Dispositional trust

Dispositional trust was assessed by five items from Van Lange et al. (1998), including “nowadays you have to be careful, otherwise people will exploit you (RC)” and “one should not trust other people, unless one knows them well (RC)” (M = 3.23, SD = 1.23, α = .81).

Political orientation

To measure political orientation, participants were asked four questions proposed by Janoff-Bulman et al. (2008). Items included: “where would you place yourself on a scale from 1 (very liberal/ strong democrat) to 7 (very conservative/strong republican),” and other two items asking participants to rate on a 1 (dislike extremely) to 7 (like extremely) scale, how much they tend to dislike or like political conservatives or political liberals (RC) (M = 3.61, SD = 1.52, α = .84). Higher values represent a more conservative user.

Trust in the classification system

Trust was conceptualized as two dimensions: attitudinal trust (referred to simply as trust throughout the article) and behavioral trust, both measured using a scale adapted from Soh et al. (2009). To measure attitudinal trust (M = 4.53, SD = 1.39, α = .95), participants were asked to rate their level of agreement with a battery of nine adjectives describing the classification system. Items included how honest, valuable, and accurate the system was perceived to be. Behavioral trust refers to user intentions to act upon the system and was measured through three questions asking how willing participants were to: rely on the system to classify user-generated content posted by other users, rely on the system to classify their own posts, and recommend the platform to family and friends (M = 4.01, SD = 1.84, α = .94).

Results

To answer the research questions and hypothesis, we conducted two hierarchical regression analyses, designed to ascertain whether dispositional trust, political orientation, power use, and fear of AI drive the invocation of positive versus the negative machine heuristic. In the first analysis, we entered the positive machine heuristic as the dependent variable and in the second, we entered the negative machine heuristic as the dependent variable. In the first step of the regressions, we entered the four social media posts (used for stimulus sampling) in the form of three 0-1 dummy-coded variables, along with type of transparency, source of classification, and classification decision (the three independent variables in the experiment reported in Molina and Sundar, 2022). As a second step, we entered the four individual-difference variables, which constitute the focus of this article.

Data revealed dispositional trust, political orientation, and fear of AI to be significant predictors of the positive machine heuristic (see Table 1). Power use was not a significant predictor. Importantly, the model’s variance increased from 3.8% to 15.5%, revealing the significant role of participants’ individual differences in predicting the operation of the positive machine heuristic, beyond the experimental conditions of the experiment. When the negative machine heuristic was the dependent variable, power use and fear of AI were significant predictors. Dispositional trust and political orientation were not (see Table 1). Notably again, the variance almost doubled after we entered the four individual-differences variables in the model.

Table 1.

Predictors of the positive and negative machine heuristics.

	Positive machine heuristic β	Negative machine heuristic β
Step 1: controls
Classification decision (1 = not flagged)	.088* [.034, .426]	−.104** [−.355, −.064]
Source (1 = AI)	.096*[.027, .502]	.193*** [.230, .582]
Source (1 = human)	−.071 ^† [−.443, .042]	−.137** [−.474, −.115]
Transparency (1 = interactive)	−.014 [−.284, .202]	.007 [−.165, .195]
Transparency (1 = no transparency)	−.084 ^† [−.465, .007]	−.003 [−.180, .169]
Post (1 = HS1)	−.040 [−.399, .157]	.107* [.040, .451]
Post (1 = HS2)	−.037 [−.393, .168]	.039 [−.118, .297]
Post (1 = SI1)	−.047 [−.421,.137]	.036 [−.125, .288]
R²	.038	.100
Step 2: independent variables
Power use	−.021 [−.143, .085]	.119** [.040, .211]
Fear of AI	−.305*** [−.479, −.272]	.341*** [.244,.398]
Political orientation	.103** [.026, .151]	−.061 ^† [−.087, .006]
Dispositional trust	−.211*** [−.303 −.145]	−.047 [−.097,.021]
Incremental R²	.117	.095
Total R²	.155	.194

Brackets indicate 95% confidence intervals. β represents standardized coefficients. Source, transparency, and stimulus sampling posts are multi-categorical variables and thus were entered as dummy-coded variables.

†

p < .10; *p < .05; **p < .01; ***p < .001.

Finally, to test RQ4, we ran a set of parallel mediation analyses using PROCESS macro’s Model 4 (Hayes, 2018), with a 95% bias-corrected confidence interval based on 5000 bootstrap iterations, to assess whether the positive and the negative machine heuristic were mediators of the relationship between fear of AI, power use, dispositional trust, and political orientation and the dependent variables of interest (trust and behavioral trust). We conducted these analyses to test whether the invocation of the negative and positive machine heuristics by the individual-difference variables translates into trust toward the content moderation system, as predicted by MAIN model (Sundar, 2008). We entered the variables that were not used as independent variables as covariates. For example, when entering fear of AI as the independent variable, power usage, dispositional trust, and political orientation were entered as covariates. We also entered the manipulated independent variables from the experiment (Molina and Sundar, 2022) as covariates.

When we entered dispositional trust as the independent variable, we found a significant mediation for the positive machine heuristic (indirect effect: −0.14, confidence interval [CI] = [−0.19, −0.09]) as evident by the confidence intervals not containing zero, but not for the negative machine heuristic (indirect effect: 0.02, CI = [−0.02, 0.06]). Data patterns (see Figure 1) revealed that users with lower dispositional trust, that is, those who trust others less, are more likely to invoke the positive machine heuristic. The positive machine heuristic, in turn, positively predicts trust toward the system. The same patterns emerged when entering behavioral trust as the dependent variable.

Figure 1.

Mediating effects of positive machine heuristic and negative machine heuristic on the relationship between dispositional trust and trust toward the system.

With political orientation as the independent variable, we found a significant mediation of the positive machine heuristic (indirect effect: 0.05, CI = [0.01, 0.09]), but not the negative machine heuristic (indirect effect: 0.02, CI = [−0.01, 0.05]). Patterns (see Figure 2) show that political conservatives are more likely to invoke the positive machine heuristic, which is positively associated with trust in the system. The same patterns emerged when we entered behavioral trust as the dependent variable.

Figure 2.

Mediating effects of positive machine heuristic and negative machine heuristic on the relationship between political orientation and trust toward the system.

When we entered power usage as the independent variable, we found a significant mediation for negative machine heuristic (indirect effect: −0.08, CI = [−0.15, −0.02]), but not for positive machine heuristic (indirect effect: −0.01, CI = [−0.08, 0.06]). Patterns of the mediation (see Figure 3) indicate that high power users are more likely to invoke the negative machine heuristic. The negative machine heuristic, in turn, reduces trust in the classification system. The same patterns emerge when entering behavioral trust as the dependent variable.

Figure 3.

Mediating effects of positive machine heuristic and negative machine heuristic on the relationship between power use and trust.

When we entered fear of AI as the predictor, we found significant mediation for both positive machine heuristic (indirect effect: −0.22, CI = [−0.29, −0.15]) and negative machine heuristic (indirect effect: −0.20, CI = [−0.26, −0.13]). Patterns reveal (see Figure 4) that fear of AI inhibits the invocation of the positive machine heuristic and facilitates the invocation of the negative machine heuristic, predicting an overall reduction in trust toward the system. The same pattern emerged when entering behavioral trust as the dependent variable.

Figure 4.

Mediating effects of positive machine heuristic and negative machine heuristic on the relationship between fear of AI and trust.

Discussion

We found that users who distrust other humans are more likely to invoke the positive machine heuristic, possibly because they distrust that others will be able to correctly classify content without bias and thus believe the machine is more accurate and objective. Similarly, conservatism was a significant predictor of the positive machine heuristic. As Samples (2019) explains, conservatives hold the belief that big tech companies (i.e. Facebook and Twitter) unfairly classify and censor right conservative speech. Consistent with this notion, conservatives in our study were more likely to invoke the positive machine heuristic because they probably believe machines are more accurate and objective than humans for content moderation.

However, fear of AI was a negative predictor of the positive machine heuristic, but a positive predictor of the negative machine heuristic. When users with high fear of AI interact with a content moderation system, they are more likely to invoke the belief that the AI lacks the ability to make nuanced subjective judgments (negative machine heuristic) and also that the machine is less accurate, precise, and objective (positive machine heuristic). Finally, we found that power use was a positive predictor of the negative machine heuristic possibly because power users are more knowledgeable about the technical difficulties associated with classifying highly subjective content, and thus do not believe that AI technology is advanced enough to tackle them.

Importantly as well, mediation analyses revealed that the invocation of the positive machine heuristic by those with low fear of AI, distrust toward others, and political conservatives translate into higher overall trust toward the system. Conversely, the invocation of the negative machine heuristic by those who fear AI and by power users decreases overall trust toward the system.

By demonstrating that individual differences directly predict cognitive heuristics, our analyses advance theoretical understanding of information processing in the new media environment. The fundamental assumption made by MAIN model (and other theoretical frameworks based on social cognition) is that prior exposure and experience guide invocation of heuristics that are triggered by interface cues. Our findings suggest that prior experience need not necessarily come in the form of prior interactions with the specific medium (or interface). It can appear in the form of general aptitude for and attitudes toward technology (power usage and fear of AI) as well as trait variables such as dispositional trust and political orientation. The fact that such predispositions serve as proxies for prior experience enables us to design better cues (aimed directly at those individual differences) that enhance the probability of invoking specific heuristics and thereby increase the size of their overall effect on trust judgments and other inferences. Furthermore, according to the HAII-TIME model (an application of the Theory of Interactive Media Effects to Human–AI Interaction) (Sundar, 2020), the invocation of cognitive heuristics is contingent on the interface manifestation of AI, the attributes of AI, and users’ prior experiences. Our findings extend the HAII-TIME model by identifying specific predispositions and individual differences as important predictors of the invocation of such heuristics.

There are several practical implications of these findings for user experience research and design of semi-automated classification systems. First, our measures and findings for attitudinal trust and behavioral trust can usefully inform hedonic and utilitarian aspects of user experience respectively, and thereby guide future research on human-centered content moderation systems. More importantly, the key insight from the current study is that personalization of interfaces based on individual differences can alter user experience by appealing to users’ differential beliefs about machine ability over that of humans. For example, in designing interfaces for audiences who tend to distrust other human beings, political conservatives, and users with low fear of AI, designers should accentuate the role of the machine in the classification task because these users are more likely to invoke the positive machine heuristic—or the perception that AI is more accurate and objective compared with humans. Similarly, designers can highlight accuracy and other important metrics of AI performance (e.g. recall and precision) to reinforce the objectivity of the system, keeping in mind ethical design practices.

From a practical point of view, the finding that dispositional distrust and political conservatism are positively related to AI trust is an encouraging sign for the incorporation of AI-based technologies for the detection of harmful content, especially considering the potential that AI has for detecting harmful content at a faster rate than human moderators. Political conservatives who argue that Facebook and Twitter are biased against their ideology, and users who have a tendency to distrust others, would be more accepting of content moderation of harmful content such as hate speech and suicidal ideation, if AI is incorporated as a decision-maker. Nonetheless, it is important to note that AI for content classification is not free from bias either—classification decisions are contingent on the rules and data used for classification (see Llansó et al. (2020) for a detailed discussion). Thus, when building AI technologies with this demographic in mind, designers should also focus on incorporating mechanisms that will increase the systematic processing of information such that users will trust the system not only because of the operation of the positive machine heuristic, but also due to an assessment of the system’s overall strengths and weaknesses. One possibility is incorporating interactivity on the interface, which has been demonstrated to increase analytical thinking online (Oh and Sundar, 2015).

However, our research suggests that power users and users who fear AI are more likely to invoke the negative machine heuristic—or the belief that AI lacks subjective judgment. For these users, including information on the system about the role that humans play in the classification decision might be a good way to reinforce human involvement, especially when classifying content that might not have a clear-cut decision. Explanations about a system should clearly outline the role of AI and of humans in the decision-making process. Explanations should assure the user of the human involvement that exists at different stages of the moderation process, especially when there are higher levels of subjectivity that typically require human attention. Social media platforms typically utilize hybrid systems for content moderation with considerable room for human intervention. On Facebook, for instance, content is first flagged by AI or reported by users, then a human content moderator (or fact-checker in the case of misinformation) makes the final classification decision (Barrett, 2020). However, the process of moderation is not clearly conveyed to users, who are left to wonder how a given classification was determined (West, 2018). A clear reference to the process on the interface might serve to assuage the concerns of power users and those who fear AI. To the extent they understand the respective roles played by AI and humans, they can calibrate their reliance on the system based on individual comfort level.

Our suggestions also have important implications for Explainable AI (XAI) “where the actions can be easily understood and analyzed by humans . . . including all factors and associations related with a given prediction” (Hagras, 2018: 29–30). Research in the area often centers on how to provide explanations such that users are informed about the functioning of the system. Findings of our research reveal that XAI should also focus on accurately conveying the source of classification to users, and prominently highlight human involvement for those who have pre-existing cognitive biases. The existence of such cognitive biases, which we have shown can predict the triggering of machine heuristics, means that oftentimes users will be guided by their individual perceptions of AI when assessing the system’s trustworthiness, instead of on transparency statements designed to provide enough information for the user to understand the system. As such, the user may over-trust or under-trust the system as a function of their belief in certain heuristics about AI, instead of their systematic and analytical evaluation of the system. To address this, designers could incorporate just-in-time alerts based on user metadata to convey the likelihood of falling for a specific heuristic when interacting with different AI technologies, as well as designing resiliency to false persuasion as a preventive measure. For example, alerts or disclosure statements on content moderation systems could explain the strengths and limitations of AI and human moderators. More fundamentally, classification systems should be transparent about the source of moderation (automated, semi-automated, human moderators), clearly conveying how the moderation is conducted and highlighting the pros and cons of the system.

This study also has important limitations to consider. First, we only assessed heuristics of AI in the content moderation context. It is possible that these individual differences operate differently in other contexts. Next, we assessed only four individual differences in this study, centering on psychological, ideological, and technological factors. However, there are other individual differences that could play a role in the invocation of positive and negative machine heuristics that should be explored in future research. For example, users’ motivations and gratifications behind the use of social media as well as other user characteristics, like the big five personality traits, could play a role in whether the negative or positive machine heuristic will be invoked. Furthermore, in our research, we did not account for the system’s actual performance, which could also predict the operation of the positive or negative machine heuristic. A system with good performance could invoke the positive machine heuristic, while a system that does not perform as expected is more likely to invoke the negative machine heuristic. Similarly, the perceived control that users have over the system might influence the invocation of the different heuristics.

Finally, according to the MAIN model (Sundar, 2008), heuristics are triggered by cues in the interaction context. While in the current research, we investigate whether individual differences predict the invocation of the positive or negative machine heuristic, we do not find sufficient evidence to predict whether cues of the interaction context influence the invocation of such heuristics based on users’ individual differences. Additional analyses assessing the interaction effect between the source cue of the experiment (AI, human, or both) and individual differences reveal that users with low fear of AI are more likely to invoke the positive machine heuristic when AI and human provide a classification together (compared to human in isolation) and that for high power users, being told that the AI (vs. human) is the source of moderation invoked the negative machine heuristic. The interactions with dispositional trust and political orientation were not significant. These limitations, however, represent rich areas of future research as we aim to get a better understanding of users’ pre-existing attitudes and perceptions of AI systems and the cognitive heuristics that govern their interaction with these technologies.

Supplemental Material

sj-docx-1-nms-10.1177_14614448221103534 – Supplemental material for Does distrust in humans predict greater trust in AI? Role of individual differences in user responses to content moderation

Supplemental material, sj-docx-1-nms-10.1177_14614448221103534 for Does distrust in humans predict greater trust in AI? Role of individual differences in user responses to content moderation by Maria D. Molina and S. Shyam Sundar in New Media & Society

Footnotes

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iDs

Maria D. Molina

S. Shyam Sundar

Supplemental material

Supplemental material for this article is available online.

Notes

Author biographies

Maria D. Molina (PhD, Penn State University) is an Assistant Professor in the Advertising and PR Department at Michigan State University. Her research investigates the real-life and societal benefits of sharing online, and the role played by technological affordances in the process. Currently, two areas define her research agenda: the role played by technological affordances in promoting sharing of both positive content (e.g. reporting workouts in fitness apps for motivating healthy lifestyle) and negative content (e.g. spreading false news via social media), as well as user responses to automated tools using artificial intelligence (AI) for flagging problematic content such as misinformation and hate speech.

S. Shyam Sundar (PhD, Stanford University) is James P. Jimirro Professor of Media Effects and founding director of the Media Effects Research Laboratory at Pennsylvania State University. He is also Director of the Center for Socially Responsible Artificial Intelligence () at Penn State. His research investigates the role played by technological affordances in shaping user experience of mediated communications, in a variety of interfaces from websites and social media to mobile media and robotics. He edited the first-ever Handbook on the Psychology of Communication Technology (Blackwell Wiley, 2015). He served as editor-in-chief of the Journal of Computer-Mediated Communication, 2013–2017.

References

Barrett

(2020) Who moderates the social media giants? A call to end outsourcing, June. Available at: https://static1.squarespace.com/static/5b6df958f8370af3217d4178/t/5ed9854bf618c710cb55be98/1591313740497/NYU+Content+Moderation+Report_June+8+2020.pdf

Behling

Williams

(1991) Influence of dress on perception of intelligence and expectations of scholastic achievement. Clothing and Textiles Research Journal 9(4): 1–7.

Bianchi

Brockner

(2012) In the eyes of the beholder? The role of dispositional trust in judgments of procedural and interactional fairness. Organizational Behavior and Human Decision Processes 118(1): 46–59.

Binns

Van Kleek

Veale

, et al. (2018) ‘It’s reducing a human being to a percentage’: perceptions of justice in algorithmic decisions. In: Proceedings of the 2018 CHI conference on human factors in computing systems, Montreal, QC, Canada, 21–26 April, pp. 1–14. New York: ACM.

Chaiken

(1980) Heuristic versus systematic information processing and the use of source versus message cues in persuasion. Journal of Personality and Social Psychology 39(5): 752–766.

Eveland

Shah

(2003) The impact of individual and interpersonal factors on perceived news media bias. Political Psychology 24(1): 101–117.

Gauchat

(2012) Politicization of science in the public sphere: a study of public trust in the United States, 1974 to 2010. American Sociological Review 77(2): 167–187.

Gollatz

Beer

Katzenbach

(2018) The turn to artificial intelligence in governing communication online. Report, 14 October. Available at: https://osf.io/preprints/socarxiv/vwpcz/

Gonçalves

Weber

Masullo

, et al. (2021) Common sense or censorship: how algorithmic moderators and message type influence perceptions of online content deletion. New Media & Society. Epub ahead of print 28 July. DOI: 10.1177/14614448211032310.

10.

Gromet

Kunreuther

Larrick

(2013) Political ideology affects energy-efficiency attitudes and choices. Proceedings of the National Academy of Sciences 110(23): 9314–9319.

11.

Hagras

(2018) Toward human-understandable, explainable AI. Computer 51(9): 28–36.

12.

Hayes

(2018) Introduction to mediation, moderation, and conditional process analysis: A regression-based approach. 2nd ed. New York: The Guilford Press.

13.

Hoff

Bashir

(2015) Trust in automation: integrating empirical evidence on factors that influence trust. Human Factors 57(3): 407–434.

14.

Janoff-Bulman

Sheikh

Baldacci

(2008) Mapping moral motives: approach, avoidance, and political orientation. Journal of Experimental Social Psychology 44(4): 1091–1099.

15.

Jhaver

Ghoshal

Bruckman

, et al. (2018) Online harassment and content moderation: the case of blocklists. ACM Transactions on Computer-Human Interaction 25(2): 1–33.

16.

Lee

See

(2004) Trust in automation: designing for appropriate reliance. Human Factors 46(1): 50–80.

17.

Lee

(2018) Understanding perception of algorithmic decisions: fairness, trust, and emotion in response to algorithmic management. Big Data & Society 5(1): 1–16.

18.

Lee

T-T

(2005) The liberal media myth revisited: an examination of factors influencing perceptions of media bias. Journal of Broadcasting & Electronic Media 49(1): 43–64.

19.

Liang

Lee

(2017) Fear of autonomous robots and artificial intelligence: evidence from national representative data with probability sampling. International Journal of Social Robotics 9(3): 379–384.

20.

Llansó

van Hoboken

Leerssen

, et al. (2020) Artificial intelligence, content moderation, and freedom of expression. The Transatlantic Working Group Papers Series, 26 February.Available at: https://www.ivir.nl/publicaties/download/AI-Llanso-Van-Hoboken-Feb-2020.pdf

21.

Mack

Miller

Chang

C-H

, et al. (2021) The politics of new driving technologies: political ideology and autonomous vehicle adoption. Telematics and Informatics 61: 1–18.

22.

Molina

Sundar

(2022) When AI moderates online content: Effects of human collaboration and interactive transparency on user trust. Journal of Computer-Mediated Communication. https://doi.org/10.1093/jcmc/zmac010

23.

Molina

Sundar

, et al. (2021) “Fake news” is not simply false information: a concept explication and taxonomy of online content. American Behavioral Scientist 65(2): 180–212. https://doi.org/10.1177/0002764219878224

24.

Ochmann

Michels

Zilker

, et al. (2020) The influence of algorithm aversion and anthropomorphic agent design on the acceptance of AI-based job recommendations. In: Forty-first international conference on information systems, Hyderabad, India, 13–16 December, pp. 1–17. Atlanta, GA: Association for Information Systems.

25.

Lee

Kim

, et al. (2017) Us vs. them: understanding artificial intelligence technophobia over the Google DeepMind challenge match. In: Proceedings of the 2017 CHI conference on human factors in computing systems, Denver, CO, 6–11 May, pp. 2523–2534. New York: ACM. Available at: https://doi.org/10.1145/3025453.3025539 (accessed 30 August 2021).

26.

Sundar

(2015) How Does interactivity persuade? An experimental test of interactivity on cognitive Absorption, elaboration, and attitudes. Journal of Communication 65(2): 213–236.

27.

Rudin

Radin

(2019) Why are we using black box models in AI when we don’t need to? A lesson from an explainable AI competition. Harvard Data Science Review 1(2): 1–10.

28.

Samples

(2019) Why the government should not regulate content moderation of social media. Available at: https://www.cato.org/policy-analysis/why-government-should-not-regulate-content-moderation-social-media (accessed 30 August 2021).

29.

Sherman

Corty

(1984) Cognitive heuristics. In: Wyer

Jr Srull

(eds) Handbook of Social Cognition, Vol. 1. Mahwah, NJ: Lawrence Erlbaum Associates, pp. 189–286.

30.

Soh

Reid

King

(2009) Measuring trust in advertising: development and validation of the ADTRUST scale. Journal of Advertising 38(2): 83–103.

31.

Sundar

(2008) The MAIN model: a heuristic approach to understanding technology effects on credibility. In Metzger

Flanagin

(eds) Digital Media, Youth, and Credibility. Cambridge, MA: The MIT Press, pp. 72–100.

32.

Sundar

(2020) Rise of machine agency: a framework for studying the psychology of human–AI interaction (HAII). Journal of Computer-Mediated Communication 25: 74–88.

33.

Sundar

Marathe

(2010) Personalization versus customization: the importance of agency, privacy, and power usage. Human Communication Research 36(3): 298–322.

34.

Sundar

Waddell

Jung

(2016) The Hollywood robot syndrome: media effects on older adults’ attitudes toward robots and adoption intentions. In: The eleventh ACM/IEEE international conference on human robot interaction, Christchurch, New Zealand, 7–10 March, pp. 343–350. New York: IEEE.

35.

Van Lange

PAM

van Vugt

Meertens

, et al. (1998) A social dilemma analysis of commuting preferences: the roles of social value orientation and trust. Journal of Applied Social Psychology 28(9): 796–820.

36.

Waddell

(2019) Can an algorithm reduce the perceived bias of news? Testing the effect of machine attribution on news readers’ evaluations of bias, anthropomorphism, and credibility. Journalism & Mass Communication Quarterly 96(1): 82–100.

37.

Wang

Molina

Sundar

(2020) When expert recommendation contradicts peer opinion: relative social influence of valence, group identity and artificial intelligence. Computers in Human Behavior 107: 106278.

38.

Wang

(2021) Moderating uncivil user comments by humans or machines? The effects of moderation agent on perceptions of bias and credibility in news content. Digital Journalism 9(1): 64–83.

39.

Weitz

Schlagowski

André

(2021) Demystifying artificial intelligence for end-users: findings from a participatory machine learning show. In: German conference on artificial intelligence, Virtual Event, 27 September–1 October, pp. 257–270. Cham: Springer.

40.

West

(2018) Censored, suspended, shadowbanned: user interpretations of content moderation on social media platforms. New Media & Society 20(11): 4366–4383.

41.

Wojcieszak

Thakur

Ferreira Gonçalves

, et al. (2021) Can AI enhance people’s support for online moderation and their openness to dissimilar political views? Journal of Computer-Mediated Communication 26(4): 223–243.

42.

Zhang

(2019) The tendency of trust in a distrustful environment: the mediation role of contextual perceptions in eWOM. Journal of Marketing Development & Competitiveness 13(5): 46–64.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.02 MB