Assumptions About Algorithms’ Capacity for Discrimination

Abstract

Although their implementation has inspired optimism in many domains, algorithms can both systematize discrimination and obscure its presence. In seven studies, we test the hypothesis that people instead tend to assume algorithms discriminate less than humans due to beliefs that algorithms tend to be both more accurate and less emotional evaluators. As a result of these assumptions, people are more interested in being evaluated by an algorithm when they anticipate that discrimination against them is possible. We finally investigate the degree to which information about how algorithms train using data sets consisting of human judgments and decisions change people’s increased preferences for algorithms when they themselves anticipate discrimination. Taken together, these studies indicate that algorithms appear less discriminatory than humans, making people (potentially erroneously) more comfortable with their use.

Keywords

automation technology algorithms discrimination

Automation is changing how decisions are made. Increasingly sophisticated algorithms—computerized processes often powered by artificial intelligence and designed to accomplish specific goals—are replacing human experts (Brynjolfsson & McAfee, 2014; Frey & Osborne, 2017), whom they can often cost less than and outperform (Ford, 2015). How do people react when algorithms, as opposed to humans, make important decisions about their own outcomes? In this article, we explore people’s assumptions about algorithms’ capacity to discriminate, and how these assumptions shape their preferences for algorithmic evaluation.

Algorithms and Decision-Making

Algorithms can inform or even make a variety of decisions, both in simple low-stakes environments like text autocorrection or search inquiries as well as in complex and high-stakes environments like predicting whether or not a patient’s symptoms reflect a specific disease or if a parolee is likely to reoffend (Brynjolfsson & McAfee, 2014). Algorithms come in a variety of forms, ranging from simple linear regression models to multilayered machine-learning approaches that mine massive data sets for patterns (e.g., Kosinski et al., 2016; see Meehl, 1954). Using algorithms to inform decisions has some drawbacks; for example, this often requires large amounts of data and extensive computational resources, and algorithms can have difficulty adapting to new or changing circumstances (James et al., 2013). Nevertheless, algorithms can outperform human experts in a variety of domains, leading to their rapid implementation in business, legal, and social contexts (Grove et al., 2000; Meehl, 1954; Stone et al., 2013). This development has raised the question of how algorithms might change the biased and discriminatory outcomes that are sometimes the product of human decision-making.

Human Bias

Human biases occur in many forms, including discrimination (e.g., Dovidio & Gaertner, 2000), or using demographic characteristics in lieu of deservingness to unfairly guide rewards like promotions (e.g., Goldman et al., 2006). In the face of the accumulated evidence documenting both conscious and unconscious discrimination (Kite & Whitley, 2016), the increasing prevalence of algorithms initially inspired a great deal of hope: Perhaps algorithmic agents provide fairer evaluation, sidestepping human biases and ultimately reducing—or even eliminating—discrimination. However, some scholarship has called this intuitive optimism into question.

Algorithmic Bias

On one hand, some scholars have argued that algorithms are less discriminatory than people because they have a superior capacity for processing or detecting patterns from large amounts of information (e.g., Brynjolfsson & McAfee, 2014; Ford, 2015). Unlike humans, who tend to simplfy decisions through the use of heuristics, algorithms can consider a large number of observations of an equally large number of variables to come up with accurate and data-driven predictions (James et al., 2013; Meehl, 1954).

On the other hand, others have argued that algorithms might also preserve—or even amplify—systematic discrimination perpetuated by humans. Many modern algorithms are built such that they likely replicate any human biases embedded in their training data sets (Mann & O’Neil, 2016; O’Neil, 2016; see also Lambrecht & Tucker, 2019). For example, an algorithm trained on a data set in which men received better performance evaluations than women will use gender as a factor in making predictions about how future candidates will perform. If, however, the association in the training data set was due to the managers’ biases—that is, if there is no gender-performance link in objective reality—an algorithm has no way of detecting this pattern, and therefore might blithely replicate this unfair bias. Moreover, it will likely do so invisibly, since many modern algorithms are opaque in terms of the weights they assign different predictor variables (but see Kleinberg et al., 2019).

Documented cases of algorithmic discrimination have already emerged. For example, the algorithm COMPAS, used across the United States to predict whether specific parolees were likely to reoffend, showed a racially biased pattern of errors: Its most common mistake was to miscategorize black defendants as likely to reoffend when ultimately they did not (Larson et al., 2016). Amazon also stopped development of a human resource algorithm after noticing that it penalized resumes that mentioned the word “women.” Even if organizations attempt to withhold demographic information from their algorithms, this information is often contained in proxy variables (like names and zip codes; Caliskan et al., 2017; Kosinski et al., 2016). This has catalyzed nationwide discussion about whether or not organizations should use algorithmic recommendation systems to inform or make decisions about people, given that this decision process can both mask (by keeping discriminatory associations opaque) and scale (a single algorithm like COMPAS can serve many jurisdictions) discrimination (Larson et al., 2016).

Lay Assumptions About Algorithmic Bias

In short, though debates continue over exactly how algorithms compare to humans when it comes to discrimination, it is clear that algorithms can and do discriminate. That said, people’s beliefs about algorithmic discrimination, and the decisions those beliefs guide, may not emerge directly from objective truth (see O’Neil, 2016). Instead, people might be influenced by psychological factors, like their intuitions and stereotypes about machines, guided by the psychology of mind- and person-perception. These factors might lead people to the conclusion that algorithms are relatively unlikely to discriminate.

Assumptions About Accuracy

One reason for this is that people perceive algorithms as exceptionally accurate, or able to detect patterns to determine “real” values of what they are trying to predict (James et al., 2013). People mentally represent machines as relatively agentic agents capable of calculation (Gray et al., 2007). They might also correctly recognize that algorithms are able to parse patterns that would escape a mere person; indeed, this capability is one of the driving forces behind the automation of decision-making (Ford, 2015). Human evaluators, on the contrary, might appear more prone to error, perhaps because people know others’ judgments sometimes rely on shortcuts and extraneous factors, potentially including biased stereotypes. If algorithms appear more accurate than humans, people might assume that their recommendations reflect less discrimination, compared to humans making those same recommendations.

Hypothesis 1 (H1): People perceive algorithms’ recommendations as less discriminatory than identical human recommendations.

Hypothesis 2 (H2): People’s beliefs that algorithms are relatively more accurate agents mediate their perceptions that algorithms are less discriminatory than humans.

Emotional Experience

Another reason why people might see algorithms as relatively nondiscriminatory is that they perceive algorithms as lacking the capacity for emotional experience. While algorithms seem relatively able to “think,” they also seem as particularly low in experience, or the ability to feel emotions (Shank & DeSanti, 2018). Negative emotions, such as anger or disgust toward a specific group, often underlie discrimination (e.g., Dasgupta et al., 2009; Devine, 1989; Tapias et al., 2007). Some people feel negative implicit (e.g., Dasgupta et al., 2000) or explicit (e.g., Jones et al., 2016) emotions toward certain genders or ethnicities and let those emotions guide their judgments and decisions about people they identify with those categories. If people represent algorithms as emotionally neutral agents with no feelings, then they may assume that algorithms do not experience the emotions that drive discrimination in humans.

Hypothesis 3 (H3): People’s beliefs that algorithms are relatively less experiential agents mediate their perceptions that algorithms are less discriminatory than humans.

Anticipated Discrimination and Increased Preferences for Algorithms

Given these lay assumptions, how might individuals feel about being evaluated by an algorithm? At baseline, there are many reasons why people might be averse to algorithms and prefer instead to be evaluated by another human being (see Bigman & Gray, 2018; Castelo et al., 2019; Dietvorst et al., 2015; Newman et al., 2020; but see also Logg et al., 2019). For example, people may have the impression that humans are more likely to notice their unique qualities (Longoni et al., 2019) or that a human evaluator is more likely to hold ultimate decision-making power or other resources, making them a more desirable point of contact.

However, our theorizing thus far implies that people might be more comfortable with algorithmic evaluation when they anticipate discrimination, for example, when they believe they will not be offered a desirable job because of their gender or ethnicity. If algorithms appear less discriminatory than humans, people might be more comfortable with algorithmic evaluation when they themselves anticipate discrimination: in such cases, people may believe an algorithm offers them better odds of success, oblivious to the fact that algorithms could be replicating or even amplifying human biases.

Hypothesis 4 (H4): People prefer algorithmic evaluation relatively more when they anticipate discrimination, compared to when they do not.

How Easily Unlearned Are These Assumptions?

If people assume algorithms are relatively nondiscriminatory, and if these assumptions guide their behavior as recipients of algorithmic evaluation, it is important to know when and why people might update these assumptions. Although people may judge algorithms as relatively agentic and unemotional (Gray et al., 2007), making salient direct, unambiguous information about how algorithms learn from human judgments might give them the opportunity to instead reflect on their potential for replicating human bias. In turn, this might reduce their reliance on heuristic assumptions and therefore their tendency to seek out algorithmic evaluation relatively more when they anticipate discrimination.

Hypothesis 5 (H5): Given information about how algorithms learn from human decisions and can replicate biases, people’s increased preferences for algorithms when they anticipate discrimination will diminish.

Implications and Overview of Studies

These studies carry interdisciplinary theoretical and practical implications. First, if people distinguish between algorithms and humans when it comes to how discriminatory they appear when making otherwise-identical recommendations, people, groups, and organizations may be able to hide discriminatory decisions behind algorithms to appear fair to stakeholders. Second, this research extends literatures on social cognition by showing how people interpret otherwise-identical actions in different ways, depending on how they represent the actor. Third, this research sheds light onto when and why people might be comfortable with automation. Specifically, whereas some recent research finds people are averse to algorithmic processes (e.g., Bigman & Gray, 2018; Dietvorst et al., 2015; but see Logg et al., 2019), we suggest that this aversion might attenuate when people anticipate discrimination.

Seven experiments and surveys tested our hypotheses. Studies 1a to 1c tested H1 by asking people to rate how discriminatory either a human or algorithm’s recommendation was; Study 1c further tested H2 and H3 by measuring perceptions of accuracy and experience. Studies 2 to 4 tested H4 by investigating people’s preferences for algorithms in both naturally occurring and experimentally induced situations where participants anticipated discrimination. Finally, Study 5 tested H5 by examining whether and how easily direct information about how algorithms work attenuated people’s relative preferences for algorithms when they do versus do not anticipate discrimination.

For all studies in this manuscript where we could directly control sample sizes, we followed current norms for experimental research and aimed to achieve 100 participants per cell. To promote generalizability of reported effects, we recruited from a variety of different participant populations (including online samples, university student, and staff samples in-laboratory, and adults from an alumni network). We report all items, measures and manipulations for these studies, and all exclusions (if any). Data and survey materials are available online at https://osf.io/b2ums/?view_only=33948ac47de0496daede65eafd4ede42.

Studies 1a to 1c

Participants reported how discriminatory they believed a particular recommendation was: a human (in one condition) or algorithm (in the other) had recommended an organization interview a candidate with a male name over either a candidate with a female name (Studies 1a and 1c) or another male candidate, but with a characteristically non-Caucasian name (Study 1b). We predicted that participants would assume the algorithm’s decision was less discriminatory, compared to a human’s identical decision (H1). In addition, Study 1c asked participants about the perceived accuracy and capacity for emotional experience of the [person/algorithm]. We predicted that these perceptions would act as independent mediators of how much discrimination participants saw in algorithm’s versus human’s recommendation (H2 and H3).

Participants

Three hundred and two alumni of a private West-Coast university (141 males, M_age = 56.60) completed Study 1a online as part of an open-enrollment alumni research experience program. One hundred and seventeen students and staff at a private West-Coast university (36 males, M_age = 22.51) completed Study 1b in-laboratory for payment as part of a mass testing session. Two hundred and nine American adults (105 males, M_age = 35.34) completed Study 1c online using Amazon’s Mechanical Turk.

Procedure

Studies 1a to 1c all used variations on the same procedure. We first randomly assigned participants to read about a recommendation made either by a human hiring manager or by an algorithm driven by artificial intelligence (AI). Participants read that organizations often employ agents like this one to help screen applicants. We selected the hiring domain because many businesses use algorithms to screen and evaluate prospective employees (e.g., Brynjolfsson & McAfee, 2014; Chalfin et al., 2016). Participants next read a short description of the referral process; condition differences are given in brackets:

In the organization, the [person/AI] works in, current employees can refer a friend, colleague, and so on after which the [person/AI] decides whether or not to offer the referred person an interview. To make a referral, a current employee needs to provide information about the referred candidate, including their personal rankings of the candidate along a variety of dimensions. This is the only information the [person/AI] uses to make a decision.

Participants next read that they would see two ostensibly real referrals. Each participant saw a single pair of referrals, but for the sake of generalizability, each study employed five different pairs manipulated between-subjects (see Figure 1 for an example pair). Aside from the names, Studies 1a to 1c used the same referral materials. Each referral contained quantitative rankings along a variety of dimensions (e.g., being at the 90th percentile on the trait “hardworking”), as well as short qualitative statements recommending the candidate for the open position. Crucially, participants saw that one of the two referrals had a characteristically male and plausibly Caucasian name (“Max Johnson,” “Caleb Brawer,” “Gary Hanson,” “Jacob Buchanan,” or “Todd Wilson”). In Studies 1a and 1c, the other referral had a female name (“Elise Hanson,” “Stephanie Johnson,” “Tracy Wilson,” “Jennifer Brawer,” or “Claire Buchanan,” respectively). In Study 1b, the other referral had a characteristically male, but obviously non-Caucasian, name (“Zhang Wei,” “Luis Rivera,” “Yasir Mahar,” “DeShawn Rogers,” or “Neeraj Chande,” respectively). We generated these names using both existing research (Fryer & Levitt, 2004) as well as searches of census name registries that included ethnicity.

Figure 1.

Sample job candidate referral materials (Studies 1a to 1c).

Each Caucasian male name was always paired with a female (Studies 1a and 1c) or non-Caucasian (Study 1b) counterpart name; the first Caucasian male name in the list above was always paired with the first female or non-Caucasian name in the respective lists, and so on. As noted above, each participant viewed only one pair of referrals, and we counterbalanced whether they saw the Caucasian male referral first or second. We also counterbalanced which referral content was associated with which name (e.g., some participants saw that “Max Johnson” attended Santa Clara University and had the associated rankings, whereas others saw that this candidate attended the University of California: Santa Cruz and had the associated rankings).

After seeing their pair of referrals, participants read that the [person/AI] had recommended in favor of giving the male candidate an interview, but against giving the female (1a and 1c) or non-Caucasian (1b) candidate an interview. They then reported perceptions of discrimination using two items along a 1 (“Strongly Disagree”) to 7 (“Strongly Agree”) scale: “This [person’s/AI’s] decision was discriminatory” and “This [person’s/AI’s] decision was prejudicial” (Study 1b’s items began merely “This decision . . .” and did not further reference the agent). Although prejudice and discrimination can refer to different constructs, these items were correlated above .74 in all studies (all p values < .001), suggesting participants saw them as relatively similar expressions of discriminatory treatment toward particular group. We combined them into a single measure of discrimination; all reported effects were the same when we analyzed items separately, with the sole exception of the “discriminatory” item in Study 1a (p = .085).

Additional Measures

In Study 1b only, participants rated an additional item asking them how likely they thought it was that the agent “received information about the candidates’ ethnicities as part of this application” using a 1 (“Extremely Unlikely”) to 7 (“Extremely Likely”) scale. This item served to ensure that participants did not merely assume that certain information from the referral was hidden from the algorithm.

In Study 1c only, participants rated the agent’s accuracy and capacity for experience. For accuracy, participants used a 1 (“Not at All”) to 5 (“Extremely”) scale to rate the following four items, which all began with “In all likelihood . . .”: “This [person/AI] can determine what aspects of referrals accurately predict job performance,” “This [person/AI] can detect real predictors of performance using referrals,” “This [person/AI] can use the information in referrals to optimally predict job performance,” and “This [person/AI] can identify which aspects of referrals indicate better candidates”; α = .92. For emotional experience, participants rated six items adapted from Bigman and Gray (2018) using the same scale and item stem: “This [person/AI] can experience happiness,” “This [person/AI] can be sensitive to pain,” “This [person/AI] can experience compassion,” “This [person/AI] can experience fear,” “This [person/AI] can experience empathy,” and “This [person/AI] can experience guilt”; α = .98.

Results

Discrimination

In each study, participants indicated that they believed the algorithm’s recommendation was less discriminatory (Study 1a: M = 3.29, SD = 1.32; Study 1b: M = 3.37, SD = 1.38; Study 1c: M = 3.33, SD = 1.76) compared to the identical human recommendation, Study 1a: M = 3.65, SD = 1.36; t(300) = 2.37, p = .019, d = .27, CI₉₅ = [.05, .68]; Study 1b: M = 4.05, SD = 1.39; t(115) = 2.66, p = .009, d = .49, CI₉₅ = [.18, 1.16]; Study 1c: M = 3.92, SD = 1.65: t(207) = 2.50, p = .013, d = .35, CI₉₅ = [.13, 1.04]. Subsequent analyses of covariance (ANCOVAs) controlling for which of the five name pairs participants saw and the two counterbalancing variables (order, and which referral content was paired with which name) yielded similar effects (ps < .03) with no statistically significant effects of the covariates, all F values < 3.62, all p values > .057).

Moreover, in Study 1b, participants believed that humans (M = 4.32, SD = 1.62) and algorithms, M = 4.05, SD = 1.64; t(115) = 0.88, p = .383, d = .17, CI₉₅ = [−.36, .85], were similarly likely to have received information about the candidates’ ethnicities in some way. When we added this measure to the ANCOVA, participants still perceived the algorithm as less discriminatory, F(1, 108) = 6.70, p = .011, $η_{p}^{2} = . 058$ , though they also believed that an agent more likely to have received the ethnicity information made a more discriminatory decision, F(1, 108) = 15.61, p < .001, $η_{p}^{2} = . 13$ .

Mediations through accuracy and emotional experience

In Study 1c, participants assumed that the algorithm was a more accurate agent (M = 3.61, SD = 0.87) and experienced fewer emotions (M = 1.69, SD = 1.17) than the person, accuracy: M = 3.32, SD = 0.92; t(205) = 2.31, p = .022, d = .32, CI₉₅ = [−.53, −.05]; emotion: M = 3.68, SD = .91; t(206) = 13.74, p < .001, d = 1.90, CI₉₅ = [1.74, 2.28]. These effects were robust in ANCOVAs controlling for the various names and counterbalances, accuracy: F(1, 199) = 4.14, p = .024, $η_{p}^{2} = . 03$ ; emotion: F(1, 201) = 188.27, p < .001, $η_{p}^{2} = . 48$ , which themselves did not significantly predict either mediator (Fs < 3.25, p values > .072).

To test H2 and H3, we constructed a 5000-iteration bootstrapped mediation model using agent (0 = person, 1 = algorithm) as the independent variable, perceived discrimination as the dependent variable, and agent accuracy and agent experience as simultaneous mediators (see Figure 2). The algorithm’s higher perceived accuracy (CI₉₅ = [−.46, −.03]) and lower perceived experience (CI₉₅ = [−1.36, −.36]) independently mediated participants’ perceptions that the algorithm’s decision was less discriminatory than human’s decision. When accounting for these two mediators, there remained a marginally significant direct effect of the agent manipulation, but its sign was reversed: After accounting for our hypothesized mechanisms, algorithms appeared if anything more discriminatory than people.¹

Figure 2.

Mediation to perceived discrimination through accuracy and experience (Study 1c).

Discussion

In three different studies, participants viewed an identical decision differently depending on whether it was made by a human or an algorithm. Consistent with H1, they assumed algorithms’ decisions reflected less discrimination that identical human decisions, even when these decisions were based on identical information. This did not happen because participants assumed algorithms had less access to demographic information (Study 1b). Instead, consistent with H2 and H3, the discrimination effect was mediated by participants’ perceptions that algorithms were relatively more accurate and less experiential than humans (Study 1c).

Study 2

Our next four studies investigated one possible consequence of people’s perceptions that algorithms are less discriminatory than humans. Specifically, we examined whether people who anticipate discrimination themselves show a stronger preference for algorithmic evaluation (H4). In these studies, we were interested not in the absolute preference for algorithmic evaluation over human evaluation, but rather in the degree to which anticipating discrimination increased people’s preferences for algorithmic evaluation, compared to their preferences when they did not.

In Study 2, community college students indicated the degree to which they believed they would encounter discrimination when it comes to hiring, as well as their relative preferences for both human and algorithmic evaluators for jobs. We predicted that those who anticipated discrimination to a greater extent would prefer algorithmic evaluation more, compared to those who anticipated discrimination to a lesser extent.

Method

Participants

One hundred and ninety-seven students from two West-Coast community colleges (49 males, M_age = 25.61; 35.8% Asian, 30.9% white, 13.7% Hispanic, 3.9% black or African American, 3.9% Native Hawaiian or Pacific Islander, 0.5% Native American or Alaska Native, 7.4% Other) completed the study online for credit as part of a research experience program.

Procedure

Participants first rated four items about their concern (1 = “Not at All Concerned,” 5 = “Extremely Concerned”) that organizations would discriminate against them when they applied for jobs: “Organizations will discriminate against me,” “Organizations will be prejudiced against me,” “Organizations will be racist and/or sexist toward me,” and “Organizations will be unfair towards me based on my demographics.” We intentionally designed these items to apply to a broad range of personal characteristics on the basis of which people might anticipate discrimination. These items exhibited high reliability (α = .96); we thus aggregated them into an individual-difference measure of anticipated discrimination. Next, participants responded to two items regarding their preferences for different evaluators for their real job applications: “I would want an artificial intelligence to evaluate me” and “I would want a human to evaluate me,” each rated using a 1 (“Strongly Disagree”) to 7 (“Strongly Agree”) scales. These two ratings showed a moderately strong negative correlation, r = −.44, p < .001, suggesting that participants who tended to like one evaluator also tended to dislike the other. We also measured gender, age, and ethnicity.

Results and Discussion

Overall, participants’ anticipated discrimination fell slightly below the scale midpoint (M = 2.61, SD = 1.24; t(195) = −4.44, p < .001). Also, consistent with previous research on algorithm aversion in human resource contexts (Dietvorst et al., 2015; Newman et al., 2020), a paired-samples t-test indicated that participants strongly preferred human evaluation (M = 5.77, SD = 1.22) to algorithmic evaluation (M = 2.99, SD = 1.61; t(195) = −16.20, p < .001, CI₉₅ = [2.45, 3.14]).

More importantly and consistent with H4, a mixed model regression revealed a significant evaluator type by anticipated discrimination interaction, b = −0.37, p = .007. Specifically, the more participants believed they would encounter discrimination when applying for jobs, the more they preferred algorithmic evaluators (b = .19, p = .007) and, conversely, the more (marginally) they wished to avoid human evaluators (b = −.13, p = .082). Moreover, while people showed an overall preference for humans over algorithms, b = 2.78, p < .001, this preference was about 50% stronger among those who anticipated less (i.e., 1 SD below the mean; b = 3.35, p < .001), compared to more (i.e., 1 SD above the mean; b = 2.21, p < .001), discrimination. These findings provide correlational support for H4: People who naturally anticipated discrimination in their lives were more comfortable with algorithmic evaluation. As these data cannot speak to causality; Study 3 experimentally manipulated participants’ anticipation of discrimination.

Study 3

Study 3 used academic affiliation as the basis for potential discrimination. We recruited students at a private university and manipulated whether the evaluation context seemed neutral with respect to discrimination (in the control condition) or seemed like it could be unduly influenced by academic affiliation. Based on H4, we predicted that participants who anticipated they might suffer through discrimination would prefer an algorithmic evaluator to a greater extent, compared to participants in the control condition.

We also included a third condition where the undue influence of academic affiliation could offer participants an advantage, as opposed to a disadvantage. We predicted that participants in this third condition would resemble those in the neutral condition. The reasoning underlying H4 is that people who fear they might suffer as a result of discrimination might seek algorithmic evaluation because they assume—per H1—that machines are less likely to rely on irrelevant demographic characteristics. By this same logic, people who believe that their demographic characteristics could yield an advantage should have no special interest in seeking out an evaluator who might disregard those very characteristics.

Method

Participants

Two hundred and seventeen students and staff at a private West-Coast university (75 males, M_age = 23.71) completed the experiment in-laboratory for payment across two mass testing sessions. We collected data across two sessions (N = 110 and N = 107, respectively) because, even though all predicted discrimination versus control or advantage effects were significant in the first round alone (as well as in the second round alone), our per-cell sample size after the first data collection fell below our minimum requirement.

Procedure

We randomly assigned participants to either a control condition, a discrimination condition, or an advantage condition. Similar to the procedure in Study 3, participants imagined that they were applying for a job that they really wanted. Participants in the control condition then proceeded to our dependent measures. Participants in the discrimination condition first read the following: “You’ve heard that the organization you are applying to has, in the past, discriminated against people. . .” with their university affiliation. Conversely, participants in the advantage condition read that “You’ve heard that the organization you are applying to has, in the past, really loved people. . .” with their university affiliation.

Participants then completed two items similar to those from Study 2: On separate scales, they rated the degree to which they would want “a human (e.g., a hiring manager)” and an “artificial intelligence (e.g., a machine-learning algorithm”) to evaluate them, 1 (“A Very Small Extent”) to 5 (“A Very Large Extent”). As in Study 2, these two ratings showed a strong negative correlation r = −.66, p < .001. These repeated findings help validate an additional measure we included in this study: Participants made a binary choice between a single human and a single algorithm driven by AI to evaluate their job application.

Results

Binary agent choice

Conceptually replicating the results of Study 2, we observed a significant chi-square statistic across the three conditions, χ² (2, N = 216) = 39.03, p < .001, ϕ = .42. While only two participants in the control condition (2/72, 2.8%) and four participants in the advantage condition (4/72, 5.6%) indicated that they would choose an algorithmic evaluator over a human one, 26 participants in the discrimination condition (26/72, 36.1%) indicated that they would choose the algorithm over a human.

Agent preferences measured separately

A 3 (condition: control vs. discrimination vs. advantage between-subjects) × 2 (evaluator: human vs. algorithm; within-subjects) analysis of variance (ANOVA) on participants’ separately rated preferences for each of the two agents revealed no main effect of condition, F(2, 213) = 1.00, p = .369, $η_{p}^{2} = . 01$ , a significant main effect of the within-subjects factor reflecting participants’ overall preference for a human evaluator, F(1, 213) = 218.16, p < .001, $η_{p}^{2} = . 51$ , and, consistent with the chi-square results, a significant interaction, F(2, 213) = 23.75, p < .001, $η_{p}^{2} = . 18$ (see Figure 3).

Figure 3.

Evaluation preference as a function of condition (Study 3). Error bars represent 95% confidence intervals of means.

Concerning participants’ preferences for an algorithmic evaluator, simple effects analyses indicated that participants preferred this to a greater extent when they anticipated suffering due to discrimination (M = 2.74, SD = 1.34) compared to the control, M = 1.97, SD = 0.99; F(2, 213) = 12.69, p < .001, and advantage, M = 1.89, SD = 0.98; F(2, 213) = 15.75, p < .001, conditions, which did not differ from each other, F(2, 213) = 0.15, p = .861. Conversely, participants in the disadvantage condition wanted a human evaluator to a lesser extent (M = 3.36, SD = 1.23) compared to participants in both the control, M = 4.31, SD = 0.66; F(2, 213) = 19.51, p < .001, and advantage, M = 4.40, SD = 0.72; F(2, 213) = 23.64, p < .001, conditions, which again did not differ from each other, F(2, 213) = 0.19, p = .827.

Discussion

Study 3 provided experimental support for H4: when participants anticipated suffering as a result of discrimination, they showed relatively stronger preferences for algorithmic evaluation compared to when they did not anticipate such suffering. This was reflected both in their Likert-type scale ratings of the two potential evaluators, and in their binary choice between them.

Study 3 also explored people’s preferences when they expected to receive an advantage due to their academic affiliation. We might have thought these participants would want a human evaluator even more to exploit that advantage; indeed, the fact that we did not may represent a floor effect: Participants in the control condition already did not want an algorithmic evaluation. Future research could examine this by studying sample with a weaker baseline aversion to algorithms, for example, a sample of people who have grown accustomed to algorithmic evaluation.

Study 4

Study 4 tested whether the results we had obtained so far would extend to an evaluation context with real potential stakes. Participants enrolled in a prescreen survey designed to help them qualify for a set of high-paying human intelligence tasks (HITs) via Mechanical Turk. Participants answered questions about anticipated discrimination drawn from Study 2, along with a battery of questions about their conscientiousness, and their MTurk habits and performance, which we afterwards told them would form the basis of their qualification for the high-paying HITs. Participants could choose what weight to assign the ratings they would receive from a human evaluator and an algorithmic evaluator. Consistent with Study 2, we predicted that participants who naturally anticipated discrimination on this particular online market would assign greater weight to the algorithm’s evaluation of their application.

Method

Participants

We requested 200 American adult participants via Amazon’s Mechanical Turk; 198 completed the survey. Of these, we excluded participants who failed either an embedded attention check (n = 16) or an English comprehension check (n = 42, including five who also failed the attention check) at the end of the survey, yielding a final sample of 145 (83 males, M_age = 35.89). We included these checks following recent concerns about participant attention on contract-labor websites such as MTurk, and planned to exclude participants who failed them. Nonetheless, results were very similar when we instead included the full sample; we report both sets of results below.

Procedure

Participants enrolled in a prescreen survey as a qualification for a set of high-paying HITs on MTurk. At the beginning of the survey, we administered a battery of demographic questions: age, race, political orientation, yearly household income, and education, and an attention check: “What day of the week is it? Regardless of what day of the week it currently is, please select the sixth option below” (this option was Saturday; all data were collected on a Friday in all American time zones). Next, we asked participants a series of MTurk-related questions. The bulk of these were questions about their conscientiousness, performance, and habits on MTurk (e.g., how often their HITs are rejected, whether they multitask while completing HITs, how many hits they complete in a typical week, an open-ended question about their conscientiousness, and so on; see supplemental methodology file for full survey text). The final three questions, adapted from Study 2, formed an individual-difference measure of anticipated discrimination (α = .96). Each one began with the stem, “When you apply for tasks on MTurk that you must be selected for, are you concerned that . . .”; they ended thus: “Organizations will discriminate against me, based on my demographics,” “Organizations will be prejudiced against me, based on my demographics,” and “Organizations will be unfair towards me, based on my demographics.” Participants responded to these items along the same 1 (“Not at All Concerned”) to 5 (“Extremely Concerned”) scale used in Study 2, and—as also observed in Study 2—their responses averaged slightly lower than the scale midpoint (M = 2.39, SD = 1.28, t(144) = −5.71, p < .001).

Next, participants learned that their answers to the questions on the previous page would serve as their application to qualify for a second set of HITs that paid $18 per hour a relatively desirable rate on MTurk. They learned that two different evaluators would both evaluate their application: a “person” and an “algorithm driven by artificial intelligence (AI).” They read that both agents would assign them ratings:

. . . the human evaluator will read your application and use that information to estimate, based on their experience with other applicants, how good of a candidate you are for the tasks we are hiring for. The algorithm has created a statistical model based on previous applicants’ data that makes a separate prediction about how good of a candidate you are for the tasks we are hiring for.

They also read that these two scores would be combined to determine their application score and that the candidates with the highest scores would automatically qualify for the relatively higher-paying HITs.

Finally, participants learned they could choose what relative weight they wanted to give to the score that the algorithm gave them:

If you want your final score to be equally determined by how the person and the algorithm rated you, you should pick a weight of 50%. If you want your final score to be determined more by the person’s rating, you should pick a weight smaller than that. If you want your final score to be determined more by the algorithm’s rating, you should pick a weight larger than that.

Participants then selected a weight ranging from 0% (“only use the human’s score”) to 100% (“only use the algorithm’s score”). Finally, at the end of the survey, we included an English comprehension check requiring them to read an article and identify a singer’s musical group membership.

Results and Discussion

Overall, participants who anticipated more discrimination assigned a greater weight to the algorithm’s evaluation (r = .38, p < .001). To examine the robustness of this effect, we regressed the weights participants selected on their anticipated discrimination as well as all of the various demographic and MTurk-related questions we included alongside the discrimination items predicting desired weight. In this model, we again found that participants gave more weight to the algorithm when they anticipated more discrimination (b = 4.99, p = .007); the only other significant predictors were the number of HITs completed in a typical week (b = −4.39, p = .048) and the number of years participants had been using MTurk (b = −4.11, p = .002), all other p values > .13. We also tested whether any of these variables moderated the association between anticipated discrimination and the weights participants assigned; no interactions reached significance (ps > .140), save for an unexpected interaction with religiosity (b = 2.08, p = .002). Simple slopes analyses revealed that among strongly religious participants (1 SD above the mean), the anticipated discrimination effect emerged strongly (b = 10.40, p < .001), but among nonreligious participants (1 SD below the mean), it was eliminated (b = 1.28, p = .583). We had no theoretical way of interpreting this result, which emerged from an exploratory analysis.

Each of these results also emerged when we analyzed the full data set, including participants who failed either or both checks: In these analyses, participants again gave more weight to the algorithm when they anticipated discrimination (r = .46, p < .001), including in a regression model accounting for all of the different demographic and MTurk-related variables we collected as simultaneous predictors (b = 6.26, p < .001).

Study 4 used a context where participants believed their application would be evaluated by both a human and algorithm, and that the outcome held real financial stakes. Even in this context, participants who anticipated discrimination on the basis of their demographics wanted their evaluation by the algorithm to carry more weight than did other participants. Although this study occurred in a potentially idiosyncratic online labor market (MTurk), it offered some evidence that people’s assumptions that algorithms are less likely to discriminate may drive them to actually seek out algorithmic evaluation.

Study 5

Study 5 provides a final test of H4, examining whether people experimentally induced to anticipate discrimination prefer algorithmic evaluation relatively more than others, this time focusing on gender as the basis for discrimination. In addition, it also tested H5: whether people’s situational preference for algorithms would respond to, or resist, explicitly presented information about algorithms’ potential for bias (O’Neil, 2016).

Similar to Study 4, participants indicated what weight they would want to give, respectively, to a hiring algorithm’s evaluation alongside a human’s evaluation of them. Similar to Study 3, we randomly assigned them to do so in a control condition, or in a discrimination condition, after they learned that the organization in question had a history of discriminating against their gender. Unique to Study 5, participants in both conditions indicated their preferred weighting twice: first before and again after reading information about how this algorithm learns to make its recommendations, and how this can potentially result in it propagating human biases. This yielded a 2 (condition: control vs. discrimination; between-subjects) × 2 (time: before vs. after information about algorithms; within-subjects) design. Comparing participants’ preinformation responses in the two conditions provided an additional test of H4: we predicted that people would seek algorithmic evaluation when they anticipated discrimination, compared to when they did not. Observing how their responses change upon learning more about algorithms provided a test of H5: We predicted that this information would diminish participants’ preferences for algorithms in the discrimination condition, compared to the control condition.

Method

Participants

We aimed for N = 100 participants per experimental cell, and given the number of attention and English comprehension failures we observed in Study 4, we requested 250 American adult participants via Amazon’s Mechanical Turk. Of these, 40 failed an English comprehension check included at the end of the survey, yielding a final sample of 210 (103 males, M_age = 37.68).

Procedure

After indicating their age and gender, participants imagined that they had recently noticed a posting for a job that they would really want, and that they were considering applying for it. Participants randomly assigned to the discrimination conditions read additional text warning them of possible discrimination against their own gender in hiring; we tailored the wording according to the gender they had indicated earlier in the survey:

However, one thing that concerns you is that you’ve heard that the organization that posted this job discriminates against [men / women / your gender]. Specifically, news outlets have reported that this organization has discriminated against [men / women / people who identify as your gender] in many past hiring decisions.

Participants next read that the application would consist of “. . . self-rankings about a number of skills, some short online personality and cognitive tests, along with some basic demographic information (name, nationality, gender, age, etc.)” They read further that the organization that posted this job has always used two agents to evaluate applications: “a human hiring manager and a hiring algorithm driven by artificial intelligence (AI).” Both agents would review applications “with no additional information,” and then assign each candidate a rating from 1 (“Very Bad”) to 10 (“Very Good”). The organization combines the human evaluator’s score with the algorithm’s score, historically by weighting them equally, and the candidate “. . . with the highest combined score is offered the job.” To reinforce the primary experimental manipulation, participants in the “discrimination” conditions additionally read: “Please keep the fact that you might face discrimination in mind as you answer the following questions” before continuing.

We provided these details about the organization and its history to ensure that participants in the discrimination condition knew that the organization’s history of mistreating their gender could only be caused by biases in the ratings provided by either the human or the algorithmic evaluator. Moreover, by noting that historically the organization weighted the two ratings equally, we gave participants no objective reason to assume that the source of the discrimination was more likely to be the human or the algorithm.

With all this information in mind, participants learned that, as part of a pilot program, they would be able to choose what relative weight to give to the score the algorithm gave them. We explained this weighting to them in the same way described for Study 4. Participants indicated the weight they would want to give to the algorithm’s rating of them, from 0% (“only use the hiring manager’s score”) to 100% (“only use the algorithm’s score”).

After indicating their desired weight, participants read additional information about the hiring algorithm. We modeled this information on how machine-learning algorithms often work in actual human resource contexts (e.g., Chalfin et al., 2016). Specifically, participants read the following text:

We wanted to tell you a bit more about how this particular hiring algorithm works. Essentially, this algorithm learns to replicate human evaluations. To “teach” it, the organization gave it big datasets where it figured out which applicant characteristics best predicted positive performance evaluations from managers. After figuring out which characteristics were most associated with positive manager evaluations, this algorithm “looks” for those characteristics in new applicants. What this means is that the algorithm can actually learn to replicate human biases, if managers tended to give biased evaluations.

After reading this information, participants read “Now that you know a bit more about how this algorithm works, we would like to ask you again: what weight would you want to give to the algorithm’s evaluation of you?” and again indicated their preferred weight for algorithmic evaluation using the same 0% to 100% scale.

At the end of the study, we posed two multiple-choice comprehension questions, to explore the degree to which participants had processed the extra information we provided on the algorithm’s functioning: how well they grasped that algorithms simply learn to replicate past human decisions, regardless of whether or not these past decisions were biased. We planned to examine whether participants’ reactions to the extra information differed depending on the depth of their understanding. One question dealt with reporting that the algorithm “. . . learns to predict how managers have rated past candidates,” and another dealt with recognizing that the algorithm would recommend a candidate who was relatively unskilled but highly rated by managers (see Supplemental methodological file for full question text and distractor choices).

Results

A 2 (condition: control vs. discrimination; between-subjects) × 2 (time: before vs. after information about how this algorithm works; within-subjects) ANOVA on participants’ desired weight for the algorithm’s evaluation indicated that participants led to anticipate discrimination based on their gender gave a larger weight to the algorithm’s evaluation across both time points, F(1, 208) = 8.33, p = .004, $η_{p}^{2} = . 04$ . Participants also preferred the algorithm significantly less, across conditions, after learning about how it worked, F(1, 208) = 14.91, p < .001, $η_{p}^{2} = . 07$ . Critically, we also observed a significant interaction, F(1, 208) = 20.76, p < .001, $η_{p}^{2} = . 09$ (see Figure 4).

Figure 4.

Weight given to algorithm’s evaluation as a function of condition (preintervention and postintervention). Error bars represent 95% confidence intervals of means.

Simple effects analyses indicated that, consistent with H4, participants initially preferred an algorithm to a greater extent in the discrimination condition (M = 64.86, SD = 28.42) compared to the control condition, M = 49.14, SD = 25.31; F(1, 208) = 17.90, p < .001, $η_{p}^{2} = . 08$ . However, this difference was completely eliminated once they learned more about how the algorithm works, discrimination: M = 53.33, SD = 25.33; control: M = 50.10, SD = 23.84; F(1, 208) = 0.91, p = .341, $η_{p}^{2} = . 004$ . Analyzed differently, while participants in the control condition did not adjust their weight after reading more about the algorithm, F(1, 208) = 0.24, p = .623, $η_{p}^{2} = . 001$ , those in the discrimination condition gave the algorithm significantly less weight in their second rating, F(1, 208) = 208, p < .001, $η_{p}^{2} = . 15$ . The significance thresholds for all above results remained the same when analyzing only participants who responded to the two comprehension check question correctly, as well as in identical models controlling for gender and age, which also did not significantly interact with the discrimination manipulation, Fs(1, 202) < 2.58, p values < .111, $η_{p}^{2} = . 01$ and .003, respectively, suggesting that people responded to the threat of discrimination similarly regardless of their own gender.

As in Study 4, each of these results also emerged when we analyzed the full data set, including participants who failed the English comprehension check: The same repeated-measures ANOVA again indicated a significant interaction between condition and time, F(1, 248) = 20.07, p < .001, $η_{p}^{2} = . 08$ . Simple effects analyses again indicated that participants initially preferred an algorithm to a greater extent in the discrimination condition (M = 64.72, SD = 27.83) compared to the control condition, M = 53.94, SD = 26.13; F(1, 248) = 9.97, p = .002, $η_{p}^{2} = . 04$ , but that this difference was also completely eliminated after they learned how the algorithm works, discrimination: M = 54.47, SD = 25.36; control: M = 54.25, SD = 24.48; F(1, 248) = 0.005, p = .945, $η_{p}^{2} < . 001$ .

Discussion

In Study 5, providing simple information about the mechanics of many machine-learning algorithms in evaluation contexts—training using data sets consisting of past human judgments and decisions—changed participants’ relative preferences for algorithmic evaluation when they anticipated discrimination. Specifically, when we told participants they might experience discrimination as a function of their gender, they assigned an algorithm greater evaluation weight compared to participants who did not receive this explanatory intervention. However, when we delivered an intervention outlining how this algorithm functioned and its potential to replicate human biases, this preference for greater algorithmic evaluation disappeared. Taken together, these findings suggest that people’s assumptions about algorithms can be responsive to new information (see also Dietvorst et al., 2015).

General Discussion

A great deal of recent research has investigated how people respond to algorithms as their usage increases, as well as when people might be (un)comfortable with their use (Castelo et al., 2019; Dietvorst et al., 2015; Logg et al., 2019; Waytz & Norton, 2014). The present studies broadly suggested that, while people might be somewhat averse to algorithms during evaluation, they believe that algorithms are less capable of discrimination than people (Studies 1a–1c) and subsequently seek them out to a greater extent when they anticipate discrimination (Studies 2–5). While many imagine a future where algorithmic evaluation is ubiquitous and free of bias (Kleinberg et al., 2019), many scholars have argued that algorithms—in their current form—can not only perpetuate and scale discrimination, but by virtue of their complexity, often do so in an opaque way (e.g., Larson et al., 2016; O’Neil, 2016). As algorithms proliferate throughout business and society, people’s attributions and assumptions may lead them to unwisely seek algorithmic evaluation in situations where such technologies maintain—or even amplify—discrimination while simultaneously obscuring it.

These studies also carry with them the conclusion that people, groups, organizations, and social systems (e.g., governments and courtrooms) can appear fair, even if they are actually behaving in discriminatory ways, by highlighting their use of technology. If systems choose to hide behind technology to appear fair and unbiased, these results suggest that people will be more comfortable with—and potentially react less to—potentially discriminatory behaviors in situations where they believe machines made decisions or were heavily involved in a process.

Future Directions

One question for future research might be at what point people become willing to spontaneously attribute discrimination to algorithms. For example, in Studies 1a to 1c, participants resisted labeling as discriminatory a single instance whereby an algorithm assigned a better outcome to a Caucasian man than to a woman or non-Caucasian man. If they saw consecutive instances of discrimination or a distribution of binary decisions, at what point would this resistance melt away? We should also note that, by directly asking participants about discrimination, we may have planted in the notion that we wanted participants to condemn the decision they had witnessed. Even so, though, and even if people responded to this notion to please us as experimenters, it remains an intriguing observation that this seems to have happened more strongly when the decision was made by a human compared to an algorithm. If raising the notion of discrimination induced demand only in the context of human decisions, but not in that of algorithmic decision, that is entirely consistent with our basic theorizing suggesting people are simply not likely to consider algorithms as equally capable of discrimination.

Another open question delves deeper into people’s tendency to seek algorithmic evaluation when they fear discrimination. One possibility is that, when people anticipate that a particular organization might discriminate against them, they become averse to humans in that organization evaluating them, preferring instead the neutrality they believe algorithms offer. A different possibility, though, is that when people consider that they might suffer discrimination, they become suspicious of humans more broadly, and would therefore not only avoid evaluation by human members of the organization in question, but even by third-party humans.

In addition, other frameworks offer insight into social cognition and how people may perceive algorithms (e.g., the warmth vs. competence framework; Cuddy et al., 2008). While perceiving the presence of mental states is tightly related to attributions of discrimination, future research investigating perceptions of algorithms will benefit from acknowledging theoretical perspectives above and beyond mind perception. Finally, the present studies focused largely on business organizations and job applications. We chose this context because it is both a context most people experience where both discrimination is often present, and major life outcomes are affected (e.g., potential for future promotions and social mobility). Future work may examine whether people’s attributions regarding algorithmic discrimination differ in other important contexts (e.g., medicine or law; see Longoni et al., 2019) or if jobs are framed differently, as people are more comfortable with machines in “thinking” but not “feeling” jobs (Waytz & Norton, 2014).

Could there be a way of aligning algorithms to people’s intuitions: of developing algorithms which we can be confident will not propagate unfounded human bias? One potential solution could be to avoid providing algorithms with information about people’s demographic characteristics. However, algorithms can often use seemingly unrelated cues as proxies for gender, ethnicity, or family background (Caliskan et al., 2017; Kosinski et al., 2016). Another solution is to ensure that algorithms are trained on data that contain no biases. However, Study 5 described an algorithm trained on human evaluations (a common practice in human resource contexts, e.g., Chalfin et al., 2016), algorithms can train on more “objective” performance data. However, even in these cases, humans still select which objective data to use for training, and those decisions themselves—as well as human decisions about what objective data to even bother collecting—are points in which human bias can alter a decision process. To reduce the discrepancy between people’s intuitions and algorithms’ actual likelihood of discrimination, it might be more efficient to take the route of Study 5, and intervene on people’s intuitions via education and/or information about how such systems function.

Conclusion

Algorithms are rapidly changing how people make decisions. As such, people now face the novel situation of a technological agent, such as an algorithm, making recommendations—or even completely autonomous decisions—that profoundly impact their lives. As societies continue to automate and implement new technologies, the present research reinforces recent perspectives that one consideration beyond their economic benefits (e.g., can a machine do a job for less compensation than a person?) is how people respond to their use in social, political, and ethical domains, relative to the people they augment and/or replace.

Supplemental Material

sj-docx-1-psp-10.1177_01461672211016187 – Supplemental material for Assumptions About Algorithms’ Capacity for Discrimination

Supplemental material, sj-docx-1-psp-10.1177_01461672211016187 for Assumptions About Algorithms’ Capacity for Discrimination by Arthur S. Jago and Kristin Laurin in Personality and Social Psychology Bulletin

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Supplemental Material

Supplemental material is available online with this article.

ORCID iD

Arthur S. Jago

Notes

References

Bigman

Y. E.

Gray

(2018). People are averse to machines making moral decisions. Cognition, 181, 21–34.

Brynjolfsson

McAfee

(2014). The second machine age: Work, progress, and prosperity in a time of brilliant technologies. W. W. Norton.

Caliskan

Bryson

J. J.

Narayanan

(2017). Semantics derived automatically from language corpora contain human-like biases. Science, 356, 183–186.

Castelo

Bos

M. W.

Lehmann

D. R.

(2019). Task-dependent algorithm aversion. Journal of Marketing Research, 56(5), 809–825.

Chalfin

Danieli

Hillis

Jelveh

Luca

Ludwig

Mullainathan

(2016). Productivity and selection of human capital with machine learning. The American Economic Review, 106, 124–127.

Cuddy

A. J.

Fiske

S. T.

Glick

(2008). Warmth and competence as universal dimensions of social perception: The stereotype content model and the BIAS map. Advances in Experimental Social Psychology, 40, 61–149.

Dasgupta

DeSteno

Williams

L. A.

Hunsinger

(2009). Fanning the flames of prejudice: The influence of specific incidental emotions on implicit prejudice. Emotion, 9, 585–591.

Dasgupta

McGhee

D. E.

Greenwald

A. G.

Banaji

M. R.

(2000). Automatic preference for White Americans: Eliminating the familiarity explanation. Journal of Experimental Social Psychology, 36, 316–328.

Devine

P. G.

(1989). Stereotypes and prejudice: Their automatic and controlled components. Journal of Personality and Social Psychology, 56, 5–18.

10.

Dietvorst

B. J.

Simmons

J. P.

Massey

(2015). Algorithm aversion: People erroneously avoid algorithms after seeing them err. Journal of Experimental Psychology: General, 144, 114–126.

11.

Dovidio

J. F.

Gaertner

S. L.

(2000). Aversive racism and selection decisions: 1989 and 1999. Psychological Science, 11, 315–319.

12.

Ford

(2015). Rise of the robots: Technology and the threat of a jobless future. Basic Books.

13.

Frey

C. B.

Osborne

M. A.

(2017). The future of employment: How susceptible are jobs to computerisation? Technological Forecasting and Social Change, 114, 254–280.

14.

Fryer

R. G.

Levitt

S. D.

(2004). The causes and consequences of distinctively black names. The Quarterly Journal of Economics, 119, 767–805.

15.

Goldman

B. M.

Gutek

B. A.

Stein

J. H.

Lewis

(2006). Employment discrimination in organizations: Antecedents and consequences. Journal of Management, 32, 786–830.

16.

Gray

H. M.

Gray

Wegner

D. M.

(2007). Dimensions of mind perception. Science, 315, 619–619.

17.

Grove

W. M.

Zald

D. H.

Lebow

B. S.

Snitz

B. E.

Nelson

(2000). Clinical versus mechanical prediction: A meta-analysis. Psychological Assessment, 12, 19–30.

18.

James

Witten

Hastie

Tibshirani

(2013). An introduction to statistical learning. Springer.

19.

Jones

K. P.

Peddie

C. I.

Gilrane

V. L.

King

E. B.

Gray

A. L.

(2016). Not so subtle: A meta-analytic investigation of the correlates of subtle and overt discrimination. Journal of Management, 42, 1588–1613.

20.

Kite

M. E.

Whitley

B. E.

Jr. (2016). Psychology of prejudice and discrimination. Psychology Press.

21.

Kleinberg

Ludwig

Mullainathan

Sunstein

C. R.

(2019). Discrimination in the Age of Algorithms. Journal of Legal Analysis, 10, 113–174.

22.

Kosinski

Wang

Lakkaraju

Leskovec

(2016). Mining big data to extract patterns and predict real-life outcomes. Psychological Methods, 21, 493–506.

23.

Lambrecht

Tucker

(2019). Algorithmic bias? An empirical study of apparent gender-based discrimination in the display of STEM career ads. Management Science, 65, 2947–3448.

24.

Larson

Mattu

Kirchner

Angwin

(2016). How we analyzed the COMPAS recidivism algorithm. https://www.propublica.org/article/how-we-analyzed-the-compas-recidivism-algorithm

25.

Logg

J. M.

Minson

J. A.

Moore

D. A.

(2019). Algorithm appreciation: People prefer algorithmic to human judgment. Organizational Behavior and Human Decision Processes, 151, 90–103.

26.

Longoni

Bonezzi

Morewedge

C. K.

(2019). Resistance to medical artificial intelligence. Journal of Consumer Research, 46, 629–650.

27.

Mann

O’Neil

(2016, December 9). Hiring algorithms are not neutral. Harvard Business Review. https://hbr.org/2016/12/hiring-algorithms-are-not-neutral

28.

Meehl

P. E.

(1954). Clinical versus statistical prediction: A theoretical analysis and review of the literature. University of Minnesota Press.

29.

Newman

D. T.

Fast

N. J.

Harmon

D. J.

(2020). When eliminating bias isn’t fair: Algorithmic reductionism and procedural justice in human resource decisions. Organizational Behavior and Human Decision Processes, 160, 149–167.

30.

O’Neil

(2016). Weapons of math destruction: How big data increases inequality and threatens democracy. Crown.

31.

Shank

D. B.

DeSanti

(2018). Attributions of morality and mind to artificial intelligence after real-world moral violations. Computers in Human Behavior, 86, 401–411.

32.

Stone

D. L.

Lukaszewski

K. M.

Stone-Romero

E. F.

Johnson

T. L.

(2013). Factors affecting the effectiveness and acceptance of electronic selection systems. Human Resource Management Review, 23, 50–70.

33.

Tapias

M. P.

Glaser

Keltner

Vasquez

Wickens

(2007). Emotion and prejudice: Specific emotions toward outgroups. Group Processes & Intergroup Relations, 10, 27–39.

34.

Waytz

Norton

M. I.

(2014). Botsourcing and outsourcing: Robot, British, Chinese, and German workers are for thinking—Not feeling—Jobs. Emotion, 14, 434–444.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

1.46 MB