What Cognitive Mechanisms Do People Reflect on When They Predict IAT Scores?

Abstract

Research indicates that individuals can prospectively predict biases they will show on the implicit-association test (IAT). The present study uses the Quadruple process model to analyze data from Hahn, Judd, Hirsh, and Blair to investigate which cognitive mechanisms people reflect on when predicting their racial bias scores on the IAT. The Quadruple process model reveals that a combination of activated associations and self-regulatory control best explains what participants report when they predict their biases on the IAT. Furthermore, it appears to be specifically the total activation of positive attitudes toward Whites and negative attitudes toward minorities rather than negative attitudes toward minorities alone, that participants use to make their predictions.

Keywords

implicit attitudes racial bias quad model consciousness

Implicit attitudes have been a construct of considerable research and debate in the psychological sciences (Payne & Gawronski, 2010). One such debate has focused on the (un)consciousness of implicit attitudes, with influential work characterizing implicit attitudes as inaccessible to conscious awareness (e.g., Jost, Banaji, & Nosek, 2004; Phelps et al., 2000). Recent work has found evidence that people are able to prospectively predict the patterns of their biases on indirect measures of attitudes, suggesting that people do have access to at least some of the underlying processes that produce bias on indirect measures (Hahn et al., 2014).¹ But which of the processes reflected in implicit-association test (IAT) scores do people base their predictions on? Are people really aware of the evaluative associations they hold toward different targets or do they simply remember their ability to immediately correct for some biases more than others? And if people have access to the evaluative associations reflected in IAT scores, which of the different associations reflected in IAT scores specifically do they base their reports on? Using the Quadruple process model of implicit social cognition, the present study seeks to determine what component processes of implicit evaluation are accessible to conscious awareness and thus permit people to accurately predict their implicit biases. Hence, this work can shed a more precise light on the conscious and unconscious nature of aspects of implicit social cognition.

Indirect Measures and the Quadruple Process Model

Voluminous research now demonstrates that indirect attitude measures, including the IAT, are not process-pure measures of attitudes (Calanchini & Sherman, 2013). The typical operationalization of attitudinal biases on the IAT is the D-score (Greenwald, Nosek, & Banaji, 2003), which compares how long it takes participants to correctly sort stimuli on compatible and incompatible trials. The magnitude of implicit bias is inferred from the size of the discrepancy in the speed people sort stimuli on compatible versus incompatible trials, standardized by their individual response latencies (Greenwald, McGhee, & Schwartz, 1998). For instance, on an evaluative minority–majority IAT, participants are asked at one point to sort stimuli pertaining to the majority group with positive words and the minority group with negative words—the block involving this sorting may be called the “compatible” blocks for majority group members because it is compatible with societal bias. In other blocks, participants are asked to sort the minority groups with positive words and the majority group with negative words—incompatible with societal bias. The D-score is evaluated as the latency difference between these two block types, divided by their inclusive standard deviation. Importantly though, there are alternative and sometimes conflicting potential influences that determine how long it takes people to sort IAT stimuli. First, the speed of responding may be influenced by the associative relationship between two of the four concepts being paired together. In this case, a person who strongly associates a minority group with negativity would more quickly sort the minority group with negative words than with positive words. In contrast, a person who very weakly associates a minority group with negativity would show little difference between sorting the minority group with negative versus positive words. Second, however, it could reflect the association of the other two concepts. In this case, a person who more strongly associates the majority ingroup with positivity than negativity will, all else being equal, also show higher D-scores (Blanton & Jaccard, 2006). A third possibility is that executive control processes influence the speed of responding. In this case, a person who exerts a high degree of control will be able to resolve conflicting response tendencies more quickly than a person who exerts a lesser degree of control (Diamond, 2013). A person who can efficiently resolve conflicting response tendencies that arise on incompatible IAT trials in which associations activate one response impulse (e.g., to erroneously sort a minority group stimulus using the key for majority/negative stimuli) and when task demands activate another response impulse (e.g., to correctly sort a minority group stimulus using the key designated for minority/positive stimuli) will, all else equal, show reduced D-scores. While an advantage of the D-score is that it is agnostic to the direction of the bias (participants who hold associations contrary to societal bias may show a negative D-score), a limitation of the IAT D-score is that it cannot disentangle these potential influences. Thus, rather than process-pure, the D-score reflects a mixture of different associative relations and executive self-regulatory processes (Klauer, Schmitz, Teige-Mocigemba, & Voss, 2010).

In fact, identical IAT D-scores can result from entirely different component processes. As an example, Gonsalkorale, Sherman, and Klauer (2013) found that according to the IAT D-score, both younger (ages 21-40 years) and older (>65 years) participants showed similar levels of “anti-aging bias”; they more easily paired positive words with pictures of younger versus older adults. However, Gonsalkorale et al.’s (2013) reanalysis with the Quadruple process model revealed substantial differences in the cognitive mechanisms underlying IAT performance. Young participants had stronger biases favoring young targets but more easily overcame those biases while completing the IAT. Adults older than 65 years had weaker evaluative biases but were less able to overcome their biases. Hence, even though D-scores did not statistically differ between the two age groups, the cognitive mechanisms underlying those scores depended on the age group of test-takers.

In addition to conflating self-regulatory and associative processes, the magnitude of IAT D-scores can also reflect different associative processes. For instance, a promajority IAT score may result from accessible negative-outgroup associations, accessible positive-ingroup associations, or a mixture of both. In a series of studies, Sherman, Calanchini, and Schmidt (2017) used the Quad process model to assess how IAT bias could be understood as resulting from positive-ingroup associations or negative-outgroup associations. Across IATs measuring biases in race, sexuality, and age, ingroup participants generally showed more ingroup positivity than outgroup negativity. This illustrates another advantage of the Quad model; in addition to distinguishing between associations and self-regulation, the model can also estimate the degree to which component associations toward different social groups (e.g., Black people vs. White people) contribute to biased IAT performance.

As illustrated in these examples, the Quad process model uncovers cognitive processes responsible for IAT performance (Conrey, Sherman, Gawronski, Hugenberg, & Groom, 2005). The Quad model mathematically specifies four latent cognitive processes: (a) association activation (AC), which can later be further decomposed into separate associations (see below); (b) detection of task-relevant responses (D); (c) the ability to overcome associations when biases conflict with task-relevant responses (OB); and (d) response biases that measure participants’ guessing behavior (G). A set of equations specifies how the four processes interact to produce resultant correct and incorrect responding (see Figure 1). Returning to the anti-aging bias example, the Quad model assumes that viewing pictures of young faces activates positive associations, and viewing pictures of old faces estimates negative associations, each with a probability estimate ranging between 0 and 1, represented by the AC parameters. In this example, AC facilitates correct responding on IAT blocks where participants must sort young faces and positive words, as well as old faces and negative words together. However, AC interferes with correct responding on IAT blocks where participants must instead sort young faces and negative words together. Participants detect the appropriate sorting of targets (young faces, old faces, positive words, negative words) with probability D. When associations are not activated (1 – AC), detection (D) always yields correct responding. When activation (AC) and detection (D) processes are simultaneously active, overcoming bias (OB) represents the probability that detection (D) drives behavior, with (1 – OB) representing the probability that activation (AC) drives behavior. Guessing (G) captures behavior in the absence of activation (1 – AC) and detection (1 – D) in terms of a left-key (values below .5) as opposed to a right-key (values above .5) propensity.

Figure 1.

Visual representation of Quad model multinomial processing tree.

The mathematical specification of these component processes and their interaction permits strong inferential tests of hypothesized relationships between cognitive processes and other social behaviors independent of the IAT. The mathematical specification of the Quad model has been extensively validated to ensure that parameters can be selectively influenced (see Sherman et al., 2008, for a review). The parameters of the Quad model relate in expected ways to IAT latency scores (e.g., ACtivation is positively correlated with D-score bias while OB is negatively correlated; Conrey et al., 2005) and with concurrently measured neural activity (e.g., ACtivation is positively correlated with amygdala activity and Detection is positively correlated with activity in the anterior cingulate cortex/dorsolateral prefrontal cortex; Beer et al., 2008). Other recent tests have found that the Quad model formulation outperforms alternative plausible relationships between the cognitive processes in statistically accounting for IAT data (Calanchini, Sherman, Klauer, & Ferrer, 2017). Finally, simulation work shows that the estimation technique employed in the present article recovers parameter estimates and their standard errors from data with known distributions (Burke, 2015; Klauer, 2010). The ability to tightly constrain and validate the theoretical, verbal meaning of the model parameters is largely possible because they have corresponding mathematical definitions specified in the Quad model.

What Mechanisms Do People Reflect on in Predictions of Implicit Racial Bias?

Hahn and colleagues (2014) demonstrate that people can predict the patterns of their IAT D-score biases and thus have conscious access to some mechanism that contributes to IAT performance. Prominent psychological models of racial prejudice propose that there are benefits to increasing people’s awareness of their biases (e.g., Monteith & Mark, 2005). Thus, determining the specific cognitive processes that are, and are not, consciously accessible has important practical implications for evaluating the likely effectiveness of interventions aimed at raising awareness of bias. In addition, increased specificity about which processes that contribute to implicit attitude scores are consciously accessible to people can further hone theories of attitudes generally, and theories of implicit attitudes specifically (see below).

We use the Quad model to analyze data from Hahn et al. (2014) to determine which cognitive processes reflected in IAT scores best account for participants’ predictions. According to the Quad model, there are alternative possibilities for what cognitive mechanisms allow people to predict their biases on the IAT. As a first possibility, participants may be aware of the underlying attitudes that contribute to IAT performance. If participants introspect directly on evaluative associations when predicting their IAT performance, we should expect their predictions to positively correlate with the Quad model formulation of association ACtivation (AC).²

A second possibility is related to the Overcoming Bias (OB) parameter. The OB parameter reflects how much inhibitory control a person exerted on a specific IAT. Hence, it may reflect enduring executive function ability, as well as temporary shifts in exerted control, for example, due to differential motivation to control different biases. The first influence—enduring executive functioning abilities—cannot fully explain IAT score pattern prediction, as enduring ability would be constant across different IATs.³ However, the second possibility—the specific amount of control exerted on a particular IAT as opposed to other IATs—can contribute to IAT score pattern predictions and could therefore be part of what people predict in advance.

We see three theoretical reasons to think that inhibitory control processes may influence people’s predictions of their IAT score patterns. These reasons are not necessarily mutually exclusive, but each would be sufficient to account for people’s ability to predict the pattern of their IAT scores. First, the reflection of inhibitory control processes in IAT score predictions may come from participants’ past experiences self-regulating their behavior toward different social groups. One’s past history of success (or failure) in self-regulating biased behavior could serve both as a cue for prediction as well as contribute to people’s performance on the IAT measure. This possibility builds on the Self-Regulation of Prejudice model (Monteith & Mark, 2005), which posits that people recognize their failures to self-regulate biases. In turn, these failures produce experienced negative affect and lead to the development of cues for control that facilitate efficient self-regulation. As an example of this, someone who has a poor history of self-regulating his or her spontaneous behavior toward Black people (e.g., engaging in microaggressions) may recall these failures when predicting that an IAT comparing Blacks and Whites will reveal a strong bias. This same person may reflect on a more successful recent history of self-regulating spontaneous bias toward Latino people, leading them to predict that an IAT comparing Latinos and Whites will reveal relatively weaker bias. However, these predictions about bias would be accurate because of the relative efficacy of self-regulation rather than differences in evaluative associations.

A second reason for thinking that control processes might be part of participants’ predictions is based on research showing that chronic pursuit of egalitarian goals can routinize self-regulatory control, even over spontaneous biases like those reflected in indirect measures such as the IAT (Moskowitz, Gollwitzer, Wasel, & Schaal, 1999). Hence, if exposure to negatively evaluated racial groups spontaneously activates self-regulatory processes, these may be reflected in the biases participants predict to show in their IAT scores, even in cases where they cannot accurately recall situations in which they exerted self-control before.

A third reason is that cultural or personal ideals about which biases are more acceptable than others lead people to exert more inhibitory control over some biases than others. This may then be reflected in both their predictions about how much bias they will show, as well as in how much executive control they will exert on a particular IAT measuring said bias.

Importantly, all three of the processes by which executive control may be part of IAT score predictions mentioned hitherto are compatible with each other and all lead to the same prediction. If certain societal standards make some biases more acceptable than others, this should also influence how much control people exert over them in their daily lives, how many “cues for control” they hence establish in relation to a specific bias, how much those self-regulatory processes become routinized, how much bias they will show on a particular IAT, and also how much bias people sense they should show on an IAT.

A last possibility, in addition to people having introspective access to biased associations and/or inhibitory control, is that people would attempt to produce certain patterns of IAT scores, rather than having any insight into evaluative associations at all. One of our goals was to test whether participants’ IAT predictions would negatively correlate with the Quad model formulation of Overcoming Bias, either in addition, or instead of being correlated with the formulation of association ACtivation.⁴

Another goal of the present analysis is to assess which evaluative associations specifically people use when making their predictions. It may be the case that people generate IAT predictions by reflecting on their attitudes toward negatively evaluated groups. However, as explained above, because the IAT is a comparative measure that conflates negative evaluations toward one group with positive evaluations toward another (Blanton & Jaccard, 2006), a second possibility is that people may instead reflect on the totality of their relative positivity toward Whites and relative negativity toward minorities. As a last option, people may in fact only reflect on how positively they feel toward the positively evaluated group in different contexts (i.e., compared with some groups as opposed to other groups). Hence, our goal was to test the extent to which people’s predictions are best explained by the strength of attitudes toward negatively evaluated groups, strength of attitudes toward positively evaluated groups in different contexts, or instead reflect on the total relative positivity and negativity toward Whites and minorities.

We believe that distinguishing between these possibilities is vitally important for several reasons. First, it is important for our understanding of how to most effectively raise awareness of the biases reflected in indirect measures of attitudes. Second, this analysis may provide further theoretical clarity regarding the (un)consciousness of evaluative expressions revealed on indirect measures. As described earlier, whereas some literature has assumed that indirectly measured attitudes are unconscious (e.g., Jost et al., 2004; Phelps et al., 2000; for a review, see Gawronski, Hofmann, & Wilbur, 2006), other models describe implicit evaluations as associations that may be consciously rejected, and hence assume that spontaneous evaluations are consciously accessible (e.g., the associative-propositional evaluation [APE], Gawronski & Bodenhausen, 2006; the motivation and opportunity as determinants [MODE] model of attitudes, Olson & Fazio, 2008). If people reflect directly on racial attitudes when forming their predictions, we may then confidently reject attitude models that imply or define implicit attitudes as unconscious. If people reflect only on their ability to efficiently self-regulate biases toward different social groups, it might still be the case that the underlying implicit attitudes are unavailable to conscious awareness. In that case, people may only have knowledge of some general societal bias, but mainly reflect on how much they believe they will be able to regulate it, or even produce patterns on behavioral tasks. Finally, if people reflect on self-regulatory processes in combination with attitudinal processes, this implies a relative degree of sophistication in people’s ability to reflect on both their spontaneous reactions and their executive control.

Method

We apply the Quadruple process model to data from four experiments reported in Hahn and colleagues (2014). Experimental sessions were similar across each of the four experiments and primarily differed with respect to task instructions (see “Design and Procedure” section). None of the experimental variants produced detectable differences in the patterns of results and are therefore collapsed into a single data set for the present study. Data and analysis script are available at https://osf.io/rfuvn/.

Participants

A total of 430 undergraduate students (39% male) from the University of Colorado Boulder participated in one of four studies in exchange for course credit. Eighty percent identified as White, whereas the other participants identified as Asian (6.7%), Latino (3.7%), Black (2.1%), or another background (7.2%).⁵ The present sample consists of all available published data that use the repeated-measures paradigm necessary to assess the accuracy of IAT predictions. G*Power sensitivity analyses indicate that the sample affords 80% power to detect an R² = .018 in our multilevel analytic approach (Faul, Erdfelder, Buchner, & Lang, 2009).

Design and Procedure

Participants were seated in individual computer cubicles. After providing informed consent, they were asked to rate how warmly they felt toward different groups of people (Blacks, Latinos, Asians, and Whites, as well as additional social groups that were not race-based). Next, participants completed a training procedure that introduced them to the concept of implicit attitudes generally and the IAT measure specifically. Participants first viewed an explanation of explicit and implicit attitudes. Explicit attitudes were always described as those attitudes that people endorse and generally think of themselves as having. Descriptions of implicit attitudes varied between the studies.⁶ However, as these descriptions of implicit attitudes did not produce reliable differences (see Hahn et al., 2014, for details), they are not considered in the present study. Participants then learned that implicit attitudes would be assessed with the IAT sorting measure they would complete next.

After IAT practice with an insect–flower and a dog–cat IAT (except Study 4 where one condition received no practice), participants made predictions for five IAT scores, three of which compared people from different racial groups. Predictions were made on 7-point Likert-type scales ranging from 1 (a lot more positive toward BLACK [or ASIAN, or LATINO, respectively]) to 7 (a lot more positive toward WHITE; see Hahn et al., 2014).⁷ Participants always saw the pictures that would actually be used as target stimuli for the groups in the IATs.

Next, participants completed five IATs, among them the critical IATs comparing Black/White people, Latino/White people, Asian/White people.⁸ Afterward, they completed demographic measures and were debriefed. For the present study, we use only data from the three race-based IATs, because our research questions concern attitudes toward different racial groups and because Quad model parameters cannot be similarly interpreted when the compatibility structure of the IAT task changes (i.e., when the White comparison group is easier to pair with negative words than with positive words as was the case for the celebrity and child IAT).⁹

Results

Data Preparation

IAT D-score

An IAT D-score was computed for each of the three IAT measures completed by participants (for details, see Greenwald et al., 2003). More positive D-score values indicate faster responses on trials with compatible pairings (White-good, minority-bad) compared with trials with incompatible pairings (White-bad, minority-good).

Multinomial model description and estimation

To investigate the cognitive mechanisms that accounted for people’s ability to accurately predict their IAT results, we fit the Quad model to data from each of the three race IATs. We first performed standard checks of model fit where we fit the model to aggregated IAT data using the freely available MultiTree software (Moshagen, 2010). The analysis indicated that Quad model predictions significantly deviated from the data, G²(3) = 104.52, p < .001. Large samples, in this case 103,200 IAT responses, increase the power to detect even small deviations between observed response frequencies and predicted frequencies generated by the multinomial model (Cressie, Pardo, & Pardo, 2003). After controlling for power the magnitude of misfit was small, φ = .031 (R² = .001; Cohen, 1988).¹⁰

To obtain Quad model parameter estimates for each individual, we fit the model to each participant’s data using the “latent-trait” hierarchical method described in Klauer (2010). Broadly speaking, this approach uses information from both the aggregate and individual levels to increase the precision of parameter estimates. Analyses of simulated data demonstrate that the latent trait approach recovers parameter values without bias and accurately quantifies standard error (Klauer, 2010; also see Burke, 2015). The modeling procedure was conducted with the “TreeBUGS” package developed by Heck, Arnold, and Arnold (2016) for the R statistical platform (R Core Team, 2018).¹¹

Descriptive Results

We first assessed patterns of Quad model parameters, IAT D-scores, and IAT predictions for each IAT type averaged across participants. As depicted in Figure 2, participants predicted and subsequently showed the strongest bias on the Latino–White IAT, followed by the Black–White IAT, and the weakest bias on the Asian–White IAT. In contrast to these patterns of predictions and IAT D-scores, Quad model parameter estimates indicated that participants on average were more likely to activate biased evaluative associations on the Black–White IAT than on the Latino–White IAT. However, participants simultaneously exhibited greater self-regulation, as indexed by OB, on the Black–White IAT than on the Latino–White IAT. These patterns suggest that participants’ predictions might not reflect merely activated evaluative biases and also participants’ capacity to quickly self-regulate those biases. To test this, we next analyzed the relationships between each individual participant’s IAT predictions and the mechanisms underlying their IAT performance.

Figure 2.

Mean IAT score predictions, IAT D-scores, and Quad model parameters for Asian–White, Black–White, and Latino–White IATs.

Multilevel Analyses

Data structure

Each participant generated three predictions and completed three IATs. We fit a multilevel model where predictor and outcome variables were nested within subjects. This multilevel analytic model is appropriate to determine which cognitive processes are related to participants’ IAT predictions (but does not assess how accurately they were in describing their standing relative to others). The fitted model is also in line with Hahn et al.’s (2014) theorizing that within-subjects analyses provide the strongest test of introspective awareness whereas between-subjects analyses assess participants’ accuracy in predicting their biases relative to other test-takers in the reference group.¹² To investigate within-participant accuracy, we standardized participant IAT predictions, IAT D-scores, and Quad model parameters within subjects. This standardization procedure is necessary to investigate whether patterns of predictions correlate with patterns of IAT outcomes (see Hahn et al., 2014).

At the first level of the multilevel model, each participant’s predictions are a function of their IAT performance and a residual term. At the second level, the model estimates a fixed slope (γ₁₀) as well as a random slope (µ_1ij). The magnitude of the fixed slope, γ₁₀, indicates the degree to which participants’ predictions are related to their performance across the three IATs on average.

Multilevel model:

\begin{array}{l} L e v e l 1 : {(I A T p r e d i c t i o n)}_{i j} \\ = β_{0 i j} + β_{1 j} {(I A T r e s u l t)}_{i j} + ε_{i j}, \\ L e v e l 2 : β_{1 j} = γ_{10} + μ_{1 i j}, \end{array}

Where i = Participant 1, 2, 3 . . . n; j = IAT 1 (Black/White), 2 (Latino/White), and 3 (Asian/White); β_0ij = 0 (reflecting within-participant centering).

This multilevel model was fit to assess the relationship between participant’s IAT predictions and estimates of their (a) IAT D-score, (b) ACtivation of attitudes toward Whites and minorities (AC_total), (c) ACtivation of minority-specific attitudes (AC_minority), (d) ACtivation of White-specific attitudes (AC_White), (e) Overcoming Bias (OB), (f) Detection (D), and (g) Guessing (G). In a first step, we looked at the zero-order relationship between each of the processes and participants’ predictions. In a second step, we then looked at which one of two parameters would explain more variations in participants’ predictions in multilevel simultaneous regression analyses.

Results

Zero-order relationships between each of the parameters and predictions are presented in Table 1; Table 2 provides results of multiple regressions with two predictors.¹³ First, when considering only race-based IATs, the pattern of participants’ predictions detectably covaried with the pattern of their IAT D-scores, γ₁₀ = .236, 95% confidence interval (95% CI) = [.160, .312], t(322) = 6.136, p < .001 (see Table 1).

Table 1.

Zero-Order Relationships: IAT Performance Predictions Regressed Onto IAT Performance Metrics.

Multilevel parameter	IAT D-score	AC _total	AC _minority	AC _White	D	G	OB
Fixed effect
Predictors	.236***	.106**	.062	.073^†	−.099*	−.010	.009
Random-effect variance
Predictors	.244***	.288***	.300***	.265***	.275***	.263***	.233***
Residuals	.468***	.468***	.465***	.487***	.478***	.492***	.513***
Model R²	.056	.011	.004	.005	.009	<.001	<.001

Note. IAT = implicit-association test; AC = activation; OB = overcoming bias; D = detection; G = guessing.

†

p < .10. *p < .05. **p < .01. ***p < .001.

Table 2.

Simultaneous Regression: IAT Performance Predictions Regressed Onto IAT Performance Metrics.

Multilevel parameter	Regression Model 1: AC_total vs. OB	Regression Model 2: AC_total vs. AC_minority	Regression Model 3: AC_total vs. AC_White	Regression Model 4: AC_total vs. D
Fixed effect
AC_total	.160***	.146*	.126*	.126**
AC_minority		−.050
AC_White			−.017
OB	−.099*
D				−.075^†
Random-effect variance
AC_total	.305***	.190***	.248***	.427***
AC_minority		.216***
AC_White			.215***
OB	.228***
D				.418***
Residuals	.346***	.403***	.378***	.215***
Model R²	.016	.011	.013	.020

Note. IAT = implicit-association test; AC = activation; OB = overcoming bias; D = detection; G = guessing.

†

p < .10. *p < .05. **p < .01. ***p < .001.

Next, we assessed the degree to which attitudinal processes and self-regulatory processes accounted for participants’ ability to predict their IAT results. There was a positive zero-order relationship between participants’ IAT predictions and the total ACtivation of positive-White and negative-minority attitudes, γ₁₀ = .106, 95% CI = [.027, .185], t(322) = 2.636, p = .009. In contrast, there was no zero-order relationship between OB and predictions, γ₁₀ = .009, 95% CI = [–.068, .085], t(322) = 0.228, p = .820. These analyses thus show that participants’ predictions were indeed related to associative processes. This result is difficult to reconcile with the possibility that people were simply exerting control to manipulate the pattern of their IAT results.

However, when we tested whether a mixture of both attitudes and self-regulation could account for participants’ forecasts, results revealed that both ACtivation and OB were significantly related to participants’ predictions: AC was positively related, see Table 2, γ₁₀ = .160, 95% CI = [.067, .253], t(365.9) = 3.383, p < .001, whereas OB was negatively related, γ₁₀ = –.099, 95% CI = [–.187, –.010], t(353.6) = −2.180, p = .030. These analyses indicate that participants’ IAT predictions do not merely reflect not only spontaneous evaluative associations but also the immediate self-regulatory control participants exert in response to activated associations. Recall that descriptive analysis reported earlier showed that participants more often activated biased evaluative associations when responding to Blacks than Latinos, and also exhibited greater self-regulatory control when responding to Blacks than Latinos. The current results indicate that participants’ predictions about their implicit biases reflect these reactions in their entirety (i.e., biased impulses and self-regulation). Participants predicted the bias they would show when accounting for both activated associations and immediate control.¹⁴

We also investigated which specific attitudinal processes corresponded with people’s ability to predict their IAT results. Again, participants could make accurate IAT predictions if they have access to attitudes toward negatively evaluated groups, access to contextual variation in the strength of the positivity of attitudes toward positively evaluated groups, or have access to the total activation of attitudes for the two groups being compared. ACtivation of negative-minority attitudes did not reliably correlate with people’s IAT predictions, γ₁₀ = .062, 95% CI = [–.018, .142], t(322) = 1.533, p = .126. Nor was the relationship detectable for positive-White attitudes, although it was marginally significant, γ₁₀ = .073, 95% CI = [–.005, .151], t(322) = 1.835, p = .067. Next, we again assessed whether the total ACtivation of White and minority attitudes related to participant’s IAT predictions above and beyond the two component attitudes. We simultaneously entered the total magnitude of ACtivation and the negative-minority ACtivation variables as predictors in the multilevel model. Results revealed that total ACtivation still related to participants’ predictions, γ₁₀ = .146, 95% CI = [.030, .261], t(596.9) = 2.482, p = .013, and that negative-minority ACtivation was not related to participants’ predictions, γ₂₀ = –.050, 95% CI = [–.166, .066], t(630.6) = −0.845, p = .398. When entered with White-positive ACtivation, total ACtivation still covaried with people’s predictions, γ₁₀ = .126, 95% CI = [.025, .226], t(478.9) = 2.449, p = .015, whereas positive-White ACtivation was not related to participants’ predictions, γ₂₀ = –.017, 95% CI = [–.116, .082], t(468.0) = −0.337, p = .736. Consistent with perspectives emphasizing the comparative nature of the IAT, it was the total ACtivation of White and minority attitudes that was related to participants’ predictions.

Finally, we conducted exploratory analyses to assess whether either Guessing or Detection processes correlated with people’s IAT predictions. There was no relationship with Guessing, γ₁₀ = –.010, 95% CI = [–.088, .068], t(322) = −0.254, p = .799. Unexpectedly, there was a relationship between IAT predictions and Detection, γ₁₀ = –.099, 95% CI = [–.177, –.020], t(322) = −2.480, p = .014. This indicates that when people predicted greater biases toward a particular racial group, they were less likely to detect correct responses on that particular IAT. Because the latter effect was unexpected, we simultaneously regressed participants’ predictions onto their Detection and ACtivation parameters. In this analysis, ACtivation continued to be related to participants’ prediction, γ₁₀ = .126, 95% CI = [.038, .214], t(257.4) = 2.82, p = .005, whereas the relationship between predictions and Detection was marginal, γ₁₀ = –.075, 95% CI = [–.163, .012], t(253.5) = −1.69, p = .091. This suggests that participants may not have in fact predicted difficulties in distinguishing the stimuli and their effects on their IAT scores (i.e., they may not have predicted the Detection parameter’s influence on the score). Instead, these results suggest that participants only predicted ACtivation and self-regulatory control, but that those who predicted and showed greater White over minority preference had more difficulty distinguishing IAT stimuli. But, this difficulty was not significantly related to their predictions over and above evaluations. Corroborating this interpretation, we found a negative within-subject relationship between ACtivation and Detection when we regressed the former onto the latter, γ₁₀ = –.081, 95% CI = [–.148, –.014], t(429) = −2.36, p = .019.

Moderation by race

We conducted exploratory analyses to determine whether participants’ self-reported race moderated the relationship between IAT predictions and the AC parameters. We created a centered dichotomous race factor (White vs. non-White) that we permitted to interact with all Level 1 predictors in our simultaneous regression analyses (Models 1-3 in Table 2).

Considering the joint prediction of ACtivation and OB (Model 1), results indicated that race marginally interacted with ACtivation, t(383.5) = −1.671, p = .096, but not with OB, t(371.7) = 0.267, p = .790. We decomposed this marginal interaction by dummy codes to estimate the slopes for White and non-White participants separately. Results showed that AC_total detectably covaried with both Whites’ and with non-Whites’ predictions, although it was stronger for non-Whites, γ = .31, 95% CI = [.113, .513], t(391.61) = 3.072, p = .002, than for White participants, γ = .12, 95% CI = [.016, .226], t(355.48) = 2.259, p = .024. One tentative interpretation of the marginally bigger slopes for non-White as opposed to White participants is that non-White participants have greater variability in their IAT scores (average SD per participant in IAT scores: M_SD = .32, SD_SD = .18), whereas White participants showed less variability on IATs for the three minority groups tested here (average per-person SD: M_SD = .26, SD_SD = .14), difference in average SD White versus non-White: t(426) = 3.630, p < .001. Hence, minority participants may have been marginally more accurate in predicting variations in their IAT scores across the three IATs because there was more variation to predict.

Common-cause mechanisms for IAT predictions and IAT D-scores

Finally, we sought to explore whether the Quad model processes we discerned so far in fact statistically explained the relationship between participants’ IAT predictions and their IAT D-scores. Our findings suggest that to make accurate IAT predictions, people introspect on their latent biases and ability to self-regulate those biases. The Quad process model assumes these same cognitive processes drive observable bias on the IAT, as most commonly indexed using the D-score index. Such a relationship is consistent with a “spurious” or common cause mediation model (see Figure 3).

Figure 3.

Path diagram illustrating the relationship between IAT predictions and IAT D-scores, with ACtivation and overcoming bias as “common causes.”

To test whether the pattern of our results was consistent with this conceptualization, we conducted a series of exploratory regression analyses in R (analytic code available on Open Science Framework [OSF]).¹⁵ First, we regressed IAT predictions and IAT D-scores onto AC_total and OB individually. We standardized and saved the residuals from these two models and regressed the residuals from the IAT D-score model onto the residuals from the IAT predictions model. This model tests whether, after controlling for AC_total and OB, participants’ IAT predictions were still related to their IAT D-scores. Results showed that this relationship was still reliable, γ = .159, 95% CI = [.071, .246], t(322) = 3.576, p < .001 (the zero-order standardized relationship between those variables was .236, see Figure 3 and Table 1). In the next step, we sought to determine whether the relationship between IAT predictions and IAT D-scores was attenuated after controlling for AC_total and OB. To do this, we bootstrapped 10,000 replicates comparing the slopes in the two multilevel models, which assessed the relationship between predictions and D-scores before and after partialling out the effects of AC_total and OB. Results from this analysis showed that the relationship between IAT predictions and IAT D-scores was statistically reduced—exhibited by a confidence interval excluding 0—after we partialled out the effects of ACtivation and Overcoming Bias, Δγ = .077, 95% CI = [.017, .168].

In sum, the pattern of results of the common cause analysis is consistent with the hypothesis that ACtivation and Overcoming Bias partially explain the relationship between people’s IAT predictions and their IAT D-scores. Future studies should seek to follow up on this exploratory finding.

Discussion

The present study investigated which cognitive mechanisms reflected in IAT scores people reflect on to predict racial bias on the IAT. We identified several potential mechanisms by which people could generate predictions. People might have awareness of their underlying racial attitudes or they might reflect on their ability to regulate racial biases, or both. Results revealed that it was in fact a mixture of associations and control that best explained within-subject variability in participants’ predictions for the biases they would show. When comparing the zero-order relationship between the Quad model parameters and IAT scores predictions, only ACtivation was significantly related while Overcoming Bias was not. However, when entered simultaneously, Overcoming Bias explained variance over and above the ACtivation alone. The pattern of relationships between participants’ IAT predictions, their IAT D-scores, and ACtivation suggested that ACtivation as formulated by the Quad model was, statistically, a common cause for both predictions and latency scores. Finally, we found evidence that the sum total of White-positive and minority-negative associations formed the basis for people’s IAT predictions.

Put in broader perspective, participants in the studies used for the present analyses had just learned about the IAT (presumably for the first time, Hahn et al., 2014) and were asked to look at the pictures to be used in the IATs to predict how they would react toward them. The current results indicate that participants were not just acknowledging their biased evaluative associations but also considering, to some extent, how much they self-regulate those evaluative associations, to make their predictions.

We outlined several ways in which participants’ predictions may be related to control processes reflected in the OB parameter. According to one of these possibilities, participants were not in fact predicting activated associations, but planning to produce a specific pattern of results on IATs. In this case, IAT score predictions should have been most strongly related to the OB parameter and not related to the AC parameters at all. Hence, the present results speak against such an interpretation. Participants’ predictions were most strongly related to activated associations and only related to additional control processes once their relationship with associations was accounted for in these analyses.

Instead, these results are much more compatible with the notion that participants make predictions based on the evaluative associations that are triggered by certain stimuli, corrected only slightly by how much differential inhibitory control they exert over those activated associations. This poses the question of whether people have access to the two component processes—evaluative associations and control—or only perceive their combined outputs. That is, one possible interpretation of these results is that participants know their initially activated affective responses and also separately feel how they may be able to exert control over them, and make a prediction based on the combined effect of those two separate processes. Another possibility is that participants only perceive the combined outcome of activated associations and control. From this perspective, a person’s intuitions or gut reactions reflect not only spontaneously activated affective reactions but also immediately activated inhibitory control that has been routinized. This suggests the intriguing possibility that self-regulatory control, as reflected on the OB parameter, may be routinely activated in concert with racial evaluations such that it gets integrated in participants’ appraisals of their gut reactions. This theorizing is consistent with research that has shown that egalitarian goals can be activated relatively automatically upon perceiving racial stimuli and can exert influence even on rapid response time measures (Moskowitz et al., 1999) and consistent with the Self-Regulation of Prejudiced Responses Model, which describes the development of cues for self-regulation (Monteith & Mark, 2005).

Although the first idea (that inhibitory control is consciously accessible) is fascinating, we find the second possibility more likely. First, Hahn et al. (2014) found that it did not matter whether participants knew how the IAT worked to make accurate predictions. If participants had conscious access to their inhibitory control and deliberately used that both to make their predictions and to complete the IATs, it should have mattered whether they knew how the test worked to make accurate predictions on how much control they could exert on them. Second, several recently published articles have highlighted the surprisingly weak correlation between people’s self-reported executive control and their performance on tasks designed to assess executive control (e.g., Buchanan, 2016). Although these low correlations do not in themselves have to denote a lack of introspective access (as low implicit–explicit correlations in attitudes do not denote lack of awareness), they provide preliminary evidence that contradicts the notion that participants are aware of their self-regulatory abilities. Hence, we find it more likely that participants based their predictions on the combined outcomes of both activated associations and immediate control. However, the current data cannot ultimately answer this question, as they are compatible with both explanations. Future research is needed to assess whether people are able to consciously perceive inhibitory control processes, or whether our perceptions of automatic responses reflect combinations of activated impulses and immediate control.

Which attitudes did participants consider when predicting their IAT performance? The present evidence indicated that participants’ predictions were related to the total ACtivation of their attitudes toward Whites and minorities. This is especially interesting as one may expect people’s forecasts to primarily rely on attitudes toward minority groups. That is, participants might show the same degree of pro-White bias on each IAT independent of the comparison minority group, as evaluations reflect the same category (i.e., WHITE) at each measurement occasion. This, in turn, leads to the expectation that participants are predicting the variation in minority attitudes. Contrary to this thinking, however, the data suggest that the total ACtivation of attitudes toward Whites and minorities accounted for participants’ predictions, for both White and non-White participants. This finding is congruent with those who assert that the IAT is a relative measure of attitudes and that IAT performance is inherently dependent on participants comparing and contrasting two different social groups (Blanton & Jaccard, 2006). It is also in line with research showing that attitudes toward the same social group (in this case White) are contextually malleable and not constant across three comparison groups (Blair, 2002; Gawronski & Sritharan, 2010). Finally, it is also compatible with large body of literature that illustrates the large role of ingroup preference (as opposed to just outgroup derogation) in producing discrimination (see Greenwald & Pettigrew, 2014, for a recent review). Our studies confirm that, rather than only predicting their varying negative associations toward minority groups, participants used a combination of these negative associations with varying positive associations toward their ingroup to make their predictions. It will be important to follow up on these exploratory analyses by directly testing this possibility in future work.

Unexpectedly, predicting larger biases was associated with reduced Detection. Additional analyses suggested that predictions did not covary with Detection after accounting for ACtivation. Rather participants predicted their evaluative biases, but those with stronger biases had more difficulty detecting IAT stimuli. Although we had no a priori expectations for this relationship, one interpretation is that participants may experience heightened levels of anxiety when completing an IAT on which they expect to show undesirable racial bias. This increase in anxiety may interfere with performance on a cognitively demanding task such as the IAT (e.g., Payne, 2005; Richeson & Shelton, 2003). This interpretation is further corroborated by the fact that ACtivation and Detection can be selectively influenced (Conrey et al., 2005), such that this relationship may be specific to cases where participants expect to show bias (as when they predict their scores). Additional work will be needed to better understand the relationship between forecasted racial bias and the ability to detect task-appropriate responses on the IAT.

In exploratory analyses, we tested whether the processes by which people predict their IAT scores were moderated by the person’s race. The results showed only a marginal difference; non-White participants’ IAT predictions were slightly more related to ACtivation. We speculated that this difference may be a function of the fact that non-White participants had greater variability in their scores on the three IATs. Although we could not test for them, there are other potential moderators that follow from theoretical models developed to explain the self-regulation of prejudice (e.g., Monteith & Mark, 2005).

Specifically, it should theoretically be only people who are highly motivated to control for their biased reactions (e.g., people high in motivation to control prejudiced reactions) who base their predictions on combined outcomes of associations and self-regulation. A person with little motivation to control for their biased associations should place relative more emphasis on their reactions only, and relatively less emphasis on their (relatively weaker) self-regulatory processes than a person who scores highly on such motivations. More generally, there may be other individual difference variables that determine whether a person relies more heavily on self-regulatory ability or associations in reporting their spontaneous reactions (or predicting IAT scores). Future research should explore these differences.

Caveats and Limitations

Considering running simultaneous regression analyses with Quad model parameters, it is important to note that parameter estimates are derived from the same data. This may pose challenges for regression analyses as predictors share measurement error. Notwithstanding such legitimate concerns, all analyses in this case yielded theoretically consistent results. These results suggest that it is sensing and comparing the activated associations between two groups and the immediate self-regulatory control that is initiated in response to these activated associations that jointly explain people’s ability to predict their IAT scores. However, given the challenges described above, this interpretation should be viewed as generative, requiring future research to disentangle the contribution of a mechanism based on evaluative comparison from the contribution of component racial attitudes (e.g., investigating people’s predictions about performance on a series of single-category IATs).

Another limitation of the current analyses is that the Quad model only permitted us to base our analyses on only three predictions and IAT scores per person (see earlier explanation of assumptions about the compatibility structure). One result is that a nontrivial proportion of participants did not have any variability with respect to their IAT predictions. This is partly due to the fact that biases toward the different groups are highly related. That is, the between-subjects correlation between the Black and Latino, the Black and Asian, and the Latino and Asian IATs are .62, .54, and .57, respectively, suggesting that every individual participant showed highly similar levels of bias against the groups and thus had limited variance to predict in his or her individual pattern. In fact, this was the basis upon which Hahn et al. (2014) included additional IATs that would presumably increase variance in each participants’ bias scores.

The present analyses nevertheless converged, finding a relationship between participants’ (variance-restricted) predictions and attitudes. This indicates that a sizable proportion of participants could predict even small variations in racial biases (although the size of the fixed effects slope for prediction accuracy was expectably lower, 0.24 vs. 0.54 in Hahn et al., 2014). This also refutes another interpretation of Hahn et al. (2014) that participants were simply predicting a normative pattern of negative-minority bias as opposed to positive bias in favor of children and celebrities. The current results show that participants were able to predict evaluative reactions even toward three groups where one would not expect a normative pattern.

Another disadvantage of basing our results on only three standardized data points per person was that we could never test for more than two Level 1 predictors simultaneously. This precluded a variety of tests, such as simultaneous tests of all Quad model predictors and tests for interactive effects between two Quad model predictors. Notwithstanding these limitations, findings from several analyses converged on a theoretically meaningful pattern of results. Future research is needed where participants complete more IATs with similar compatibility structures to test these additional hypotheses.

Outlook

In the first 2016 presidential debate, then-presidential candidate Hillary Clinton raised the importance of implicit bias in modern society. Many in the press and in the academic world marked Clinton’s mention as an important moment of public exposure for the term (e.g., Powell, 2016). Specifically, Clinton implored citizens to take stock of their potentially biased implicit attitudes, “I think we need all of us to be asking hard questions about, you know, ‘Why am I feeling this way?’” (American Presidency Project, 2016). Some social-psychological conceptualizations of attitudes are incompatible with the possibility that people could have conscious access to their implicit or “unconscious” biases (e.g., Jost et al., 2004; Phelps et al., 2000). Other models of attitudes such as the APE model (Gawronski & Bodenhausen, 2006) or the MODE model (Olson & Fazio, 2008) reject the assumption that implicit attitudes are unconscious. The work of Hahn et al. (2014) supports the latter models and hence Clinton’s conceptualization; people can reflect on their implicit biases. The present work extends the work of Hahn et al. (2014) further by demonstrating that people do in fact reflect on the underlying evaluative associations as well as on rapid self-regulatory control processes that jointly determine behavior on measures like the IAT. In other words, people are able to prospectively account for their spontaneous biases and their ability to self-regulate those biases.

The present research can inform recent discussions about the potential moral hazards of emphasizing the unconscious nature of spontaneous forms of racial bias. Philosophers have vigorously debated the degree to which “unconscious” bias implies that people are less responsible for discrimination (e.g., Holroyd, 2012). However, if people can introspect on their spontaneous racial attitudes and on their ability to control them, focus on (un)consciousness would miss the point. These discussions have also emerged in circles of social psychologists and in public media (Goldhill, 2018; Onyeador, Shapiro, Henderson, Daumeyer, & Richeson, 2018). Emphasizing the unconscious nature of implicit bias, perpetrators of discrimination are viewed as less morally responsible for their actions (Cameron, Payne, & Knobe, 2010; also see Redford & Ratliff, 2016). Thus, the literature has documented potential costs to framing implicit biases as unconscious. The present work demonstrates further that this framing is imprecise at best; people can introspect on the processes responsible for implicit biases when asked to do so. It seems necessary to begin a new conversation to weigh the costs and benefits of using the term “unconscious” to describe the biases measured by indirect measures such as the IAT.

People’s awareness of even sensitive racial attitudes raises important questions for psychologists. What are the consequences of introspecting upon one’s biases? Do people differ in their ability to introspect? How can we leverage awareness of implicit attitudes to change people’s attitudes and/or behavior? Assuming that we can identify conditions where increasing acknowledgment of implicit attitudes is beneficial, what situations facilitate acknowledgment? The present research suggests that people naturally attend to evaluative differences between racial groups when making predictions about their implicit biases. Thus, it is possible that focusing attention on evaluative differences between different racial groups would be a productive route to raising awareness of one’s biased attitudes (see also Hahn & Gawronski, in press).

The present research demonstrates that people’s own introspective sense of their “implicit bias” is best explained by a combination of activated evaluative associations and immediate control of behavioral impulses resulting from those activated associations. These findings bring us closer to understanding what attitudinal mechanisms people have introspective access to. We hope that this knowledge in turn facilitates effective methods for raising awareness and acknowledgment of racial biases.

Supplemental Material

Rivers_OnlineAppendix – Supplemental material for What Cognitive Mechanisms Do People Reflect on When They Predict IAT Scores?

Supplemental material, Rivers_OnlineAppendix for What Cognitive Mechanisms Do People Reflect on When They Predict IAT Scores? by Andrew M. Rivers and Adam Hahn in Personality and Social Psychology Bulletin

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Notes

Supplemental Material

Supplementary material is available online with this article.

References

American Presidency Project. (2016, September 26). Presidential debate at Hofstra University in Hempstead, New York. Santa Barbara: University of California at Santa Barbara. Retrieved from http://www.presidency.ucsb.edu/debates.php

Bates

Maechler

Bolker

Walker

(2015). Fitting linear mixed-effects models using lme4. Journal of Statistical Software, 67, 1-48.

Beer

J. S.

Stallen

Lombardo

M. V.

Gonsalkorale

Cunningham

W. A.

Sherman

J. W.

(2008). The Quadruple process model approach to examining the neural underpinnings of prejudice. NeuroImage, 43, 775-783.

Blair

I. V.

(2002). The malleability of automatic stereotypes and prejudice. Personality and Social Psychology Review, 6, 242-261.

Blanton

Jaccard

(2006). Arbitrary metrics in psychology. American Psychologist, 61, 27-41.

Buchanan

(2016). Self-report measures of executive function problems correlate with personality, not performance-based executive function measures, in nonclinical samples. Psychological Assessment, 28, 372-385.

Burke

C. T.

(2015). Process dissociation models in racial bias research: Updating the analytic method and integrating with signal detection approaches. Group Processes & Intergroup Relations, 18, 402-434.

Calanchini

Sherman

J. W.

(2013). Implicit attitudes reflect associative, non-associative, and non-attitudinal processes. Social and Personality Psychology Compass, 7, 654-667.

Calanchini

Sherman

J. W.

Klauer

K. C.

Ferrer

(2017). An empirical validation of the Quadruple process model of implicit attitudes against alternate, theoretically defensible specifications. Retrieved from PsyArXiv. doi:10.17605/OSF.IO/7B6NZ

10.

Cameron

C. D.

Payne

B. K.

Knobe

(2010). Do theories of implicit race bias change moral judgments? Social Justice Research, 23, 272-289.

11.

Canty

Ripley

(2017). Functions and datasets for bootstrapping. Retrieved from https://CRAN.R-project.org/package=boot

12.

Cohen

(1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum.

13.

Conrey

F. R.

Sherman

J. W.

Gawronski

Hugenberg

Groom

C. J.

(2005). Separating multiple processes in implicit social cognition: The Quad model of implicit task performance. Journal of Personality and Social Psychology, 89, 469-487.

14.

Cressie

Pardo

M. C.

(2003). Size and power considerations for testing loglinear models using φ-divergence test statistics. Statistica Sinica, 13, 555-570.

15.

Diamond

(2013). Executive functions. Annual Review of Psychology, 64, 135-168.

16.

Eager

C. D.

(2017). Standardize: Tools for standardizing variables for regression in R. Retrieved from https://CRAN.R-project.org/package=standardize

17.

Faul

Erdfelder

Buchner

Lang

(2009). Statistical power analyses using G*Power 3.1: Tests for correlation and regression analyses. Behavior Research Methods, 41, 1149-1160.

18.

Gawronski

Bodenhausen

G. V.

(2006). Associative and propositional processes in evaluation: An integrative review of implicit and explicit attitude change. Psychological Bulletin, 132, 692-731.

19.

Gawronski

Hofmann

Wilbur

C. J.

(2006). Are “implicit” attitudes unconscious? Consciousness and Cognition, 15, 485-499.

20.

Gawronski

Sritharan

(2010). Formation, change, and contextualization of mental associations: Determinants and principles of variations in implicit measures. In Gawronski

Payne

B. K.

(Eds.), Handbook of implicit social cognition: Measurement, theory, and applications (pp. 216-240). New York, NY: Guilford Press.

21.

Goldhill

(2018). “Implicit bias” tests help people feel morally superior, even when their results show bias. Retrieved from https://qz.com/1236594/

22.

Gonsalkorale

Sherman

J. W.

Klauer

K. C.

(2013). Measures of implicit attitudes may conceal differences in implicit associations: The case of antiaging bias. Social Psychological & Personality Science, 5, 271-278.

23.

Greenwald

A. G.

McGhee

D. E.

Schwartz

J. L. K.

(1998). Measuring individual differences in implicit social cognition: The implicit association test. Journal of Personality and Social Psychology, 74, 1464-1480.

24.

Greenwald

A. G.

Nosek

B. A.

Banaji

M. R.

(2003). Understanding and using the implicit association test: I. An improved scoring algorithm. Journal of Personality and Social Psychology, 85, 197-216.

25.

Greenwald

A. G.

Pettigrew

T. F.

(2014). With malice toward none and charity for some. American Psychologist, 69, 669-684.

26.

Hahn

Gawronski

(in press). Facing one’s implicit biases: From awareness to acknowledgement. Journal of Personality and Social Psychology. Retrieved from https://osf.io/n58tv/

27.

Hahn

Judd

C. M.

Hirsh

H. K.

Blair

I. V.

(2014). Awareness of implicit attitudes. Journal of Experimental Psychology: General, 143, 1369-1392.

28.

Heck

D. W.

Arnold

N. R.

Arnold

(2016). TreeBUGS: An R package for hierarchical multinomial modeling. Retrieved from https://github.com/denis-arnold/TreeBUGS

29.

Holroyd

(2012). Responsibility for implicit bias. Journal of Social Philosophy, 43, 274-306.

30.

Jost

J. T.

Banaji

M. R.

Nosek

B. A.

(2004). A decade of system justification theory: Accumulated evidence of conscious and unconscious bolstering of the status quo. Political Psychology, 25, 881-919.

31.

Klauer

K. C.

(2010). Hierarchical multinomial processing tree models: A latent-trait approach. Psychometrika, 75, 70-98.

32.

Klauer

K. C.

Schmitz

Teige-Mocigemba

Voss

(2010). Understanding the role of executive control in the implicit association test: Why flexible people have small IAT effects. The Quarterly Journal of Experimental Psychology, 63, 595-619.

33.

Monteith

M. J.

Mark

A. Y.

(2005). Changing one’s prejudiced ways: Awareness, affect, and self-regulation. European Review of Social Psychology, 16, 113-154.

34.

Moshagen

(2010). MultiTree: A computer program for the analysis of multinomial processing tree models. Behavior Research Methods, 42, 42-54.

35.

Moskowitz

G. B.

Gollwitzer

P. M.

Wasel

Schaal

(1999). Preconscious control of stereotype activation through chronic egalitarian goals. Journal of Personality and Social Psychology, 77, 167-184.

36.

Olson

M. A.

Fazio

R. H.

(2008). Implicit and explicit measures of attitudes: The perspective of the MODE model. In Petty

R. E.

Fazio

R. H.

Brinol

(Eds.), Attitudes: Insights from the new implicit measures (pp. 19-63). New York, NY: Psychology Press.

37.

Onyeador

I. N.

Shapiro

Henderson

Daumeyer

Richeson

(February, 2018). Presumed unintentional: The ironic effects of implicit bias framing on whites’ perceptions of discrimination. In Daumeyer

Onyeador

(Chairs), Rethinking implicit and explicit bias in contemporary American society. Paper presented at the 19th Annual Meeting of the Society for Personality and Social Psychology, Atlanta, GA.

38.

Payne

B. K.

(2005). Conceptualizing control in social cognition: How executive functioning modulates the expression of automatic stereotyping. Journal of Personality and Social Psychology, 89, 488-503.

39.

Payne

B. K.

Bishara

A. J.

(2009). An integrative review of process dissociation and related models in social cognition. European Review of Social Psychology, 20(1), 272-314.

40.

Payne

B. K.

Gawronski

(2010). A history of implicit social cognition: Where is it coming from? Where is it now? Where is it going? In Gawronski

Payne

B. K.

(Eds.), Handbook of implicit social cognition: Measurement, theory, and application (pp. 1-15). New York, NY: Guilford Press.

41.

Phelps

E. A.

O’Connor

K. J.

Cunningham

W. A.

Funayama

E. S.

Gatenby

J. C.

Gore

J. C.

Banaji

M. R.

(2000). Performance on indirect measures of race evaluation predicts amygdala activation. Journal of Cognitive Neuroscience, 12, 729-738.

42.

Powell

J. A.

(2016, September 27). Implicit bias in the presidential debate [Blog post]. Retrieved from http://blogs.berkeley.edu/2016/09/27/implicit-bias-in-the-presidential-debate/

43.

R Core Team. (2018). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing.

44.

Redford

Ratliff

K. A.

(2016). Perceived moral responsibility for attitude-based discrimination. British Journal of Social Psychology, 55, 279-296.

45.

Richeson

J. A.

Shelton

J. N.

(2003). When prejudice does not pay: Effects of interracial contact on executive function. Psychological Science, 14, 287-290.

46.

Sherman

J. W.

Calanchini

Schmidt

(July, 2017). Intergroup bias (generally) reflects more positivity than negativity. Paper presented at the 18th General Meeting of the European Association of Social Psychology, Granada, Spain.

47.

Sherman

J. W.

Gawronski

Gonsalkorale

Hugenberg

Allen

T. J.

Groom

C. J.

(2008). The self-regulation of automatic associations and behavioral impulses. Psychological Review, 115, 314-325.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.16 MB