Using the Implicit Association Test as an Unconsciousness Raising Tool in Psychology

Abstract

People are reluctant to admit they harbor implicit biases. Students (N = 68) from four social psychology courses completed an assignment designed to raise awareness about implicit biases. After completing an Implicit Association Test (IAT), students answered six essay questions, read two articles on the IAT, and answered five additional essay questions. Before the readings, students showed uncertainty about the IAT’s ability to measure their implicit attitudes. The main reason students gave in support of or against the IAT’s validity was the congruency between their implicit and explicit attitudes. After the readings, more students agreed that the IAT measured prejudice. This assignment was a useful tool in raising students’ awareness about their unconscious biases and teaching them about implicit attitudes.

Keywords

implicit attitudes unconscious awareness teaching tool IAT

As the United States becomes more diverse, the discussion and eradication of implicit biases has become more important. Today, contemporary and subtle forms of discrimination such as modern, symbolic (Blatz & Ross, 2009; McConahay & Hough, 2010), and aversive racism (Penner et al., 2010) are more common than old-fashioned prejudice and overt discrimination against stigmatized groups (Dovidio & Gaertner, 2004; Glick & Fiske, 2001). Although many individuals explicitly endorse egalitarian beliefs, they may be unaware that they harbor unconscious implicit biases (Dovidio & Gaertner, 2004; Greenwald, McGhee, & Schwartz, 1998). These subtle, unconscious biases make it difficult for educators to raise awareness of continuing inequality and the persistence of racism, sexism, and other forms of discrimination in the United States (Jones, Engelman, Turner, & Campbell, 2009).

Social psychologists have developed response latency measures such as priming (Fazio & Olson, 2003) and the Implicit Association Test (IAT; Greenwald et al., 1998) to measure implicit, or unconscious and uncontrollable, biases. The IAT assesses reaction time and error rate in categorizing two target concepts (e.g., Black vs. White) with an attribute (e.g., good vs. bad). When the categorization of stimuli (e.g., Black/White names, good/bad words) is consistent with cultural prejudice (e.g., Black/bad, White/good), reaction time, and error rate are lower for individuals with implicit biases compared to the inconsistent pairing (e.g., Black/good, White/bad). These techniques offer advantages over self-report measures because they measure unconscious, automatic, and uncontrollable responses rather than conscious, controlled responses (Fazio & Olson, 2003) and thus are useful for measuring socially undesirable biases.

In addition to being a useful tool for measuring implicit biases, educators can use the IAT to raise awareness among students about their unconscious biases (Morris & Ashburn-Nardo, 2010). A popular website shares the science behind the IAT with the general public, as well as provides demonstrations for people to test their own biases (IAT Corp., 2011). These research tools are available for classroom instruction as a form of unconsciousness raising to help students become aware of implicit biases they may harbor (Jost et al., 2009; Morris & Ashburn-Nardo, 2010; Thompson, 2003).

Recent unpublished research examining students’ reactions to their race IAT scores has documented a wide variety of responses (Bennett, Handy, Guenther, & Nolan, 2011; Frazier, Nosek, Banaji, & Greenwald, 2011). Frazier, Nosek, Banaji, and Greenwald (2011) found that the greater participants’ implicit pro-White bias, the more they criticized the validity of the IAT. Common justifications for negative IAT feedback included describing performance as driven by in-group preference rather than prejudice, the structure of the IAT, family upbringing, lack of exposure to other cultures, and cultural influences (Bennett et al., 2011; Frazier et al., 2011). Furthermore, Morris and Ashburn-Nerdo (2010) used the IAT in classroom instruction and found that after taking the IAT students generally had more positive affective responses, were more likely to acknowledge that implicit biases exist, and believed that the classroom demonstration was a useful tool, regardless of their IAT performance.

The present study differs from Morris and Ashburn-Nerdo’s (2010) study by focusing on cognitive factors rather than students’ affective responses to taking the IAT. In addition, we examined how students’ actual IAT scores related to their perceptions of the test’s validity. We are interested in students’ reasons for “explaining away” their negative feedback on the IAT (Bennett et al., 2011; Frazier et al., 2011). To our knowledge, this research is the first published work to report students’ criticisms of the IAT’s validity when using the tool in an instructional setting.

Providing students with a learning experience that can reveal their unconscious biases is an important first step in raising awareness about the persistence of racism, sexism, and other “isms” in modern societies (Jones et al., 2009; Whitley & Kite, 2010). Because students find it easy to “explain away” their negative feedback on the IAT (Bennett et al., 2011; Frazier et al., 2011), class discussion or a related assignment is necessary to further advance students’ knowledge of the IAT methodology and theories on implicit and modern prejudices. Once students have this additional knowledge, they may be more willing to accept that they harbor implicit biases, which is a necessary first step in eradicating prejudice (Jones et al., 2009) and training culturally competent, global psychology citizens (American Psychological Association [APA], 2007, 2011).

The Current Study

The current study examined students’ reactions to bringing their unconscious attitudes into consciousness through a course assignment designed to raise awareness about implicit biases. The objective of this study was to examine variation and changes in students’ reactions to IAT feedback and to assess changes in students’ acceptance of the IAT after reading the empirical literature. This study differs from previous research by providing students an educational component after taking the IAT and using a postassessment to measure changes in reactions. Furthermore, the assignment allowed students to report their reasons regarding why they believed the IAT was valid or invalid and also allowed them to demonstrate their knowledge of implicit attitudes. We hypothesized the following: (a) Before completing the assigned readings, students will not accept any implicit biases shown by the IAT; (b) students with higher implicit bias will be less accepting of the IAT’s validity than students with lower implicit bias (Frazier et al., 2011); (c) explanations for students’ performance will reflect well-documented criticisms of the IAT (Bennett et al., 2011; Frazier et al., 2011; Karpinsky & Hilton, 2001); (d) after completing the assigned readings, students will be more accepting of their own implicit biases and the IAT as a tool for measuring implicit biases; and (e) the assignment will effectively improve students’ knowledge of implicit attitudes.

Method

Participants

Participants (N = 68, 48 women, 20 men) were students in a social psychology course at a private liberal arts college (26%, n = 18) or a public comprehensive university (74%, n = 50) who completed the assignment as part of their course requirement. We used participants’ names to match their pre–post essays and then assigned random identification (ID) numbers for the coding stage. Participants were White (44%, n = 30), Asian American (24%, n = 16), Latino (22%, n = 15), or identified as other (10%, n = 7).

Materials and Procedure

Students took an online version of the IAT (IAT Corp., 2011) and chose any IAT (e.g., racism, sexism), although the majority took a race IAT (69%, n = 47). A subset of the sample (n = 12) took the race IAT in a campus computer lab, thus raw IAT scores were available. After taking the IAT, students wrote a reflection essay in response to six open-ended questions modified from Thompson (2003):

Which test did you take?

Describe the rationale behind the test.

In your own words, what is an implicit attitude?

Do you think that the test accurately assessed your implicit attitudes on the topic? Why or why not?

Does the IAT measure prejudice? Why or why not?

If the IAT does not measure prejudice, what does it measure?

After writing their responses, students read the assigned course readings, which included empirical research articles (Fazio & Olson, 2003; Greenwald et al., 1998) on the methodology and validity of the IAT and a textbook chapter discussing implicit prejudice (Aronson, Wilson, & Akert, 2007). The purpose of these readings was to give students more insight into the nature of the IAT with empirical data that both questioned (Fazio & Olson, 2003) and supported its validity (Greenwald et al., 1998).

After completing the readings, students wrote an additional reaction essay discussing their IAT feedback in light of this new knowledge. Students responded to the following essay prompts:

In your own words, what is an implicit attitude?

Now that you’ve read more about the IAT, do you believe the IAT measures prejudice? Why or why not?

If the IAT does not measure prejudice, what does it measure?

What are your reactions to your test performance now that you have completed the readings?

Did the readings change your opinion about the IAT? Why or why not?

The professor graded each pre and postessay for compliance with the assignment. Although the professor knew the students’ identities, demand characteristics do not seem to provide a likely explanation for the results. Variation in responses indicated there was no clear “right” answer and subsequent work showed similar results using a confidential, online format (Casad & Flores, 2013). To determine whether students’ knowledge about implicit attitudes changed after completing the course readings, the professor graded the question “What is an implicit attitude?” asked in both the pre and postessay prompts, on a scale from 1 (poor) to 5 (excellent).

Two research assistants read all essays and grouped similar responses into themes using guidelines established by the content analysis technique (Creswell, 2007). Essays were content coded by two research assistants for common themes with an overall interrater agreement of 91%. Consistent with standard content analysis strategies, disagreements in coding were discussed, and the final codes were determined by mutual agreement, with a resulting κ = .80 (Lombard, Snyder-Duch, & Bracken, 2002). Although we computed the overall interrater agreement of 91% for initial codes prior to discussion, Kappa does not reflect independent judgments. However, both coders were blind to hypotheses.

Results

Students’ Initial Reactions to Their IAT Performance

First, we examined whether students believed the IAT accurately assessed their implicit attitudes. The most common response was the IAT was not accurate (46%, n = 28), whereas 33% (n = 20) of students said that the IAT was accurate, and 21% (n = 13) answered “unsure.” Seven students failed to answer this question, which we attributed to their oversight insofar as answering each part of the essay prompt rather than to their uncertainty. The proportions of responses approached statistical significance due to the number of respondents stating “yes” or “no” (n = 48) compared with “unsure” (n = 13), χ²(2, N = 61) = 5.54, p = .063, ν = .301; however, if we had included the seven missing responses in the “unsure” category, results of this analysis would not approach significance, p = .492. There was no difference in the number of “yes” versus “no” responses. When limiting the analysis to only participants who took the race IAT, the difference between “yes” (36%, n = 12) and “no” (64%, n = 21) responses approached significance, χ²(1, N = 33) = 2.46, p = .11, ν = .273.

We examined the reasons students gave for the accuracy or inaccuracy of the IAT, which indicated four major themes (see Table 1 for more details). Of the 20 participants who believed the IAT was accurate, 75% (n = 15) explained that their results were congruent with their explicit attitudes. Of the 28 participants who believed the IAT was inaccurate, 54% (n = 15) indicated their results were incongruent with their explicit attitudes, 43% (n = 12) reported the result was incongruent with their behavior, and 54% (n = 15) blamed the structure of the test as the source of error. Percentages total more than 100% because some responses had multiple codes.

Table 1.

Most Common Codes Used for Pre and Postessays.

Code	Definition	Example
Attitude congruence	Student stated that their implicit attitudes were congruent with their explicit attitudes	I do think the test is accurate in assessing my implicit attitudes on the topic because it is congruent with my real belief and attitude toward Blacks
Attitude incongruence	Student stated that their implicit and explicit attitudes were incongruent, or did not match	In my case, I do not believe that the test results accurately reflected my implicit attitudes toward Black Americans. I believe that this is untrue. I believe that, on the whole, I am relatively race neutral
Behavior incongruence	Student stated that their implicit attitudes did not match their explicit behavior	The [IAT] did not accurately assess my attitude toward homosexual individuals. The test revealed that I have a strong preference for heterosexuals … I do not believe this is true because the majority of the people I associate with are gay
Structure of the test is problematic	Student stated that the structure of the IAT was flawed and/or problematic	The reason that I do not believe that the test results are accurate is because the organization and design of the way the test was presented seems to expect for certain outcomes… this is problematic.
Cognitive associations	Student believes the IAT measures cognitive associations	I believe that the IAT measures the strength of association between a concept and an attribute conditioned into people over time, not necessarily prejudices that they personally have
Learned cultural stereotypes	Student believes the IAT measures cultural stereotypes	The IAT detects attitudes not reflective of an individual’s implicit attitude rather a cultural bias that is independent of the individual's own attitudes
Personal stereotypes	Student believes the IAT measures personal endorsement of cultural stereotypes	I still believe that the IAT does not measure prejudice, but rather the strength to which people endorse stereotypes
Reflex reaction to stimuli	Student believes the IAT merely measures reaction time to stimuli presented	I think that the [IAT] simply measures reaction time to stimuli presented on a computer
Ingroup preference	Student believes the IAT measures ingroup preference	A preference for one’s own race over another in a single instance is not enough to determine that an individual is prejudiced
Associations not prejudice	Student believes the IAT measures associations and states the IAT does not measure prejudice	I am still convinced that the IAT does not accurately measure prejudice, but instead it measures associations between two things
Snap judgments	Student believes that the IAT measures “snap judgments”	I think it measures quick judgments or simply how we judge the stimuli with different categories based on our knowledge of certain characteristics
Cultural influence	Student believes his/her performance on the IAT was influenced by culture	I am now, however, open to the possibility that performance on the IAT reflects associations in our memory that are based on societal standards
Doubts about the IAT’s validity	Student does not believe the IAT measures prejudice	I still don’t believe that the IAT can measure prejudice
Confidence in IAT’s validity	Student is confident the IAT’s ability to measure prejudice	I initially suspected that the test might be testing stereotypes but not until reading the assigned articles did I realize that the test was measuring prejudice
Acceptance of IAT results	Student feels accepting of his/her IAT performance feedback	After completing the readings, I am more accepting of my [IAT] score

Note. IAT = implicit association test.

Examples reflect actual responses given by students in the pre and postessays.

In response to asking students if the IAT measured prejudice, 36% (n = 24) of students did not believe that the IAT was a valid measure of prejudice, 26% (n = 17) believed it was, and 38% (n = 25) answered “unsure,” indicating no significant differences, p = .422.¹ We have listed the most common reasons for beliefs against the IAT’s validity in Table 2.

Table 2.

Reasons for Students’ Beliefs Against the IAT’s Validity.

	Preessay
Answer	%	n
1. Structure of the test is problematic	25	17
2. In-group preference	15	10
3. Explicit and implicit attitude incongruence	13	9
4. Measures cultural stereotypes	8	6
5. Measures associations not prejudice	6	4
6. Measures snap judgments	6	4

Note. “Preessay” represents students’ initial responses to their IAT performance before completing the assigned readings.

As hypothesized, students with IAT scores indicating greater implicit bias were less likely to report that the IAT measures prejudice than participants with IAT scores indicating less implicit bias, F(2, 9) = 4.16, p = .05, R ² = .48. A Tukey’s post hoc analysis indicated the significance difference was between the group that stated “no” (Mean IAT D score = .69, SD = .37) compared to “unsure” (Mean IAT D score = .03, SD = .30), p = .05.

Students’ Reactions to Their IAT Performance After the Readings

After students completed the readings, the most common response was agreement that the IAT is a valid measure of prejudice (45%, n = 27), followed by disbelief (33%, n = 21), and uncertainty (22%, n = 13).¹ After the readings, the distribution of responses was significantly different than it was before the readings, Friedman’s test χ²(1, N = 60) = 5.26, p = .022, ν = .296. When including the missing responses in the “unsure” category, the differences were still significant, Friedman’s test χ²(1, N = 68) = 3.85, p = .05, ν = .238. The mean rank before the readings was 1.59 (due to more no, 36% or 24, and unsure, 38% or 25, responses than yes, 26% or 17) whereas the mean rank after the readings was 1.41 (due to more yes responses, 44% or 27, than no, 34% or 21, or unsure, 21% or 13).

Similar to the before results, students with IAT scores indicating greater implicit bias were less likely to report after the readings that the IAT measures prejudice than participants with IAT scores indicating less implicit bias, F(2, 9) = 6.77, p = .023, R ² = .66, with a Tukey’s post hoc analysis showing a significant difference between the group that stated “no” (Mean IAT D score = .79, SD = .32) compared to “unsure” (Mean IAT D score = .03, SD = .30), p = .023.

Finally, we examined students’ reactions to their initial test performance after completing the readings. The most common responses to their initial test performance included attributing their performance to cultural influences (25%, n = 17), lingering doubts about the IAT’s validity (22%, n = 15), acceptance of their IAT results (13%, n = 9), and confidence that the IAT is valid (13%, n = 9; see Table 1 for code definitions and examples).

A comparison of students’ grades on the essay question “What is an implicit attitude?” indicated significant learning from the pre and postessays. Before the readings, students’ average score was 2.05 (Fair; SD = 1.21), but after the readings students’ average score was 3.21 (Good; SD = 1.35), t(41) = −5.63, p = .001, d = 0.90. Because the first version of the assignment did not include this question on the postessay, we excluded data for 18 students from this analysis.

Discussion

This study described a classroom assignment designed to raise students’ awareness about their own implicit biases and educate them about implicit attitudes. Results supported our hypotheses that initially most students did not believe the IAT was accurate, and the reasons given were consistent with our predictions, prior literature, and criticisms raised in the readings (Bennett et al., 2011; Frazier et al., 2011; Karpinsky & Hilton, 2001; Nosek, 2005, 2007). The results indicate that people may be unwilling to accept the IAT’s accuracy due to their disbelief that they hold prejudiced attitudes. Indeed, students who initially believed the IAT was accurate cited the congruity between their implicit and explicit attitudes, whereas students who thought the IAT was inaccurate cited the incongruity between their implicit and explicit attitudes. Knowledge of the IAT’s methodological and theoretical underpinnings changed students’ views toward the IAT. After the readings, the proportion of students supporting the validity of the IAT was significantly greater than those who disagreed. In addition, students’ knowledge about implicit attitudes significantly improved from pre to postessay.

Undoubtedly, part of the resistance in accepting the IAT as a valid measure of implicit attitudes is students’ unwillingness to admit they are biased. In support of this conclusion, students whose IAT indicated they had strong implicit bias (D scores ± .65) rejected the validity of the IAT as a measure of prejudice. Interestingly, students who stated they were unsure whether the IAT measured prejudice had IAT scores that indicated no automatic preference for one group compared with another (D scores between ± .15). Thus, without a definitive bias reported by the IAT, students may not have been certain about the test. These findings give important insight into the theoretical understanding and practical application of using the IAT as an unconsciousness-raising tool in the classroom. By understanding what factors students will use to “explain away” their negative feedback on the IAT, educators can help students recognize their own capacity for bias, and perhaps students will be more willing to accept that implicit biases can influence their behavior.

This activity is applicable to many domains within psychology including social, personality, cognitive, and introductory classes. The assignment is modifiable to fit curricular needs, such as substituting a lecture on the IAT literature rather than assigning readings (see online Appendix for assignment details, http://www.csupomona.edu/~bjcasad/IAT). In conclusion, raising students’ awareness about their own implicit biases is an important first step in developing culturally competent, global citizens, which is one goal of a liberal arts education in psychology (APA, 2007, 2011). Incorporating the IAT into a classroom assignment can advance this purpose.

Footnotes

Acknowledgments

The authors thank Mary E. Kite for her helpful comments on an earlier version of this article.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

Note

References

American Psychological Association. (2007). APA guidelines for the undergraduate psychology major. Retrieved from http://www.apa.org/ed/precollege/about/psymajor-guidelines.pdf

American Psychological Association. (2011). Principles for quality undergraduate education in psychology. Retrieved from http://www.apa.org/education/undergrad/principles-undergrad.pdf

Aronson

Wilson

T. D.

Akert

R. M.

(2007). Social psychology (6th ed.). Upper Saddle River, NJ: Pearson Education.

Bennett

C. E.

Handy

J. L.

Guenther

M. Z.

Nolan

J. M.

(2011, January). Indifference to race-IAT feedback among implicit racists. Poster presented at the 12th annual meeting of the Society for Personality and Social Psychology, San Antonio, TX.

Blatz

C. W.

Ross

(2009). Principled ideology or racism: Why do modern racists oppose race-based social justice programs? Journal of Experimental Social Psychology, 45, 258–261. doi:10.1016/j.jesp.2008.08.008

Casad

B. J.

Flores

A. J.

(2013). Confronting racism: Students’ reactions to conflicting explicit and implicit attitudes. Manuscript in preparation.

Creswell

J. W.

(2007). Qualitative inquiry and research design: Choosing among five traditions (2nd ed.). Thousand Oaks, CA: Sage.

Dovidio

J. F.

Gaertner

S. L.

(2004). Aversive racism. In Zanna

M. P.

(Ed.), Advances in experimental social psychology (pp. 1–52). San Diego, CA: Academic Press.

Fazio

R. H.

Olson

M. A.

(2003). Implicit measures in social cognition research: Their meaning and use. Annual Review of Psychology, 54, 297–327. doi:10.1146/annurev.psych.54.101601.145225

10.

Frazier

R. S.

Nosek

B. A.

Banaji

M. R.

Greenwald

A. G.

(2011, January). Participants’ responses to completing the race implicit association test. Poster presented at the 12th annual meeting of the Society for Personality and Social Psychology, San Antonio, TX.

11.

Glick

Fiske

S. T.

(2001). An ambivalent alliance: Hostile and benevolent sexism as complementary justifications for gender inequality. American Psychologist, 56, 109–118. doi:10.1037//0003-066X.56.2.109

12.

Greenwald

A. G.

McGhee

D. E.

Schwartz

J. L. K.

(1998). Measuring individual differences in implicit cognition: The implicit association test. Journal of Personality and Social Psychology, 74, 1464–1480. doi:10.1037/0022-3514.74.6.1464

13.

IAT Corp. (2011). Project implicit website. Retrieved from https://implicit.harvard.edu/implicit/

14.

Jones

J. M.

Engelman

Turner

C. E.

Jr. Campbell

(2009). Worlds apart: The universality of racism leads to divergent social realities. In Demoulin

Leyens

J. P.

Dovidio

J. F.

(Eds.), Intergroup misunderstandings: Impact of divergent social realities (pp. 117–133). New York, NY: Psychology Press.

15.

Jost

J. T.

Rudman

L. A.

Blair

I. V.

Carney

D. R.

Dasgupta

Glaser

Hardin

C. D.

(2009). The existence of implicit bias is beyond reasonable doubt: A refutation of ideological and methodological objections and executive summary of ten studies that no manager should ignore. Research in Organizational Behavior, 29, 39–69. doi:10.1016/j.riob.2009.10.001

16.

Karpinsky

Hilton

J. L.

(2001). Attitudes and the implicit association test. Journal of Personality and Social Psychology, 81, 774–788. doi:10.1037//0022-3514.81.5.774

17.

Lombard

Snyder-Duch

Bracken

C. C.

(2002). Content analysis in mass communication: Assessment and reporting of intercoder reliability. Human Communication Research, 28, 587–604. doi:10.1093/hcr/28.4.587

18.

McConahay

J. B.

Hough

J. C.

Jr. (2010). Symbolic racism. Social Issues, 32, 23–45. doi:10.1111/j.1540-4560.1976.tb02493.x

19.

Morris

K. A.

Ashburn-Nardo

(2010). The implicit association test as a class assignment: Student affective and attitudinal reactions. Teaching of Psychology, 37, 63–68. doi:10.1080/00986280903426019

20.

Nosek

B. A.

(2005). Moderators of the relationship between implicit and explicit evaluation. Journal of Experimental Psychology: General, 134, 565–584. doi:10.1037/0096-3445.134.4.565

21.

Nosek

B. A.

(2007). Implicit–explicit relations. Current Directions in Psychological Science, 16, 65–69. doi:10.1111/j.1467-8721.2007.00477.x

22.

Penner

L. A.

Dovidio

J. F.

West

T. V.

Gaertner

S. L.

Albrecht

T. L.

Dailey

R. K.

Markova

(2010). Aversive racism and medical interactions with Black patients: A field study. Journal of Experimental Social Psychology, 46, 436–440. doi:10.1016/j.jesp.2009.11.004

23.

Thompson

(2003). Course materials for PSYC 180 seminar in social cognition. Claremont, CA: Pomona College.

24.

Whitley

B. E.

Kite

M. E.

(2010). The psychology of prejudice and discrimination (2nd ed.). Belmont, CA: Wadsworth.