Using Video to Increase Gender Bias Literacy Toward Women in Science

The Importance of Increasing Bias Literacy in the Sciences

The underrepresentation of women across many STEM disciplines has persisted in the United States at all levels of education and in the STEM workforce (National Science Foundation, 2013). Of importance, this disparity is not a result of women’s lack of talent in STEM (see Lindberg, Hyde, Petersen, & Linn, 2010). Rather, women may opt out of STEM because they believe these fields are not well aligned with their career goals (e.g., to help people) or will be difficult to balance with motherhood (Ceci & Williams, 2011; Ceci, Williams, & Barnett, 2009; Diekman, Clark, Johnston, Brown, & Steinberg, 2011). In addition, widespread gender stereotype-based biases may create an unwelcoming environment for women in STEM, which may then cause women to either avoid or abandon STEM careers altogether (Cheryan, 2012).

The stereotypes associated with women in science may reflect basic processes that produce different expectations of men and women generally. According to social role theory (Eagly & Wood, 2012), because historically men have engaged in paid employment, while women have engaged in more domestic work (Shelton, 1992), people perceive men as being agentic (e.g., assertive, ambitious, and competitive) and women as being communal (e.g., warm, supportive, and nice; Eagly, 1987; Eagly & Wood, 2012). As a result of men being overrepresented in STEM fields, people assume successful scientists possess stereotypically masculine traits (Diekman, Brown, Johnston, & Clark, 2010; Nosek, Banaji, & Greenwald, 2002; Nosek et al., 2007). For example, when asked to identify the career goals of scientists and engineers, participants were more likely to select agentic goals (power, achievement) than communal goals (intimacy, affiliation; Diekman et al., 2010). This stereotypical representation of scientists as male may ultimately cause gatekeepers (e.g., professors in STEM classes) to systematically view and treat women less favorably than men, which further perpetuates men’s dominance in STEM (Cejka & Eagly, 1999; Glick, Wilk, & Perraeault, 1995). For example, researchers have found evidence of science faculty’s bias against female students (Moss-Racusin et al., 2012; Milkman et al., 2015) and women faculty in STEM (Bilimoria & Liang, 2013; Renzulli, Grant, & Kathuria, 2006; Wright et al., 2003; cf. Williams & Ceci, 2015).

This gender discrimination and bias can manifest in subtle ways (e.g., commenting that women are more passive or nice, less committed to their careers, and are in need of more help generally; Glick & Fiske, 1996, 2001; Rudman & Glick, 2008; Swim, Hyers, Cohen, & Ferguson, 2001). For example, one workplace study found that although men and women both reported desiring challenging experiences at work (i.e., experiences that tested their limits but also promoted growth), managers were less likely to assign challenging tasks to women than to men (King et al., 2012). Although these behaviors may appear innocuous, they can still create a hostile work environment for women in STEM and negatively affect women’s well-being (Cortina, 2008). Furthermore, because this unfair treatment often is difficult to detect, individuals ultimately may not recognize that these biases are impeding women’s success in the sciences. Thus, increasing awareness of gender bias and stereotypes is critical for addressing the gender disparity in STEM.

With a few notable exceptions (e.g., Becker & Swim, 2011; Carnes et al., 2015; Case, 2007; Cundiff, Zawadzki, Danube, & Shields, 2014; Zawadzki, Danube, & Shields, 2012), interventions aimed at increasing awareness of gender bias remain infrequent, and theoretically grounded diversity interventions designed to address the gender disparity in STEM are particularly rare (see Moss-Racusin et al., 2014, for an in-depth discussion of the paucity of theoretically grounded randomized controlled trials of bias reduction interventions in STEM). In addition, recent work suggests that many well-intentioned, but not empirically validated, “diversity” programs may not be effective at reducing bias (Kalev, Kelly, & Dobbin, 2006; Paluck, 2006).

Interventions designed to enhance the fundamental components of bias literacy may be an effective way to address gender bias in STEM (Carnes et al., 2012). The first two components of bias literacy are increased awareness of gender bias in the sciences and general knowledge about the gender inequity in society (Zawadzki et al., 2012) and are highly interrelated. Both constructs are rooted in the idea that sexism often manifests in very subtle ways and that making explicit communication about the nonobvious aspects of gender biases is a crucial precursor to motivating change. Individuals will not engage in positive change, however, if they lack the belief that they have the ability to change their behavior (i.e., to become better at detecting and noticing gender bias; Bandura, 1977; Carnes et al., 2015). As a result, bias literacy is also characterized by feelings of self-efficacy to positively change and recognize gender bias (Carnes et al., 2015).

Finally, and perhaps most important, bias literacy should promote behaviors that help mitigate bias in science. In particular, recognizing and intending to confront bias by planning to engage in behaviors such as speaking up and pointing out when a behavior or a comment is unfair can help others become aware of bias and can aid in preventing gender discrimination during everyday interactions (Ashburn-Nardo, Blanchar, Petersson, Morris, & Goodwin, 2014; Ashburn-Nardo, Morris, & Goodwin, 2008; Czopp & Ashburn-Nardo, 2012). Confronting individuals who are acting in a biased manner reduces future expressions of bias (Czopp, Monteith, & Mark, 2006) because it tends to evoke feelings of guilt from those engaging in such behavior (Czopp & Monteith, 2003).

Recently, researchers have created a limited number of new interventions to positively affect gender bias literacy. For example, Carnes and colleagues (2015) developed an interactive workshop on gender bias for faculty at a large Midwestern university. Compared to participants who did not take part in the workshop, the workshop increased knowledge of gender bias, self-efficacy in combating bias, and changes in behavior (Carnes et al., 2015). Another successful intervention relied on experiential learning to promote increased awareness and knowledge of gender bias in the workplace (Zawadzki et al., 2012). During this workshop, participants played an interactive game called Workshop Activity for Gender Equity Simulation (WAGES). While playing the game, participants were indirectly exposed to common difficulties women face in the workplace. Relative to control conditions, participants who played WAGES felt greater self-efficacy, showed increased knowledge of gender inequity, and perceived sexism as harmful (Cundiff et al., 2014; Zawadzki et al., 2012).

These past workshops provide useful examples of successful trainings and demonstrate how newly developed interventions should aim to increase bias literacy. However, both the interactive workshop and WAGES required not only the presence of trained facilitators to administer them, but also a substantial time commitment on the part of participants, as they were required to be physically present during the training. Thus, one goal of the current research was to develop a brief intervention that could easily be administered from virtually anywhere (e.g., online) and did not necessarily require the physical presence of a trained facilitator. Specifically, we tested the efficacy of videos at increasing bias literacy; the videos were inspired by research in both psychology and communications (Green & Brock, 2000; Pornpitakpan, 2004; Slater & Rouner, 1996).

Using Different Forms of Media to Increase Bias Literacy

If carefully constructed, media can create positive beliefs about various social groups. For example, in a field experiment in Rwanda, Paluck (2009) found a radio soap opera designed to promote reconciliation between the Tutsi and Hutu increased intergroup trust and cooperation. Other successful media diversity interventions have increased positive beliefs about stigmatized groups using storybooks (Cameron et al., 2011), television (Schiappa et al., 2005), and film (Riggle et al., 1996).

Media may influence perceptions and beliefs both indirectly (by encouraging people to care about the characters and the story; Green & Brock, 2000; Slater & Rouner, 2002) and directly (by providing information and strong facts; Petty & Briñol, 2010; Petty & Cacioppo, 1986). We created a set of high-quality videos to increase bias literacy based on both of these routes to persuasion and behavior change (direct and indirect). The first set of videos (the narratives) employed entertaining, emotionally evocative stories to illustrate results of social science research revealing gender bias in the sciences. The second set of videos (the expert interviews) portrayed a psychology professor (as a credible expert) describing the same empirical findings shown in the narratives using a straightforward, fact-based approach.

Mechanism underlying media’s effectiveness

Although we predicted that both the narratives and expert interviews would increase all components of bias literacy, compared to control videos, we believed that the narratives and expert interviews would be efficacious via different mechanisms. For example, narratives promote changes in attitudes and beliefs by being transporting (i.e., immersing and engaging viewers into the story) and promoting identification with the characters (Cohen, 2001; Green & Brock, 2000; Johnson, Jasper, Griffin, & Huffman, 2013; Slater & Rouner, 2002). Thus, narratives invest and engage people in the message or story by being transporting and emotionally evocative, but do not cause people to critically analyze the quality of the argument (Prentice & Gerrig, 1999; see also Green, Garst, & Brock, 2004; Green, Garst, Brock, & Chung, 2006). Past research found narratives were particularly effective for creating positive attitudes toward stigmatized social groups (e.g., lesbian and gay people, African Americans) for individuals who rated high on a trait measure of “transportability” (i.e., who were dispositionally prone to feeling transported upon reading a story; Mazzocco, Green, Sasota, & Jones, 2010). Moreover, increasing identification with characters in visual media, who are members of a stigmatized group, resulted in more positive evaluations of the social group as a whole (Chung & Slater, 2013). As a result, we anticipated the narratives would function as efficacious interventions by transporting participants and stimulating identification with the characters.

In contrast, expert interviews are persuasive because they stimulate careful and logical analysis of the issue presented by providing factual information from a credible source (Petty & Briñol, 2010; Petty & Cacioppo, 1986; Pornpitakpan, 2004; Slater & Rouner, 1996; Tormala, Briñol, & Petty, 2007). An expert presenting a strong argument is better at changing people’s beliefs when compared to an expert using a weak argument or a nonexpert presenting a strong argument (Pornpitakpan, 2004). In addition, when individuals believe the source is credible, they perceive the information presented in the message as more valid and feel more confident in their thoughts about the message (Briñol, Petty, & Tormala, 2004; Kaufman, Stasson, & Hart, 1999). For example, a persuasive message with strong arguments (e.g., immigrants help stimulate the economy) created lasting bias reduction toward immigrants among Madrid college students (Cárdaba, Briñol, Horcajo, & Petty, 2014).

Although we predicted that both the narratives and expert interviews would increase general knowledge of gender inequity, we anticipated that, relative to control and narrative videos, the expert interviews would encourage perceptions of being knowledgeable about gender bias facts because the expert interviews rely on a credible source (Kaufman et al., 1999). Previous research has found small correlations between perceived and actual knowledge (e.g., Glenberg, Wilkinson, & Epstein, 1982; Krosnick, Boninger, Chuang, Berent, & Carnot, 1993; Radecki & Jaccard, 1995). Research in the persuasion literature has also shown that individuals are more convinced of a persuasive message when they perceive they gained knowledge, versus when they perceived they did not gain knowledge, regardless of actual information learned from the message (Tormala & Petty, 2007). Likewise, attitudes and beliefs are more predictive of behavior when individuals believe they are knowledgeable about a topic (Davidson, Yantis, Norwood, & Montano, 1985). Thus, we predicted that the expert interviews would be effective interventions by presenting clear and logical arguments and by increasing participants’ perceived knowledge of gender bias facts.

The Current Research

In the current research, we examined whether our experimental videos would positively affect multiple aspects of bias literacy and whether different characteristics of the narratives versus expert interviews would function as important mediators for greater bias literacy. In Experiment 1, we tested whether both experimental videos increased awareness of gender bias in science and included important manipulation checks to make sure the narratives and expert interviews were perceived as intended (i.e., the narratives were seen as more entertaining than the expert interviews and the expert interviews were viewed as more informative than the narratives).

In Experiment 2, we more fully explored the efficacy of both experimental videos as diversity interventions to increase bias literacy. We again tested whether, as expected, the videos would increase awareness of gender bias in science. We also assessed the experimental videos’ capacity to increase knowledge about gender inequity, self-efficacy beliefs in ability to notice gender bias, and the likelihood of recognizing and intending to confront subtle bias across various everyday situations. We predicted that both videos would positively affect all components of bias literacy. Our first hypothesis (Hypothesis 1) was: Compared to the control condition, the experimental videos will increase bias literacy. Specifically, we predicted the videos would increase awareness of gender bias in the sciences, general knowledge about gender inequity, feelings of self-efficacy in noticing gender bias, and recognition and intentions to confront subtle bias across everyday situations.

To illuminate the dynamics underscoring the narratives versus expert interviews videos, in Experiment 2, we examined the different mechanisms by which the narratives and expert interviews promoted bias literacy. We explored whether the narrative videos would increase transportation and identification with characters, compared to the expert interview and control conditions, and whether the expert interview condition would produce more logical thinking and perceived knowledge of gender bias facts relative to the narrative and control conditions. We also tested whether these measures functioned as mediators of greater bias literacy. We predicted systematic differences would emerge between the two sets of videos on transportation, identification with characters, logical thinking, and perceived knowledge of gender bias facts.

Our second hypothesis (Hypothesis 2a) was: The narrative condition will produce higher levels of transportation and identification relative to the expert interview and control conditions, whereas the expert interview condition will result in more logical thinking about a clear strong message and increased perceived knowledge about gender bias facts relative to the narrative and control conditions.

We also predicted (Hypothesis 2b): Transportation and identification will both function as independent mediators for differences between the narrative and control condition on bias literacy outcome measures, whereas logical thinking and perceptions of being knowledgeable about gender bias facts will function as independent mediators for differences between the expert interview condition and control condition on bias literacy outcome measures.

Experiment 1

The primary goal of Experiment 1 was to initially validate our new diversity intervention videos. We presented participants with narrative, expert interview, or control videos and investigated whether the narrative and expert interview conditions increased awareness of gender bias in science relative to the control condition (Hypothesis 1). We also tested whether the narratives would be viewed as more entertaining than the expert interviews and the expert interviews would be seen as more informative than the narratives.

Method

Participants

Participants were recruited from Amazon’s Mechanical Turk website and completed the experiment for US$0.75 payment. Participants recruited through Mechanical Turk are an older and more ethnically diverse population than traditional university samples (Behrend, Sharek, Meade, & Wiebe, 2011; Buhrmester, Kwang, & Gosling, 2011). Our final sample consisted of 506 U.S. residents, who were mostly educated and majority White: 53% (267) women; average age 35.70, range 18–76, SD = 12.62; 8.5% (42) worked in STEM; 0.8% (4) had completed less than high school education, 33.4% (169) high school degree/General Educational Development (GED) degree, 21.5% (109) 2-year college degree, 33% (165) 4-year college degree, 7.9% (40) master’s degree, 1.8% (9) doctorate degree, 1.4% (7) professional degree; 76% (383) White, 8.7% (44) African American, 4.9% (25) Latino, 3.0% (15) East Asian, 1.8% (9) South Asian, 0.8% (4) Southeast Asian, 0.2% (1) Native American/Pacific Islander, 3.2% (16) Multiracial, 1.2% (6) Other; political orientation (measured on a 1 = extremely liberal to 7 = extremely conservative scale) M = 3.43, SD = 1.55.

Development of Videos

Our goal was to create high-quality media that effectively demonstrated empirical evidence about various instantiations of gender bias and stereotypes. Consequently, we created six short clips (average length = 5 min, 10 s), representing different elements and examples of gender bias. The videos we developed were comparable in length to those that commonly appear on popular video sharing platforms (e.g., YouTube). Furthermore, having multiple short videos allows future diversity intervention leaders to select the most relevant videos for their workshop or training.

The coauthors of this article (whose academic expertise lies in the identification and reduction of gender biases) independently generated an initial list of 10 articles that each believed were critical in explicating gender bias and stereotypes in STEM. There was substantial overlap in the authors’ selection of articles, and the authors met and discussed any discrepancies. We ultimately selected six articles that all agreed dealt with common types of gender bias experienced in STEM environments and also portrayed a range of subtle to more overt instances of bias directed toward different groups (e.g., young women, mothers, men; see Table 1 for a list of these research papers).

OPEN IN VIEWER

Table 1.

Brief Descriptions of Narrative and Expert Interview Scenes and Bias Portrayed.

Scene	Target Article	Major Point of Article (Described in Expert Interview Condition)	Brief Description of Bias in Scene (Shown in Narrative Condition)
1.	Madera, Hebl, and Martin (2009)	Letter writers use more communal words (e.g., nice) to describe women and more assertive words (e.g., competent) to describe men.	Two male professors think a man is the stronger applicant (another female professor does not agree) because they do not think the words the female applicant’s letter writers used to describe her (e.g., nice) make for a strong candidate.
2.	Rudman and Glick (1999)	Women are seen as more competent when they act assertively but are less likable. When men are assertive, they are not rated as less likable.	When a female graduate student tries to act more assertively, she is criticized for this behavior. A male graduate student is not criticized for being assertive by two male professors and one female professor.
3.	Murphy, Steele, and Gross (2007)	Women in STEM feel more threatened and less sense of belonging when they watch a video in which women are underrepresented at a conference relative to one in which there is equal gender representation.	A male professor is not concerned when he goes into the men’s bathroom with two male graduate students while explaining a research technique, leaving the only woman in the group trying to learn from him out in the hallway.
4.	Moss-Racusin, Dovidio, Brescoll, Graham, and Handelsman (2012)	Both male and female science faculty members show a clear bias in favor a male applicant over a female (viewing him as more competent, hirable, and deserving of more mentoring and higher pay).	A male professor does not think a female graduate student is competent enough to help with a complicated microscope.
5.	Moss-Racusin, Phelan, and Rudman (2010)	Men are viewed less favorably when they behave modestly and not assertively.	A male professor gets upset with a male graduate student for behaving modestly and failing to self-promote. This male professor then discusses revoking the male student’s travel award with another male professor.
6.	Heilman and Okimoto (2008)	Women applying for a promotion were seen as less competent when they were mothers, relative to their male counterparts.	A female professor thinks a pregnant job candidate will not be dedicated to her work after having her child, even though the candidate asserts that her husband will do most of the childcare. A female graduate student is unsettled upon learning that the professor did not hire the candidate because of the pregnancy.

Note. STEM = science, mathematics, technology, and engineering.

Guided by the research in the six selected articles, we then created a script for two different sets of scenes with different presentational styles—the six narrative scenes and the six expert interview scenes. The narratives were entertaining stories illustrating the empirical results of selected published papers on gender bias and stereotypes (see Table 1 for a complete list of the narratives) and were written by a professional playwright who was instructed to make sure that the script was entertaining, emotionally engaging, and transporting. Prior to writing the script, the playwright read the six psychological research articles about gender bias and stereotypes and discussed these articles with us over many meetings to ensure adequate comprehension of the research. To guarantee that the scenes accurately portrayed everyday experiences in academic science, the playwright also interviewed men and women graduate students, postdoctoral associates, and professors in the sciences. The expert interviews described the same psychological research displayed in the narrative films, but in a straightforward, fact-based manner during an interview with a psychology professor (the expert). Because the main goal was to present facts about gender bias, we wrote these scenes. However, we received feedback from the playwright to help create naturally flowing interactions between the interviewer and the professor.

Once we had final drafts of both the six narrative and six expert interview scenes, we held formal read throughs of the script (i.e., volunteers read the lines for each of the characters in the script across all six scenes), with 15 academic natural scientists and psychologists in the audience. We made adjustments to the scripts based on the feedback from the audience members. We also requested feedback on the filmed read throughs from social psychologists (graduate students, postdoctoral students, and professors at variety of institutions), to make sure that each of the six scenes was perceived as demonstrating the intended bias. We received responses from 14 social psychologists (57% [8] female), and on a 1 = completely disagree to 7 = completely agree scale, these 14 pilot participants were above the midpoint in agreeing that the videos demonstrated the prevalence of stereotypes (M = 5.45, SD = 0.99), that bias can be unintentional (M = 5.72, SD = 0.84), and that science can be unwelcoming toward women (M = 5.61, SD = 0.90). We also sent the filmed read through of the expert interviews to nonpsychologists (postdoctoral and graduate students in the natural and physical sciences) to test whether these interviews discussed psychological research in accessible terms. We received seven responses (28% [2] female), and due to feedback that the expert interview condition was difficult to follow at specific points, we adjusted the language to make it more accessible to a general audience. (In total, the script development for the scenes took approximately 14 months.)

Once the scripts were finalized, we hired a professional production crew (including a director, producer, director of photography, and actors) to film all 12 scenes. The first author and a natural scientist consultant were present at the initial rehearsal with actors and the filming to ensure that the videos correctly displayed psychological and scientific research. The final narrative videos ultimately featured professional female and male actors playing science professors, graduate students, and laboratory technicians interacting in science laboratories and classrooms (see Table 1 for more specific information about the stories depicted in the videos). The final expert interview videos highlighted two male professional actors who portrayed a psychology professor (“Dr. James Alder” and “Dr. Jerry Thompson”) describing research on gender bias in the sciences during a formal interview. Each individual actor played a psychology professor in three of the six interview videos. The “experts” or professors were always men because men are perceived as more authoritative, particularly when discussing issues related to gender bias (Abel & Meltzer, 2007). Our goal in casting was to guarantee the expert interviews were as persuasive as possible and not discounted by viewers. However, to create gender representations in the expert interview videos, a professional actress played the role of the interviewer. Finally, the expert interviews were situated in a science laboratory environment to hold the science environment constant across the narrative and expert interview videos. In total, we created 12 experimental videos ranging between 4 and 6 min in length (average length = 5 min, 10 s). All 12 videos are available at https://www.youtube.com/watch?v=O7_KFGJl0JY&list=PLuA42xbR-FAzgkaPyGbjD5dlqLUpnQiyj (in a YouTube playlist), and a website featuring the videos will be available soon (information about this website will be made accessible in the descriptions of the videos on YouTube). In addition, readers are encouraged to contact the first/corresponding author of this article, if they wish to receive the data presented in this article. Participants in the narrative condition watched a single narrative video (i.e., were randomly assigned to watch one of the six narrative videos), and participants in the expert interview condition watched a single expert interview video (i.e., were randomly assigned to watch one of the six expert interview videos).

For the control videos, we used 4- to 6-min clips from existing science documentaries. At least one female and one male scientist were featured in each of these short clips in order to hold exposure to women and men in science constant between the experimental and control videos. However, the control videos had no mention of gender bias. In total, there were 12 potential control videos, and participants in the control condition each watched a single control video. We chose the 6 of the 12 control videos that best matched the narratives on being entertaining and also best matched the expert interviews on being informative. Comparisons between the narrative, expert interview, and control videos will be discussed in more detail in the Results section. The six control videos that we did not include in our analyses were judged significantly more entertaining than the narratives (p < .001), and more informative than the expert interviews (p = .001), and thus were not well matched to the experimental videos. Nevertheless, when we compared the six control videos we did not select, to the narrative and expert interviews on our outcome measures of awareness of gender bias in the sciences (see below for description of measure), we find that the narrative and expert interview videos significantly increased awareness of gender bias, t(501) = 2.93, p = .015, d = 0.26.

Attention Checks

Immediately following the videos, we asked participants three straightforward questions about the video they just watched to ensure that they had paid sufficient attention (e.g., “According to Professor Alders, what does data suggest about men and women’s performance in math and science?”) Participants were presented with four potential answers and had to choose the correct one (e.g., “a. The data are inconclusive, b. There is a very small difference between men and women’s performance, c. Women perform much better than men, d. Men perform much better than women”).

Manipulation Checks

Immediately after watching the videos, participants rated their level of agreement from 1 (strongly disagree) to 5 (strongly agree) with the statements, “This video was entertaining” and “This video was informative.”

Awareness of Gender Bias in Science

Participants rated their level of agreement (1 = strongly disagree, 5 = strongly agree) with eight statements about bias toward women in science. Three items (“R” indicates reverse scoring) assessed awareness of general bias against women in science (“In my opinion, women in science fields often do not face discrimination based on their gender [R],” “In my opinion, women in science fields often are not taken as seriously as their male colleagues,” “In my opinion, women in science fields often don’t have to work harder than their male colleagues to show they are equally as competent as the men [R]”). Three items examined recognition of backlash against women for acting aggressively (“In my opinion, women in science fields often face negative reactions for being aggressive,” “In my opinion, women in science fields often face negative reactions for being assertive,” “In my opinion, women in science fields often face negative reactions for being ambitious”). Two items tested for awareness of bias against mothers in science (“In my opinion, women in science fields often have trouble getting hired if they are pregnant,” “In my opinion, people who work in science fields often do not want to hire women because they worry that the women might become pregnant and be unable to do their job adequately [R]”). We reversed scoring of the items marked by an “R” and then averaged all the items to an index of awareness of gender bias in science (full sample: M = 3.43, SD = 0.71, α_{all participants} = .86, α_{female participants} = .85, α_{male participants} = .84).

Procedure

The experiment was advertised on Amazon’s Mechanical Turk website, and participation was restricted to only U.S. residents. The experiment was presented as an exploration of how people remember and form impressions of short videos. As previously mentioned, each participant was randomly assigned to watch one video and an average of 28 participants (range: n = 24–32) watched each video.

Prior to watching the video, all participants read short descriptive instructions. The instructions for the narrative condition indicated that the featured story was based on psychological research as well as the real experiences of men and women in the sciences. Likewise, in the expert interview and control conditions, participants were told that the research discussed in the video was based on actual experiments published in the psychological or the natural sciences.

Immediately after watching the video, participants answered the attention check questions (memory questions). Participants then completed manipulation checks (i.e., indicating how entertaining/informative the videos were), followed by the measure of awareness of gender bias in science. All outcome measures were administered after the videos. Finally, participants reported their demographic information. Participants were fully debriefed, thanked for their participation, and compensated. (For detailed analyses with demographic variables, see the online Supplemental Materials and Table S3.)

Results and Discussion

Preliminary Analyses

We initially tested (using between-subjects analyses of variance [ANOVAs] and Tukey’s post hoc tests) whether there were differences between the six individual narrative videos and six individual expert interview videos on awareness of gender bias in science. We did not find any notable variation within each set of videos, suggesting that the individual narrative and expert interview videos were equivalently effective (for narratives: all ps > .530, range of means = 3.25–3.56; for expert interviews: all ps > .461, range of means = 3.38–3.80).

We ran additional analyses to investigate any effects of the different actors playing the professor in the expert interviews videos. We found that both actors playing the professor in the expert interviews were equally liked (on a −3 = dislike very much to +3 = like very much scale), M = 1.28, SD = 1.50; M = 1.62, SD = 1.34, t(162) = 1.55, p = .12, d = 0.24.

Although the videos were all between 4 and 6 min long, there was still some variability in length. To ensure that the length of the videos did not influence our main dependent variable, we calculated the exact length of each video in seconds. In the narrative condition, r(161) = .10, p = .205, expert interview condition, r(163) = .05, p = .534, and control condition, r(179) = .08, p = .280, video length did not correlate with awareness of gender bias in science. Thus, we collapsed across the six narrative videos to create the narrative condition, the six expert interview videos to make the expert interview condition, and the six control videos to form the control condition.

Attention Checks

Fifteen participants (2.9% of the sample) did not correctly answer two of the three attention check questions about the stimulus videos. As a result, their data were excluded from the analysis (though results were unchanged when their data were included). Excluded participants did not differ significantly across condition, χ²(2, N = 506) = 2.92, p = .232. Similar memory attention checks have been used in past research to exclude participants (e.g., Green & Brock, 2000).

Manipulation Checks

Our first goal was to establish that the experimental condition videos were perceived as intended (i.e., the narrative videos were entertaining and the expert interviews were informative). To do so, we conducted one-way ANOVAs on the manipulation check variables. We ran focused orthogonal contrasts to compare the effects of the narrative condition to the expert interview condition (+1 −1 0), the narrative condition to the control condition (+1 0 −1), and the expert interview to control condition (0 +1 −1). The contrast comparing the narrative condition to the expert interview condition was significant on perceived entertainment, t(501) = 4.72, p < .001, d = 0.42, and perceived informativeness, t(580) = −10.79, p < .001, d = 0.96. As predicted, we found that participants perceived the narratives (M = 3.43, SD = 1.00) to be more entertaining than the expert interviews (M = 2.91, SD = 1.01) and the expert interviews (M = 4.18, SD = 0.78) to be more informative than the narratives (M = 3.20, SD = 1.07).

As anticipated, the contrast comparing the narrative condition to the control condition (M = 3.54, SD = 0.93) on perceived entertainment was not significant, suggesting these two sets of videos were seen as was equivalently entertaining, t(501) = −1.15, p = .252, d = 0.10. However, unexpectedly, the contrast comparing the expert interview condition to the control condition was significant, t(501) = −2.19, p = .029, d = 0.20. Participants viewed the control videos (science documentaries on topics unrelated to gender bias) as more informative than the expert interview videos. When we controlled for perceived informativeness, our results predicting awareness of gender bias did not change. These manipulation check results indicated that our newly created videos were perceived as anticipated.

Video Condition Differences for Awareness of Bias Toward Women in Science

Hypothesis 1 predicted that both experimental video conditions would increase awareness of gender bias in science relative to the control condition. To evaluate this possibility, we conducted a one-way ANOVA and ran focused orthogonal contrasts to compare the effects of both experimental video conditions to the control condition (+1 +1 −2) and the effects of the narrative condition to the expert interview condition (+1 −1 0). As expected, the contrast comparing the experimental conditions to the control condition (M = 3.30, SD = 0.71) was significant, t(501) = 3.04, p = .002, d = 0.27, whereas the contrast comparing the narrative condition (M = 3.47, SD = 0.67) to the expert interview condition, M = 3.53, SD = 0.72; t(501) = −0.72, p = .47, d = 0.06, was not.

Additional analyses including gender as a factor in the ANOVA are available in the Supplemental Materials (see Tables S4 and S5). In line with previous research (Luzzo & McWhirter, 2001), we found that women were more aware of gender bias in the sciences (p < .001), but gender did not interact with video condition to predict awareness of gender bias (p = .807). Experiment 1 provided initial validation of our newly created experimental videos and demonstrated that both the narratives and expert interviews increased awareness of gender bias in science. Furthermore, the narratives were viewed as more entertaining than the expert interviews, and the expert interviews were perceived as more informative than the narratives.

Experiment 2

In Experiment 2, our main goal was to more fully explore whether the videos functioned as diversity interventions that increase bias literacy (Hypothesis 1). To accomplish this task, we again examined the narrative and expert interview videos’ influence, relative to control videos, on awareness of gender bias science (as in Experiment 1). However, because we employed a newly developed measure of awareness of gender bias in science in Experiment 1, in Experiment 2, we utilized the Awareness of Male Privilege Scale, a previously validated measure that also captures the awareness of gender bias construct (Case, 2007). Experiment 2 also investigated whether, relative to the control videos, the narrative and expert interview videos increased general knowledge about gender inequity and self-efficacy in being able to notice gender bias. In addition, in Experiment 2, we examined whether, compared to the control videos, the experimental videos helped participants accurately recognize subtle instances of bias across various everyday situations. Specifically, participants were presented with situations that either showed evidence of gender bias or showed no evidence of gender bias.

Finally, in Experiment 2, we examined whether, compared to the control videos, the experimental videos promoted intentions to confront the perpetrator of discriminatory behavior in everyday hypothetical situations. Although intentions to confront are distinct from confrontation behavior, behavioral intentions can be predictive of future actions (see Webb & Sheeran, 2006). In addition, previous research has utilized measures of intentions to confront to assess future confrontation behavior (see Ashburn-Nardo et al., 2014). However, in the second experiment, we also wanted to investigate whether our experimental videos would encourage an actual behavioral change. Thus, we also assessed whether participants left a public comment pointing out the subtle bias toward a woman in a short video clip. This behavior is distinct from actively confronting biased individuals; however, it still represents a somewhat indirect form of confrontation (Chaney, Young, & Sanchez, 2015; Foster, 2015). For example, previous research has promoted indirect confrontation by having women publicly post a statement about sexism in society on Twitter (a social media website) and found this indirect form of confrontation had similar benefits (e.g., decreased negative affect, increased psychological well-being) for women, as those associated with directly confronting a transgressor (Foster, 2015).

An important additional goal of Experiment 2 was to illuminate the different processes underscoring the effectiveness of the narrative and expert interview videos. Thus, we investigated whether the narrative videos would result in more transportation and identification with the characters relative to the expert interview and control videos, and whether the expert interview videos would produce more logical thinking and perceived knowledge of gender bias facts than the narrative and control videos (Hypothesis 2a). We then examined whether these measures functioned as important mediators of greater bias literacy (Hypotheses 2b).

Finally, in Experiment 2, we aimed to further validate our new videos by adding new manipulation/comprehension check items. These questions examined whether participants viewed the narratives and expert interviews as both presenting evidence of gender bias and as equally believable.

Method

Participants

Participants were again recruited from Amazon’s Mechanical Turk website. This time participants were paid US$1.00 for completing the study. Our final sample consisted of 583 U.S. residents, who were mostly educated and majority White. The breakdown of the sample is the following: 56% (329) women; average age 35.70, age range 18–75, M = 35, SD = 12.31; 18.9% (110) worked in STEM; 0.3% (2) had completed less than high school education, 28.4% (165) high school degree/GED, 17.9% (104) 2-year college degree, 35.1% (204) 4-year college degree, 13.7% (80) master’s degree, 2.1% (12) doctorate degree, 2.6% (15) professional degree; 73.8% (430) White, 7.6% (44) African American, 6% (35) Latino, 3.3% (19) East Asian, 2.2% (13) South Asian, 1.2% (7) Southeast Asian, 0.3% (2) Native American/Pacific Islander, 3.8% (22) Multiracial, 1.4% (8) Other; political orientation M = 3.43, SD = 1.52.

Materials

We used the same videos as in Experiment 1 and participants again watched a single video. Participants completed the same attention and manipulation checks as in Experiment 1. We also included two new comprehension check measures, to ensure participants viewed both the narrative and expert interview videos as providing evidence of gender bias and as being equally believable. Specifically, participants rated their level of agreement (1 = strongly disagree, 5 = strongly agree) with the two statements: “This movie presented evidence about gender bias” and “This movie was believable.”

To examine awareness of gender bias in science, we utilized the same measure as that employed in Experiment 1 (full sample: M = 3.61, SD = 0.71, α_{all participants} = .87, α_{female participants} = .84, α_{male participants} =.86). We added to our assessment of the awareness of gender bias in science construct by including an established scale—a modified version of the Awareness of Male Privilege Scale (Case, 2007). Table 2 presents the means, standard deviations, αs, number of, and examples of, items for all the new measures employed in Experiment 2. To view a list of all new or modified items, see the Online Supplemental Materials (Table S1).

OPEN IN VIEWER

Table 2.

List of New Scales in Experiment 2.

Scale	M (SD)	αfull sample (αfemale, αmale)	# of Items	Example Item
Bias literacy
Awareness of male  privilege in the sciences	3.49 (0.79)	.88 (.87, .88)	7	Men have privileges that women do not have in the sciences
Knowledge of gender  equity questionnaire	3.61 (0.55)	.91 (.88, .91)	21	Women are often not acknowledged by their male colleagues for their hard work
Self-efficacy in recognizing  gender bias	3.21 (1.02)	.96 (.96, .96)	6	After watching the movie, I feel confident in my ability to recognize instances of gender bias
Recognizing bias in clip	3.12 (0.93)	.83 (.82, .83)	3	This video clip was an example of subtle (hard to notice) gender bias
Recognizing bias in  situations with bias	3.59 (0.62)	.90 (.88, .90)	4 (across five situations)	This situation is an example of gender discrimination
Recognizing bias in  situations without bias	1.63 (0.64)	.94 (.94, .93)	4 (across three situations)	This situation is an example of gender discrimination
Responsibility to confront	3.64 (0.77)	.77 (.77, .77)	1 (across five situations)	It would be my place to say something to Jason about his behavior
Intentions to confront	3.44 (0.74)	.74 (.72, .75)	1 (across five situations)	I would tell Jason he is behaving unfairly
Mediators
Transportation	3.28 (0.55)	.75 (.78, .69)	11	While watching the program, I could easily picture the events taking place
Identification scale	3.54 (0.63)	.84 (.85, .83)	7	While viewing the movie, I could feel the emotions in the characters’ head
Logical thinking	3.91 (0.68)	.86 (.86, .85)	4	This video made me think logically about the evidence it presented
Perceived knowledge  of gender bias facts	3.08 (1.11)	.96 (.95, .97)	4	This video increased my understanding of the gender bias literature

Note. All measures were on a 5-point scale (1 = strongly disagree, 5 = strongly agree).

To assess participants’ general knowledge about gender inequity, we employed the Knowledge of Gender Equity Questionnaire (21 items, M = 3.61, SD = 0.55, α_{full sample} = .91; Shields, Zawadzki, & Johnson, 2011). We also adapted a measure, closely modeled after the self-efficacy index utilized by Zawadzki, Danube, and Shields (2012; 6 items, M = 3.21, SD = 1.02, α_{full sample} = .96), to assess self-efficacy beliefs in ability to recognize gender bias (see Tables 2 and S1).

To examine recognizing and pointing out subtle bias in media, we had participants watch a short 30-s video clip that featured an example of subtle bias. We utilized a very short clip because past research has found that short presentations of subtle bias in media can transmit harmful bias and result in more negative attitudes toward stigmatized groups (Weisbuch, Pauker, & Ambady, 2009). During the clip, a female medical intern is running late for rounds and appears frazzled when she arrives. She is immediately asked a question by her male supervisor. She spends a second thinking about the answer, when a man colleague (who is trying to flirt with her) whispers the correct answer in her ear. After rounds, the woman asserts she did in fact know the answer to the question, but the man colleague clearly does not believe her and teases her. The clip is somewhat funny and lighthearted. However, the content of the clip was similar to the subtle instances of bias women report experiencing in their daily lives (Swim et al., 2001), and perpetuated the stereotype that women are in need of saving and are less competent than men.

After watching the clip, we assessed how well participants noticed the subtle bias in the clip (see Tables 2 and S1 for a full list of items). After participants completed these initial ratings, they were given an opportunity to comment on the clip (i.e., to point out the gender bias). Specifically, we told participants that other participants have been, and will be, assigned to watch this same short clip and that we gave past participants the opportunity to write a comment about the clip “similar to that done on video sharing sites, such as YouTube.” Participants then had the option of leaving a comment that other future participants would ostensibly see. Two independent coders rated the comments for mention of unfair treatment or gender bias behavior in the clip (1 = yes, 0 = no). For example, one participant commenting on the subtle bias in the clip wrote, “The exchange in the video is meant to be comedic or to illustrate an adversarial relationship that might develop into a romantic one. However, it does portray a male behavior that rests in stereotypes.” Another participant commented, “I found this to be an example of a subtle bias because he assumed she did not know the answer on her own.” There was high initial agreement between the two coders on the presence or absence of mentioning bias (κ = .93, p < .001). The two coders discussed any discrepant ratings and came to an agreement on the final code for each comment.

To examine recognizing and intending to confront subtle bias across hypothetical everyday situations, we first asked participants to imagine they worked in a science and technology company and then presented them with eight situations they could encounter in an organization or workplace. Five of these situations featured an example of subtle bias (e.g., “A coworker, Jason, mentions he is having difficulty on a project. You suggest he ask your other coworker, Samantha, for help because she’s dealt with a similar situation. Jason scoffs and says Samantha is not exactly the most skilled in the lab and then rolls his eyes and laughs”). Three of these situations had no unfair treatment (e.g., “Your coworker, Sean confides in you that he is thinking about firing his research assistant Anna. He tells you he feels like he has no choice lately. Anna has been consistently late to work and has left an hour early on multiple occasions without asking permission. Last week Anna missed work without calling to say she would not be in. Anna came in the next day and shrugged and said her allergies were acting up. The final straw was when Anna broke a very expensive piece of equipment and didn’t bother to tell Sean right away”). For the full list of situations with subtle bias and no bias, see the Online Supplemental Materials (Table S2).

Participants then indicated how much they recognized or noticed bias in each situation. We were able to assess how well participants accurately recognized subtle bias (i.e., noticed bias in the five situations with examples of sexism) and incorrectly recognized bias (i.e., perceived bias in three situations without any evidence of sexism). We also examined participants’ feelings of responsibility and intentions to confront the biased individual (these items were modeled after those used by Ashburn-Nardo et al., 2014; see Tables 2 and S1 for a full list of items).

Mediators

To examine participants’ level of transportation and identification with the characters while watching the videos, participants completed the Transportation Scale (Green & Brock, 2000) and Identification Scale (Cohen, 2001; see Table 2 for the means, standard deviations, αs, number of items, and example item). Participants also completed a measure that assessed their level of logical thinking while watching the movies. Finally, participants completed a scale examining their perceptions of having knowledge about gender bias facts, research, and literature (see Tables 2 and S1 for a full list of items).

Procedure

This experiment was advertised on Amazon’s Mechanical Turk website, was restricted to only U.S. residents, and was promoted in a similar manner as Experiment 1. As in Experiment 1, each participant was randomly assigned to watch one video with an average of 32 participants per condition (range: n = 27–39). Prior to watching the video, all participants read the same short descriptive instructions as in Experiment 1. Immediately after watching the video, participants completed the attention check questions (memory questions) and the manipulation and comprehension checks items. Participants then watched the short 30-s video clip and responded to the questions about the subtle bias in the clip and chose whether or not to leave a comment. Following this task, participants were presented in random order with the everyday situations with and without instances of subtle gender bias and responded to the questions about each situation. Next, participants completed the bias literacy main dependent variables in random order and then the mediator measures in random order. All outcome measures were administered after the videos. Finally, participants provided the same demographic information as in Experiment 1, and they were fully debriefed and compensated. (For detailed analyses with demographic variables, see the online Supplemental Materials and Table S3.)

Results and Discussion

Preliminary Analyses

As we found in Experiment 1, there were no notable differences between the six narrative and the six expert interview videos on awareness of gender bias in science (for narratives: all ps > .152, range of means = 3.41–3.86; for expert interviews: all ps > .311, range of means = 3.52–3.92). Thus, for all of our subsequent analyses, we collapsed the six narrative videos into the narrative condition, the six expert interview videos into the expert interview condition, and the six control videos into the control condition.

Attention, Manipulation, and Comprehension Checks

Thirty-seven (5.3% of the sample) participants did not correctly answer two of the three attention check questions about the stimulus videos correctly. As a result, their data were excluded from the analysis (our results did not meaningfully change when we included these participants). Excluded participants did not differ by video condition, χ²(2, N = 620) = 1.48, p = .478.

As in Experiment 1, we conducted one-way ANOVAs on the manipulation check variables. We compared the effects of the narrative condition to the expert interview condition (+1 −1 0), the effects of the narrative condition to the control condition (+1 0 −1), and the expert interview to control condition (0 +1 −1) via orthogonal contrasts. The contrast comparing the narrative to the expert interview conditions was significant on perceived entertainment, t(580) = 5.04, p < .001, d = 0.42, and perceived informativeness, t(580) = −10.19, p < .001. Replicating Experiment 1, we found that participants felt the narratives (M = 3.60, SD = 0.95) were more entertaining than the expert interviews (M = 3.09, SD = 1.04) and the expert interviews (M = 4.13, SD = 0.82) were more informative than the narratives (M = 3.29, SD = 0.99).

However, the contrast comparing the control condition to the narrative condition was significant on perceived entertainment, t(580) = −2.34, p = .017, d = 0.19, and the contrast comparing the expert interview condition to the control condition on perceived informativeness, t(580) = −3.67, p < .001, d = 0.30, was also significant. Participants viewed the control videos (M = 3.84, SD = 1.01) as more entertaining than the narrative videos and as more informative (M = 4.43, SD = 0.58) than the expert interview videos. When we controlled for level of entertainment and informativeness as a covariate in our analyses, the effect of condition remained significant.

We next examined the comprehension checks. The contrast comparing the narrative (M = 3.82, SD = 1.20) to the control condition (M = 1.71, SD = 0.97), t(580) = 20.76, p < .001, d = 1.72, and the contrast comparing the expert interview (M = 2.37, SD = 0.81) to the control condition, t(580) = 25.15, p < .001, d = 2.09, were both significant. Thus, participants perceived both the narrative and expert interview videos as providing evidence of gender bias. In addition, the contrast comparing the narrative and expert interview conditions was significant, t(580) = −4.47, p < .001, d = 0.37. Because the expert interviews relied on clear and logical facts, it makes sense that participants viewed the expert interviews as providing more evidence about gender bias than the narratives.

Second, the contrast comparing the narrative (M = 4.01, SD = 0.87) and expert interview (M = 4.01, SD = 0.88) conditions demonstrated that participants viewed both sets of videos as equally believable, t(580) = 0.00, p = .999, d = 0.00. However, the contrast comparing the narrative to control condition (M = 4.33, SD = 0.68), t(580) = −3.97, p < .001, d = 0.33, and the contrast comparing the expert interview to the control condition, t(580) = −3.95, p < .001, d = 0.33, were both significant. Participants viewed the control videos (science documentaries on topics other than gender bias) as more believable than the narratives and expert interviews. Our results did not meaningfully change when we controlled for this variable. It is also worth noting that all three sets of videos were seen as highly believable (all means above 4 on a 5-point scale).

Bias Literacy Measures

We then investigated how the narrative and expert interview videos influenced the various aspects of bias literacy. To explore the effect of video condition on our bias literacy measures, we again ran one-way ANOVAs predicting each measure of bias literacy. We compared the effects of both experimental video conditions to the control condition and the effects of the narrative condition to the expert interview condition via orthogonal contrasts (+1 +1 −2 and +1 −1 0). All the means and standard deviations for each condition are reported in Table 3. Additional analyses including gender as a factor in the ANOVA are available in the Supplemental Materials (see Tables S4 and S5). Women were higher on all the bias literacy measures than men (all ps < .014). Gender did not interact with video condition to predict any of the bias literacy outcome variables (all ps > .211). Thus, the videos were equally effective at increasing bias literacy across both men and women.

OPEN IN VIEWER

Table 3.

Means and Standard Deviations for Video Condition in Experiment 2.

Means (Standard Deviations)
	Condition
	Narrative	Expert	Control
Bias literacy
Awareness of gender bias  in science	3.65 (.75)a	3.70 (.71)a	3.48 (0.67)b
Awareness of male  privilege in science	3.51 (.80)ab	3.61 (.79)a	3.36 (0.77)b
Knowledge of gender  bias equity	3.64 (.58)a	3.70 (.52)a	3.50 (0.54)b
Self-efficacy in recognizing  gender bias	3.57 (.82)a	3.63 (.73)b	2.45 (1.00)a
Recognizing bias in clip	3.02 (.97)ab	3.19 (.91)a	2.82 (0.92)b
Recognizing bias in  situations with bias	3.51 (.64)a	3.66 (.60)b	3.50 (0.61)a
Recognizing bias in  situations without bias	1.67 (.66)a	1.62 (.62)a	1.61 (0.65)a
Responsibility to confront	3.22 (.79)a	3.35 (.74)a	3.22 (0.76)a
Intentions to confront	3.37 (.77)a	3.52 (.74)a	3.42 (0.69)a
Reactions to videos
Transportation	3.49 (.58)a	3.21 (.51)b	3.14 (0.49)b
Identification	3.84 (.58)a	3.40 (.55)b	3.36 (0.65)b
Logical thinking	3.70 (.77)a	4.00 (.59)b	4.04 (0.62)b
Perceived knowledge  of gender bias facts	3.23 (.96)a	3.76 (.80)b	2.28 (1.03)c

Note. Shared subscripts indicate no significant difference between video conditions on a Tukey’s post hoc test.

Supporting Hypothesis 1, we found the orthogonal contrast comparing the experimental video conditions to the control condition was significant for awareness of gender bias in science, t(580) = 3.20, p = .001, d = 0.27; awareness of male privilege in the sciences, t(580) = 2.98, p = .003, d = 0.25; knowledge of gender inequity, t(580) = 3.60, p < .001, d = 0.30; and self-efficacy at recognizing gender bias, t(580) = 15.31, p < .001, d = 1.27. However, the contrast comparing the narrative and expert interview condition was not significant for awareness of gender bias in science, t(580) = −0.83, p = .409, d = 0.07; awareness of male privilege in the sciences, t(580) = −1.14, p = .261, d = 0.09; knowledge of gender inequity, t(580) = −1.01, p = .314, d = 0.08; and self-efficacy at recognizing gender bias, t(580) = −0.73, p = .466, d = 0.06. Thus, compared to the control condition, the experimental videos increased awareness of gender bias and male privilege in the sciences, knowledge of gender inequity, and self-efficacy at recognizing gender bias.

Providing additional support for Hypothesis 1, for recognizing the subtle bias in the video, the orthogonal contrast comparing the experimental video conditions to the control condition was significant, t(580) = −3.42, p = .001, d = 0.28. The contrast comparing the narrative and expert interview was marginal, t(580) = −1.79, p = .075, d = 0.15, suggesting that the expert interview video helped participants recognize subtle bias in the video clip slightly better than the narrative video.

Next, we looked at the likelihood that participants would leave a comment pointing out the subtle bias in the video clip. Ninety-six participants left some sort of comment (16.5% of the total sample), and there was no difference between condition on leaving a comment, χ²(2, N = 583) = .17, p = .918; narrative: 31 participants, expert interview: 31, control: 34. However, we were curious whether video condition would influence the nature of comments (i.e., pointing out the subtle bias). We ran a logistic regression focusing on participants who left comments, and predicting whether or not they left a comment about the subtle gender bias in the clip (yes = 1, no = 0) with condition dummy coded (control condition as the reference group) and controlling for gender. There was no significant difference between the narrative versus the control condition, (B = −0.00, Z = 0.00, p = .994). However, there was a significant difference between the expert interview versus control conditions (B = 1.16, Z = 4.59, p = .032). Participants were more likely to point out the subtle bias in the video clip in the expert interview condition, compared to the narrative and control conditions (expert interview: 51.6% commented, narrative: 25.8% commented, and control: 26.4% commented).

To further investigate Hypothesis 1, we tested whether participants were more likely to recognize bias in the hypothetical everyday situations. Predicting recognition of bias in the situations with bias, the contrast comparing the experimental videos to the control videos was not significant, t(580) = −1.54, p = .125, d = 0.13. However, the contrast comparing the narrative and expert interviews was significant, t(580) = −2.42, p = .016, d = 0.20. As Table 3 demonstrates, participants were better able to recognize the subtle bias in these everyday situations after viewing the expert interviews, relative to the narrative and control videos. Both contrasts comparing the experimental to control conditions, t(580) = −0.649, p = .516, d = 0.05, and comparing the narrative and expert interview condition, t(580) = −0.646, p = .519, d = 0.05, predicting recognizing bias in situations without bias were not significant. Thus, the expert interview condition increased recognition of bias in everyday situations where there was evidence of subtle bias, and not in instances without any indication of unfair treatment.

Finally, we examined whether the experimental videos promoted feelings of responsibility and intentions to confront the discriminator in the hypothetical situations with subtle bias. Predicting feelings of responsibility to confront, the contrast comparing the experimental videos to the control videos was not significant, t(580) = 0.92, p = .357, d = 0.08, whereas the contrast comparing the narrative and expert interviews was marginal, t(580) = −1.65, p = .100, d = 0.14. Examining intentions to confront, the contrast comparing the experimental videos to the control videos was again not significant, t(580) = 0.35, p = .725, d = 0.03, but the contrast comparing the narrative and expert interviews was significant, t(580) = −2.04, p = .042, d = 0.17. The expert interview condition resulted in marginally higher feelings of responsibility to confront and significantly higher intentions to confront in everyday situations than the narrative or control condition (see Table 3).

Mediators

We next looked at the effect of video condition on our mediators of interest. We again ran one-way ANOVAs predicting each measure and compared the effects of both experimental video conditions to the control condition and the effects of the narrative condition to the expert interview condition via orthogonal contrasts (+1 +1 −2 and +1 −1 0). All the means and standard deviations for condition are again reported in Table 3.

Supporting Hypothesis 2a, the contrast comparing the experimental conditions to the control condition was significant for transportation, t(580) = 4.40, p < .001, d = 0.37, and identification, t(580) = 4.95, p < .001, d = 0.41. In addition, the contrast comparing the narrative condition to the expert interview condition was significant for transportation, t(580) = 5.08, p < .001, d = 0.42, and identification, t(580) = 7.22, p < .001, d = 0.60. As Table 3 shows, the narrative condition resulted in higher transportation and identification with characters than either the control or the expert interview conditions.

The contrast comparing the experimental conditions to the control condition was also significant for logical thinking, t(580) = −3.17, p = .002, d = 0.26, in that the control condition resulted in more logical thinking than the combination of the narrative and expert interview conditions. The contrast comparing the narrative condition to the expert interview condition was also significant, t(580) = −4.41, p < .001, d = 0.37. As predicted, the expert interview condition resulted in more logical thinking than the narrative condition.

Finally, the contrast comparing the experimental conditions to the control condition was significant for perceived knowledge of gender bias facts, t(580) = −14.90, p < .001, d = 1.24. Participants reported higher perceptions of being knowledgeable about gender bias facts in the experimental conditions than in the control condition. However, the contrast comparing the narrative to the expert interview conditions was also significant, t(580) = −5.56, p < .001, d = 0.46. As anticipated, after watching the expert interview videos, participants perceived that they had more knowledge about gender bias facts than after viewing the narrative videos.

Testing Proposed Mediation Models

With the exception of recognizing bias in situations without bias, all of the self-report measures indexing bias literacy were highly correlated (all ps < .001; see Table 4). Thus, to test our mediation models, we created a composite measure of bias literacy. We calculated z-scores for each of the self-report measures and took the average of the z-scores to assess bias literacy (excluding recognizing bias in situations without bias). All of our proposed mediators correlated significantly with the bias literacy measure (see Table 5).

OPEN IN VIEWER

Table 4.

Correlations Between Bias Literacy Measures.

Bias Literacy Measure	1	2	3	4	5	6	7	8	9
1. Awareness of gender bias in science	—
2. Awareness of male privilege in science	.77***	—
3. Knowledge of gender bias equity	.82***	.78***	—
4. Self-efficacy in recognizing gender bias	.32***	.29***	.34***	—
5. Recognizing bias in clip	.33***	.37***	.38***	.28***	—
6. Recognizing bias in situations with bias	.50***	.48***	.55***	.31***	.43**	—
7. Recognizing bias in situations without bias	−.11*	−0.05	−.16***	.11**	.18***	0.02	—
8 Responsibility to confront	.37***	.31***	.36***	.26***	.33***	.69***	0.06	—
9. Intentions to confront	.40***	.35***	.40***	.28***	.29***	.31***	.34***	.44***	—

***p < .001. **p < .01. *p < .05.

OPEN IN VIEWER

Table 5.

Correlations Between Mediators and Bias Literacy Composite Measure.

Measure	1	2	3	4	5
1. Bias literacy	—
2.Transportation	.42***	—
3. Identification	.31***	.55***	—
4. Logical thinking	.30***	.37***	.35***	—
5. Perceived knowledge of gender bias facts	.46***	.35***	.31***	.22***	—

***p < .001. **p < .01. *p < .05.

To examine the indirect effects and test our mediation models, we utilized Hayes and Preacher’s (2014) mediate macro for SPSS Version 23, which allowed us to examine the indirect effects of categorical variables (i.e., three or more conditions). For all models, we utilized 5,000 bootstraps and controlled for participant gender. We also ran the mediation models without controlling for gender and found similar results. We first ran a parallel mediation model (i.e., included and controlled for both mediators in the same model) with transportation and identification (see Figure 1). Supporting Hypothesis 2b, we found that for the narrative versus control condition, there were significant specific indirect effects (i.e., the confidence interval did not cross 0) on bias literacy through transportation (.137, 95% CI [0.082, 0.207]) and identification (.078, 95% CI [0.016, 0.154]). After watching the narrative videos, participants felt more transported and had higher identification with the characters and both related to higher bias literacy.

OPEN IN VIEWER

Figure 1.

Mediational model testing the indirect effect of video condition on bias literacy through transportation and identification. The total effects are shown without parentheses, and the direct effects (i.e., controlling for transportation and identification) are shown with parentheses. B indicates the unstandardized regression coefficient. *p < .05. **p < .01. ***p < .001.

There was no difference between expert interview versus control condition on logical thinking, and as a result, we could not test for logical thinking as a mediator for these condition differences. However, the expert interview did lead to significantly more perceived knowledge of gender bias facts than the control condition, and thus, we ran an analysis with perceptions of gender bias facts as a potential mediator. As predicted by Hypothesis 2b, for control versus expert interview, we found a significant indirect effect on bias literacy through perceived knowledge of gender bias facts (.443, 95% CI [0.342, 0.557]; see Figure 2). As Figure 2 shows, we also found a significant indirect effect of the narrative versus control condition on bias literacy via perceptions of gender bias research knowledge (.283, 95% CI [0.207, 0.371]). Both the narrative and expert interview condition resulted in more perceived knowledge of gender bias facts, which subsequently led to more bias literacy.

OPEN IN VIEWER

Figure 2.

Mediational model testing the indirect effect of video condition on bias literacy through perceived knowledge of gender bias facts. The total effects are shown without parentheses, and the direct effects (i.e., controlling for perceived knowledge of gender bias facts) are shown with parentheses. B indicates the unstandardized regression coefficient. *p < .05. **p < .01. ***p < .001.

Finally, because the expert interview also caused participants to be more inclined to point out subtle bias in the video clip, compared to the narrative condition, we were curious if logical thinking and perceived knowledge of gender bias facts would help explain this unexpected difference. We again ran a parallel mediation analysis predicting pointing out versus not pointing out the subtle bias (yes = 1, no = 0), with logical thinking and perceived knowledge of gender bias facts as mediators. The specific indirect effect on pointing out the gender bias via logical thinking was not significant (.316; 95% CI [−0.319, 1.196]; see Figure 3). However, for the narrative versus expert interview condition, there was a significant specific indirect effect on pointing out the gender bias through perceived knowledge of gender bias facts (.995; 95% CI [0.049, 2.556]; see Figure 3). Thus, being in the expert interview, compared to narrative condition, led to higher perceptions of having knowledge about gender bias facts which predicted greater likelihood of pointing out the biased nature of the video clip.

OPEN IN VIEWER

Figure 3.

Mediational model testing the indirect effect of narrative versus expert interview condition on committing on clip through logical thinking and perceived knowledge of gender bias facts. The total effects are shown without parentheses, and the direct effects (i.e., controlling for logical thinking and perceived knowledge) are shown with parentheses. B indicates the unstandardized regression coefficient. *p < .05. **p < .01. ***p < .001.

Experiment 2 demonstrated that our newly created experimental videos positively influenced multiple components of bias literacy. Compared to the control videos, both the narrative and expert interview videos increased (a) awareness of gender bias in science, (b) knowledge about gender inequity generally, (c) self-efficacy beliefs in ability to notice gender bias, and (d) recognizing gender bias in a new video clip. The predicted differences between the narrative and expert interview conditions also emerged. After watching the narratives, participants felt more transported and identified more strongly with the characters in the video compared to the expert interview. In contrast, the expert interview resulted in more logical thinking and perceptions of being knowledgeable of gender bias facts, and perceptions of being knowledgeable of gender facts ultimately functioned as a mediator for greater bias literacy.

We also found some unexpected benefits associated with expert interview presentational style. Compared to both the narrative and control condition, participants in the expert interview condition better recognized subtle bias in hypothetical everyday situations and had greater felt responsibility and higher intentions to confront the perpetrator of bias. In addition, relative to the narrative and control condition, in the expert interview condition, participants were more likely to leave a comment pointing out the subtle gender bias in the video clip that ostensibly future participants would see. After watching the expert interviews, participants felt as though they had a good comprehension of the facts and research relevant to gender bias, which may have in turn legitimized their decision to comment on the video clip and to point out the subtle gender bias in the clip.

General Discussion

To address the need for additional resources to increase bias literacy, in the current research, we created and tested an easy-to-administer intervention that relied on two forms of visual media—narratives and expert interviews. The goals of this research were to (a) investigate whether the narratives and expert interviews affected gender bias literacy, (b) examine the different processes characterizing these two presentational styles, and (c) explore whether these divergent mechanisms acted as mediators for greater bias literacy.

Across two experiments, we found compelling evidence that the narratives and expert interviews influenced multiple aspects of bias literacy: (a) awareness of gender bias in the sciences, (b) knowledge about gender bias inequity, (c) self-efficacy in detecting gender bias, and (d) recognizing bias and having intentions to confront subtle bias across various everyday situations. In Experiments 1 and 2, we found that the narratives and expert interviews increased awareness of gender bias in the sciences, relative to control videos. Furthermore, in Experiment 2, compared to the control condition, participants in the narrative and expert interview conditions reported more general knowledge about gender bias research, higher self-efficacy beliefs at being able detect gender bias, and increased recognition of subtle bias in a short clip featuring an unfair interaction between a man and a woman.

Although the narrative and expert interviews were both efficacious at increasing many aspects of bias literacy, some differences between the two experimental conditions emerged. Compared to the narrative and control conditions, the participants in the expert interview condition were more likely to leave a comment that ostensibly future participants would see, pointing out the subtle bias in the video clip. Furthermore, participants in the expert interview condition recognized bias in hypothetical everyday situations (presented in written form) in a science and technology company, felt responsible for confronting it, and intended to confront the perpetrator of bias—more so than participants in the narrative and control conditions.

The different processes associated with the narratives versus expert interviews helped illuminate why both presentational styles influenced responses on many of the bias literacy measures and explain why the expert interviews were more effective than the narratives on some of the outcomes. The narratives promoted feelings of transportation and identification with the characters, which related to increased bias literacy and functioned as significant mediators. In contrast to the narratives, the expert interviews encouraged logical thinking and perceptions that one is knowledgeable about gender bias facts and research, both of which correlated with increased bias literacy. In addition, the expert interviews, but not the narratives, resulted in participants pointing out subtle bias (i.e., leaving a comment on the clip). Perceived knowledge of gender bias facts was a significant mediator for the expert interview versus narrative condition on leaving a comment, whereas logical thinking was not. The expert interviews may have been better than the narratives for stimulating pointing out gender bias behavior, because the expert interviews legitimized participants’ actions by encouraging the perception that participants possessed knowledge about gender bias. Participants may have felt more confident in their comprehension of the gender bias literature, which may have then encouraged them to speak-up about this bias. This finding is in line with previous research, which has found that attitudes and beliefs are more predictive of behavior when individuals perceive they are knowledgeable about the topic (Davidson et al., 1985).

Practice Implications

Harmful gender stereotypes and subtle forms of gender bias impeding women’s advancement in the sciences most likely perpetuate the lack of gender parity in STEM (Milkman et al., 2015; Moss-Racusin et al., 2012; Swim et al., 2001). Unfortunately, validated interventions to increase bias literacy remain rare (Moss-Racusin et al., 2014; Paluck, 2006), and many of the existing efficacious gender bias trainings require that participants commit a fair amount time and be physically present during the workshop (Carnes et al., 2015; Cundiff et al., 2014; Zawadzki et al., 2012). As a result, we aimed to develop a set of easy-to-administer video interventions, which diversity practitioners could utilize independently or in conjunction with existing workshops. For example, our intervention videos may be a useful resource for STEM organizations that would like to increase awareness of subtle bias, extend recognition of the different forms it might take, and enhance intentions to confront gender bias in the workplace, but lack the money or person power to conduct formal in-person trainings. In addition, employees may be reticent to take part in a time-consuming workshop; organizations can use our videos as a more attractive alternative. To ensure diversity practitioners have access to the videos, the coauthors of this article plan to circulate the links of the videos as well as a short blurb about the videos, to STEM department chairs and across relevant e-mail listservs. Furthermore, we will present information about the videos at a variety of pertinent conferences (e.g., the annual Conference on Understanding Interventions that Broaden Participation in Science Careers http://understanding-interventions.org).

It is interesting and informative for organizations using these intervention videos that we did not find any differences between the six individual narrative videos and six individual expert interview videos on awareness of bias in science, suggesting that all 12 videos are efficacious interventions. Nevertheless, having the videos cover different topics may still be beneficial. Diversity trainings often touch on a variety of topics (balancing work and family, bias in hiring, faculty–student relationships), and it may be useful to show a single video related to the training’s specific subject matter in order to facilitate discussion. Thus, we felt it was valuable to develop a wide range of videos in order to have multiple resources available for workshop leaders.

Encouraging people to watch the videos may also have important consequences outside of academic or workplace settings. For example, previous research found even subtle nonverbal cues (such as an uncomfortable facial expression directed toward a member of a stigmatized group) in media portrayals can perpetuate biases against stigmatized groups (Weisbuch et al., 2009). And an analysis of popular films revealed that male scientists often displayed subtle forms of bias against their female colleagues (such as focusing on her appearance rather than her scientific work and competence; Steinke, 2005). Thus, by encouraging participants to recognize difficult-to-detect bias in television and movies, the videos may help mitigate the potentially detrimental effects of media’s portrayal of subtle bias against women in STEM.

The results of the current experiments also speak more generally to using different forms of media as diversity interventions. We found both the narrative and expert interview presentational styles were effective at promoting awareness and knowledge of gender bias and self-efficacy at recognizing gender bias. However, expert interviews were better at promoting detection of subtle bias and confrontation intentions because participants felt more knowledgeable about gender bias. This finding suggests that even if diversity trainings rely heavily on entertaining narratives to demonstrate bias, they would benefit by also incorporating clear facts and evidence based in empirical research.

Limitations and Future Directions

Future research might further explore differences between the two presentational styles and any potential unique benefits associated specifically with the narratives. For example, because the narratives increase caring about the characters, who are victims of bias, the narratives most likely result in feeling upset and uneasy, which may then motivate participants to research facts about gender bias to try to make sense of these aversive feelings (Gilbert, Pinel, Wilson, Blumberg, & Wheatley, 1998). If this were true, the narratives (but not the expert interviews) would motivate viewers to acquire information about gender bias research and facts.

Another limitation of the current research was that our control videos tended to be more entertaining and informative than the experimental videos. Although the narratives presented compelling stories, they also highlighted examples of unfair treatment and discrimination. As a result, participants may have perceived the science documentaries, featuring lighthearted jokes and fun facts about science research, as more entertaining than the narratives. Furthermore, the typical participant from the general population may view information about research in the biological and physical sciences as more informative generally than information about sexism. Thus, future research might compare the narratives and expert interviews to less entertaining and less informative control conditions. For example, future experiments might productively explore whether watching the narrative and expert interview videos is more effective at increasing bias literacy than simply reading facts about gender bias in STEM. Nevertheless, we found that compared to the control videos, the narratives and expert interviews increased bias literacy, even while controlling for perceptions of video entertainment and informativeness. In addition, compared to the control videos, the narrative videos encouraged more transportation and identification with characters, and the expert interview promoted higher perceived knowledge about gender bias research.

It is also possible that the nature of the audience—in terms of background, motivation, and interest—could moderate whether the narratives or expert interviews are most useful. For example, research in persuasion finds that when a message is personally relevant, individuals are more motivated to process and think thoroughly about the content and, as a result, are more persuaded by direct and strong arguments (Blankenship & Wegener, 2008; Petty & Cacioppo, 1990). It is plausible that academic scientists, for whom the content of the videos is personally relevant, would be more convinced by the direct and logical facts in the expert interviews than in the indirect nature of the narratives. However, individuals who strongly believe gender bias is not an issue may be more influenced by the narratives, because narratives are convincing via an indirect route. As a result, viewers may not detect the persuasive content of the video and ultimately may not argue against the message (de Wit, Das, & Vet, 2008; Moyer-Gusé & Nabi, 2010).

Future research also might explore how long the effects of the videos persist; we presented participants with the videos and assessed bias literacy during the same session. Thus, it is possible that our findings may represent a brief effect that has only a temporary influence on participants’ beliefs and behaviors. However, it may also be the case that our videos have enduring consequences, because the videos help participants notice subtle, easy to miss gender bias. By continuing to detect new forms of bias, participants will remain aware of sexism’s persistent influence on everyday interactions. Thus, future research might specifically examine whether the intervention videos encourage people to notice instances of subtle bias in their everyday interactions.

The focus in the present research was on increasing awareness bias literacy because it is a precursor for actions aimed at combating and reducing bias. However, follow-up experiments might also examine how our videos influence attitudes of participants within STEM organizations. Increasing bias literacy should result in more positive attitudes toward women in STEM and lower sexism generally and may also encourage individuals working in STEM to hire and mentor women (Moss-Racusin et al., 2012). Both of these possibilities should be fully explored in subsequent research.

We acknowledge that both the narratives and the expert interviews were dominated by portrayals of male scientists, which could further promote gender stereotypical beliefs. In the narratives, seven of the nine people playing the role of scientists were men; in the expert interview segments, the professor (who was interviewed by a woman) was a man. However, the gender composition in both presentational formats was strategic. Because gender disparity still exists in many academic STEM disciplines (especially at the professor level; National Science Foundation, 2013), it was important to represent this lack of parity in the narratives. The gender composition of the narratives was also meant to highlight the “chilly climate” (Walton, Logel, Peach, Spencer, & Zanna, 2015; Whitt, Nora, Edison, Terenzini, & Pascarella, 1999) for women in STEM. Furthermore, researchers have found that when a man and a woman present identical lectures on sexism, the man is perceived as providing a more honest and valid depiction of the material than the woman (Abel & Meltzer, 2007). Thus, we decided to cast men as most of the experts in the interviews to ensure that the facts about the research were not discounted or ignored by the participants. Although it might be argued that the gender imbalance in our videos could contribute to gender stereotyping in science, we note that both videos promote bias literacy and help viewers recognize the unjust nature of the gender disparity portrayed in the videos. Despite these limitations and opportunities for future research, this research represents a critical first step toward examining and understanding the mechanisms behind the narrative and expert interview videos as diversity interventions that increase gender bias literacy in STEM. We found across two experiments that even a brief exposure to our intervention videos (only 4–6 min) had an immediate effect on participants’ bias literacy, suggesting that these short videos may function as powerful diversity interventions. Thus, we hope these videos will be a useful tool to help diversity practitioners encourage recognition of gender bias in the sciences.