Assessing the Impact of Holocaust Education on Adolescents’ Civic Values: Experimental Evidence from Arkansas

Abstract

Background

American adults overwhelmingly agree that the Holocaust should be taught in schools, yet few studies investigate the potential benefits of Holocaust education.

Objectives

We evaluate the impact of a Holocaust education conference on knowledge of the Holocaust and several civic outcomes, including “upstander” efficacy (willingness to intervene on behalf of others), likelihood of exercising civil disobedience, empathy for the suffering of others, and tolerance of others with different values and lifestyles.

Research Design

We recruit two cohorts of students from three local high schools and randomize access to the Arkansas Holocaust Education Conference, where students have the chance to hear from a Holocaust survivor and to participate in breakout sessions led by Holocaust experts.

Results

We find evidence that the conference increased participants’ upstander efficacy, but fail to reject the null hypothesis that the conference would increase participants’ knowledge or other civic attitudes.

Keywords

civics tolerance empathy Holocaust field trips social studies upstander efficacy

The things I saw beggar description… The visual evidence and the verbal testimony of starvation, cruelty and bestiality were so overpowering… I made the visit deliberately, in order to be in a position to give first-hand evidence of these things if ever, in the future, there develops a tendency to charge these allegations to propaganda.

—Dwight D. Eisenhower, April 15, 1945

Upon liberating the concentration camp near Ohrdruf, Germany and witnessing evidence of the horrifying crimes the Nazis committed, General Dwight D. Eisenhower, Supreme Commander of the Allied Forces, immediately resolved to preserve a record of these crimes for fear they would be considered too unbelievable to have taken place. The crimes in question refer to the Holocaust, a part of the Final Solution to the Jewish Question in which the Nazi government systematically exterminated an estimated 17 million victims, including six million Jews and several hundred thousand Romani (Gypsies), homosexuals, patients with mental and physical disabilities, and others the Nazis deemed “subhuman.” Many of these killings took place in dedicated extermination camps such as Auschwitz-Birkenau and Treblinka, spread across Germany, Poland, Czechoslovakia, and other occupied territories. The Holocaust, often referred to as the Shoah, the Hebrew word for “destruction,” is considered by many to be one of the greatest tragedies of modern human history (Crowe, 1970; Gilbert, 1987; Landau, 1992; Dwork & Van Pelt, 2002; Longerich, 2010; etc.).

Seven decades after the events of the Holocaust, American adults overwhelmingly agree that the Holocaust is an important period of modern history to study yet demonstrate limited knowledge of its events. Researchers from Schoen Consulting, a private research firm, conducted a nationally representative survey of Holocaust knowledge and awareness and found that 93% of all US adults “believe all students should learn about the Holocaust in school” (Schoen Consulting, 2018, p. 6). Despite this strong sense of the subject’s importance, some have expressed concern that Holocaust knowledge and awareness is declining (Astor, 2018).

Scholars are divided on whether Holocaust education can effectively promote civic values. University of Connecticut associate professor Alan Marcus argues that expanded Holocaust education has the potential to “create a better society” as students of the Holocaust would “need to grapple with complicated moral issues” (Marcus, 2018, p. 1). Others express skepticism that Holocaust education interventions can improve tolerance and are concerned that such interventions may inadvertently reinforce feelings of anti-Semitism (Ain, 2020). In addition to these concerns, school districts may have to choose between many subjects on which to spend their constrained time and financial resources and may wonder whether Holocaust education initiatives in particular should be preferred. These school leaders may feel like they must concentrate on subjects that will be tested in end-of-year exams or that alternative initiatives may be more effective than Holocaust education programming at promoting similar outcomes.

Though it is still unclear what benefit, if any, Holocaust education offers, legislators have begun passing laws to mandate or promote Holocaust education programming. As of November 2021, eighteen states require some form of Holocaust education, and two more states have already passed legislation and will begin requiring it by 2023 (U.S. Holocaust Memorial Museum, 2021). At the federal level, on January 27, 2020, the U.S. House of Representatives passed the “Never Again Education Act” to promote various forms of Holocaust education programming (Rich, 2020).

In our study, we estimate the effects of Holocaust education programming on students’ civic values. Our study makes several valuable contributions to the literature. First, our evaluation is set in an experimental context, allowing us to estimate the causal effects of the Arkansas Holocaust Education Conference on students’ knowledge and civic values. Second, our evaluation examines a high-quality intervention that features a Holocaust survivor and subject-matter experts, including university professors, lecturers, and museum fellows, addressing the concern that poor Holocaust education programming may convey inaccurate information or draw inappropriate comparisons. Finally, in our study, we control for baseline measures of students’ knowledge and civic values, which allows us to estimate the causal effects of the conference on student growth along these dimensions.

The remainder of the paper is divided into five sections. In the following section, we review prior literature on Holocaust education both inside and outside the classroom. Next, we establish a theoretical framework and our expected findings for this study. We then describe the intervention, participants, survey instruments, and econometric approach for our analysis. We report our findings and discuss our study’s limitations in the penultimate section. The final section offers concluding thoughts.

Literature Review

Education researchers and curriculum experts conjecture that studying the Holocaust imparts essential lessons of civic values, including justice, tolerance, and the importance of democratic liberties (Carrington & Short, 1997; Doering & Pekarik, 1996; Lindquist, 2006; Marcus, 2018; Russell, 2005; Shiman & Fernekes, 1999). Unfortunately, it remains unclear whether and how Holocaust education programming affects the students who experience it (Brabham, 1997; Totten, 2012). The existing literature examines, mainly by case studies, the effects of Holocaust education through two different delivery modes—(1) Holocaust education in the classroom, led primarily by a teacher, and (2) Holocaust education outside the classroom, such as visiting a Holocaust museum or a memorial site.

Holocaust Education Inside the Classroom

The majority of scholarly work on Holocaust educational programming in the classroom, such as reading a Holocaust-related book or receiving instruction on the Holocaust, examines the association between Holocaust education and civic outcomes. For example, Carrington & Short, 1997 provide a case study of 43 students between ages 14 and 15 in the United Kingdom, asking them through semi-structured interviews whether or not their study of the Holocaust in the previous year had affected their “notions of citizenship … [and] understanding of human rights issues” (p. 273). The authors find that for the students involved, Holocaust education is associated with greater preparation for active citizenship and greater understanding of racism, but cautioned that educators may need to combat against complacency; several students were not worried that events like those of the Holocaust could happen again because of watchdog agencies (Carrington & Short, 1997, p. 280).

Other descriptive longitudinal studies find evidence suggesting possible benefits of Holocaust education on civic outcomes. Cowan and Maitles (2005) find that in the short term, Holocaust education is associated with higher levels of self-reported tolerance for minority groups among Scottish children. In the long-term follow-up to their previous work, Cowan and Maitles find that attitudes about tolerance remain higher for students who received Holocaust education (2007). They also find that in comparison to a group of students who did not receive Holocaust education, those students also felt a greater sense of “collective responsibility” for working against racist attitudes, being less likely to agree with the statement “I think racism has nothing to do with me” (Cowan & Maitles, 2007, p. 126). In the United States, Starratt et al. (2017) find that studying the Holocaust in the classroom is moderately correlated with what they broadly consider “citizenship values” in American adults later in life.

Researchers who have examined Holocaust education in the classroom express the concern that teachers seem to encounter pedagogical “pitfalls” (Lipstadt, 1995, p. 27) when implementing Holocaust education programs in the classroom. For example, several researchers note concerns that Holocaust education may be narrow or shallow, failing to establish the broad historical context in which the Holocaust took place, including decades of anti-Semitic laws passed before the Nazi government implemented the Final Solution (Lindquist, 2006; Riley & Totten, 2002; Schweber, 2003; Wieser, 2001). Lipstadt (1995) notes that teachers often lead students to make inappropriate comparisons to other human rights violations in history such as the passage of Jim Crow laws or the internment of Japanese-Americans, in which groups faced intense persecution and discrimination, but were not killed on a scale comparable to the Holocaust. Finally, Riley and Totten (2002) express the concern that teachers may sometimes convey inaccurate information to students.

Holocaust Education Outside the Classroom

Beyond classroom instruction, students experience Holocaust educational programming through visits to memorial sites where horrific events of the Holocaust took place or to museums that chronicle the events leading up to and occurring during the Holocaust and commemorate its victims. A handful of studies analyze how visits to museums and memorial sites affect students’ knowledge of the Holocaust or civic values and overall suggests null to mildly positive impacts. For example, researchers evaluating educational trips to Yad Vashem, the Holocaust memorial and museum in Jerusalem, find that the visit to the museum did not affect knowledge of the Holocaust and produced only a minimal effect in reducing anxiety when reflecting on the Holocaust for Israeli teens (Bickman & Hamner, 1998). An experimental evaluation of three different types of educational programing at the United States Holocaust Memorial Museum in Washington, D.C. did not find differential effects on students’ knowledge and mixed results on students’ attitudes across the three types of programs (Downey, 2000). Following Bickman and Hamner’s (1998) study design, Bowen & Kisida, 2020 brought middle and high school students to the Holocaust Museum Houston and find that the experience strengthened students’ commitment to protect civil liberties and improved their knowledge of the Holocaust, but surprisingly reduced their levels of religious tolerance. It should be noted that each of these three studies did not have a true control group. Bickman & Hamner, 1998; Bowen & Kisida, 2020 studies leverage exogenous variation in students’ birthdates so that students were randomly assigned to complete the survey instrument either before or after attending the museum. All groups in Downey’s (2000) study visited the Permanent Exhibition, but were randomly assigned to additional programming—a combination of the teacher guide, orientation program, and/or follow-up session. Finally, a descriptive survey of Holocaust knowledge by the Pew Research Center finds that Holocaust museum attendance is positively associated with Holocaust knowledge, and that higher levels of Holocaust knowledge are associated with warmer feelings towards Jews (Pew Research Center, 2020).

The Future of Civics Education?

In addition to providing important historical information about the genocide the Nazis perpetrated against the Jews and other minority groups, Holocaust education could play an important role in providing general civics education for students, which will help them participate in a larger and more diverse community. Since the days of Horace Mann’s common school, education scholars broadly agree that education serves not only to train the minds of students, but also to prepare them in civics to become active and engaged citizens in a democratic society (Gutmann, 1987; Hirsch, 2009; Levinson, 2012; Peterson, 2010). Today, most states require a civics course, though only nine states and the District of Columbia require a full year of civics education and only 17 states require passing a civics exam to graduate from high school (Shapiro & Brown, 2018). An investigation into how social studies teachers promote citizenship finds that public and private school teachers agree on the priorities of teaching citizenship, but at the same time “appear uncertain about what the precise content of a proper civic education should be” (Hess et al., 2010, p. 1). Unfortunately, there is a growing concern that accountability testing has made social studies and civics an afterthought (Brown, 2015; Farkas & Duffett, 2010; West, 2007)

Theoretical Framework

Considering the previous literature, we hypothesize that students randomly assigned to attend the Holocaust Education Conference might become more knowledgeable about the Holocaust and more likely to report desirable civic attitudes. We theorize that the mechanisms for this change come from the following components:

1. Exposure to information will lead to an increase in knowledge in a given subject. Exposure to information presented in a salient manner or in an intense setting will make an even greater impression.

2. Knowledge about a period of history in which civic values or attitudes were tested will cultivate or fortify those values and attitudes.

Exposure to Information will Lead to an Increase in Knowledge

Prior research demonstrates that experiential learning can be effective at improving knowledge, civic values and critical thinking skills. Previous research finds that experiential learning such as service learning projects, social activism, and charitable involvement can positively influence students’ civic values (Bryant et al., 2012; Hunter & Brisbin, 2000; Morgan & Streb, 2001). Other examples of experiential learning include visiting art museums (Bowen et al., 2014; Greene et al., 2014; Kisida et al., 2016), visiting science educational institutions (Weinstein et al., 2014; Whitesell, 2016), and viewing live theater (Greene et al., 2018). We theorize that hearing from a Holocaust survivor or expert may have a more meaningful impact on students than receiving instruction from someone with no personal connection to or expertise in the Holocaust.

Descriptive longitudinal studies of Holocaust education in the classroom raise concerns about how the Holocaust is taught. These concerns include failing to place the Holocaust in its proper historical context (Lindquist, 2006; Russell, 2005; Totten, 2012; Foster et al., 2016), making inappropriate comparisons to other periods of history (Lipstadt, 1995), and including inaccuracies in content (Riley & Totten, 2002). An intervention such as the conference we evaluate, which features university professors, certified Holocaust museum fellows, and other experts in the field, should alleviate these concerns about providing students an inaccurate or incomplete Holocaust education.

Knowledge about History can Cultivate Civic Values and Attitudes

The study of the Holocaust may help to cultivate civic values and attitudes. Just as teachers are unsure how to approach teaching the Holocaust, there are similar uncertainties about teaching civics (Hess et al., 2010). Totten (2013) speculates that including controversial issues in the classroom can improve reflective thinking. Holocaust education particularly can help instill these civic values in students as they wrestle with difficult moral questions (Carrington & Short, 1997; Landau, 1992). U.S. state legislators have proposed it as a means of reducing hate crimes (Astor, 2018; Marcus, 2018; Ziv, 2017). Descriptive longitudinal studies of Holocaust education find it to be associated with students’ reduced racist perceptions of minority groups (Cowan & Maitles, 2005, 2007) and positively associated with students’ democratic and civic values (Carrington & Short, 1997; Starratt et al., 2017). Others theorize that learning about the Holocaust can improve altruism (Tec, 1995), empathy (Jennings, 2010), and commitment (Shiman & Fernekes, 1999).

Unfortunately, experimental evaluations of Holocaust education programming do not find that Holocaust education realizes these theoretical expectations. Bickman and Hamner (1998) find some benefits of visiting Yad Vashem, as students randomly assigned to visit the museum became less anxious about reflecting on the Holocaust and became less likely to agree that Jewish identity was weakened because of the Holocaust. Downey (2000) finds that students who visited the Permanent Exhibit and were randomly assigned to one of three additional forms of education programming at the U.S. Holocaust Memorial Museum felt more engaged with history, but felt less strongly that the Holocaust was personally relevant to them. Finally, Bowen and Kisida (2020) find that students randomly assigned to be surveyed after attending the Holocaust Museum Houston were more likely to prefer civil liberties over order but reported lower levels of religious tolerance. Thus, experimental evaluations of Holocaust educational programming do not consistently find it to fortify students’ civic values.

We theorize that improved knowledge about the Holocaust will strengthen in students the civic values that were tested during the Holocaust: willingness to serve as an upstander on behalf of others (that is, to intervene on behalf of others), proclivity to civil disobedience, empathy for the suffering of others, and tolerance of others with different values or lifestyles. These values are immediately relevant to the students in their respective school communities and will continue to be relevant to them as they enter into adulthood and begin participating in a democratic society.

Methodology

Intervention

To study the potential benefits of Holocaust education on students’ civic values, between 2018-19, we randomly assigned participating students to attend the annual Arkansas Holocaust Education Conference. In the first year of our evaluation, our first cohort of students attended the 27^th annual conference, held on November 16^th, 2018. In the second year of our evaluation, our second cohort of students attended the 28^th annual conference, held on October 25^th, 2019.

Each year, the conference features a Holocaust survivor as its keynote speaker. The theme of the 2018 conference was “The Holocaust: What Was It? Who Knew? Who Cared?” and featured as its keynote speaker Pieter Kohnstam, a childhood acquaintance of Anne Frank and the author of A Chance to Live. The theme of the 2019 conference was “The Holocaust: From Persecution to the Final Solution” and featured as its keynote speaker Dr. Inge Auerbacher, a survivor of the Terezin concentration camp and author of four books, including I Am A Star: Child of the Holocaust. In addition to the two addresses, students had the opportunity to attend three of seven concurrent sessions held throughout the day. These sessions were led by various Holocaust experts, including university professors, lecturers, and Holocaust Museum Fellows. Students attending the 2019 conference also had the chance to hear four of the session leaders in a panel discussion on anti-Semitism. A summary of available breakout sessions can be found in the Appendix.

Participants

In two years, we recruited 163 students representing three local high schools. Each year, we simulated 100 randomizations and chose the simulation with the best balance on baseline characteristics. Overall, 81 participants were randomly assigned to attend the conference and 82 participants were randomly assigned to the control group. Of the participants randomly assigned to attend the conference, 71 students attended, a compliance rate of 86.6%. No students randomly assigned to the control group attended the conference.

Descriptive statistics for our sample of participants are summarized in Table 1. Overall, our sample is predominantly White, with some representation of Hispanic, African-American, and Asian students. The majority of our sample is female. On average, participants are between 16 and 17 years of age. Our sample demonstrates balance on all observable demographic characteristics (p > 0.05).

Table 1.

Baseline Characteristics on Administrative Demographic Data for Combined Cohorts (2018–19).

	Treatment		Control		Difference
	μ	SD	μ	SD	(1) - (2)	SE	p-value
	(1)	(2)	(3)	(4)	(5)	(6)	(7)
Race
White	69.1	46.5	70.0	46.3	−0.1	7.3	0.959
Minority	30.9	46.5	28.0	45.2	2.8	7.2	0.696
Hispanic	18.5	39.1	15.9	36.7	2.7	5.9	0.654
African-American	4.9	21.8	2.4	15.5	2.5	3.0	0.400
Asian	2.5	15.6	4.9	21.7	−2.4	3.0	0.417
Other	4.9	21.8	4.9	21.7	0.0	3.4	0.986
Female	61.7	48.9	59.8	49.3	−2.0	7.7	0.798
Age (in days)	6,105.7	431.4	6,113.5	417.3	−7.8	66.5	0.906
School 1	55.5	50.0	54.9	50.1	−0.7	7.8	0.931
School 2	28.4	45.4	30.5	46.3	−2.1	7.2	0.771
School 3	16.0	36.9	14.6	35.6	1.4	5.7	0.804
n	81		82

Notes. Means and standard deviations for treatment and control groups are reported. All statistics are reported as a percentage of treatment or control group, with the exception of age (reported in days) and N. The difference was calculated as a t-test by treatment status. Standard errors and p-values for differences are reported.

We also tested treatment and control groups for balance on exposure to Holocaust-themed literature or film. We asked subjects if they had read six books about the Holocaust such as The Diary of Anne Frank or seen two movies about the Holocaust, Schindler’s List and Life is Beautiful. In addition, we asked subjects to estimate how many class periods of instruction on the Holocaust they had received. Treatment and control groups were statistically similar in the probability they had read or seen each of these books or films and reported receiving a similar number of class periods with instruction on the Holocaust (all p > 0.05). These descriptive characteristics are summarized in Table 2.

Table 2.

Baseline Measures of Exposure to Holocaust Instruction, Books, and Movies for Combined Cohorts (2018-19).

	Treatment		Control		Difference
	μ	SD	μ	SD	(1) - (2)	SE	p-value
	(1)	(2)	(3)	(4)	(5)	(6)	(7)
The Boy in the Striped Pajamas	61.0	49.1	63.1	48.5	−2.1	7.6	0.780
The Chosen	6.1	24.1	2.4	15.3	3.7	3.1	0.236
The Diary of Anne Frank	72.0	45.2	71.4	45.4	0.5	7.0	0.941
Night	35.4	48.1	23.8	42.8	11.6	7.1	0.104
Number the Stars	30.5	46.3	29.8	46.0	0.7	7.2	0.919
Maus	6.1	24.1	8.3	27.8	−2.2	4.0	0.581
Schindler’s List	22.0	41.6	22.6	42.1	−0.7	6.5	0.918
Life is Beautiful	18.3	38.9	26.2	44.2	−7.9	6.5	0.224
# Classes on the Holocaust	11.9	19.7	9.1	15.7	2.7	2.8	0.324
n	81		82

Notes. We calculated the percentage of treatment and control group participants that had seen or read each of the following books or movies, and the mean number of classes with instruction on the Holocaust. The difference was calculated as a t-test by treatment status. Standard errors and p-values for differences are reported.

Participants completed surveys approximately one month before and one month after the intervention, which measured participants’ knowledge and self-reported civic values, which are described in Tables 3 –5. Our final analytic sample includes 163 participants (81 treatment and 82 control) who completed surveys both before and after the intervention. Overall, study participants demonstrate high levels of civic attitudes as measured by our survey constructs, which may limit our ability to detect effects following the conference. The following section describes the various survey instruments used to measure participants’ knowledge and civic values.

Table 3.

Self-Reported Civic Values at Baseline for Combined Cohorts (2018–19).

	Treatment		Control		Difference
	μ	SD	μ	SD	(1) - (2)	SE	p-value
	(1)	(2)	(3)	(4)	(5)	(6)	(7)
Upstander (1 to 4)	3.31	0.40	3.28	0.50	0.03	0.07	0.627
Civil Disobedience (1 to 4)	2.83	0.54	2.74	0.57	0.09	0.09	0.294
Empathy Efficacy (1 to 4)	3.27	0.60	3.35	0.44	−0.07	0.08	0.366
Empathic Behaviors (1 to 5)	3.95	0.71	3.91	0.66	0.03	0.11	0.747
Tolerance (1 to 4)	3.52	0.45	3.56	0.43	−0.05	0.07	0.499
Knowledge (0–10)
Q1–Q10	7.35	1.89	7.44	1.79	−0.09	0.29	0.762
Q11–Q20 (2019 only)	5.13	1.83	5.77	1.89	−0.65	0.47	0.171
n	81		82

Table 4.

Pre-Treatment and Post-Treatment Measures for Combined Cohorts (2018–19).

	Pre- Conference		Post- Conference		α	Range
	μ	SD	μ	SD	α	Range
	(1)	(2)	(3)	(4)	(5)	(6)
Upstander	3.29	0.45	3.32	0.50	0.80	1 (Very unlikely) to 4 (Very likely)
Civil Disobedience	2.78	0.56	2.73	0.56	0.88	1 (Definitely obey) to 4 (Definitely disobey)
Empathy Efficacy	3.32	0.50	3.31	0.55	0.88	1 (Very unlikely) to 4 (Very likely)
Empathic Behaviors	3.92	0.68	3.86	0.86	0.91	1 (Almost never) to 5 (Almost all the time)
Tolerance	3.54	0.44	3.47	0.53	0.94	1 (Very uncomfortable) to 4 (Very comfortable)
Knowledge
Q1–Q10 (both cohorts)	7.40	1.82	7.50	2.04		0 (Incorrect) to 1 (Correct) on 10 questions
Q11–Q20 (2019 cohort only)	5.44	1.87	5.48	1.80		0 (Incorrect) to 1 (Correct) on 10 questions
n	163		163

Notes. Cronbach’s alphas reported from combined pre- and post-treatment surveys. Cronbach’s alphas by pre- or post-treatment survey and by cohort available upon request.

Table 5.

Survey Measures and Sample Items.

Outcome	Sample Item	Range		Number of items
Outcome	Sample Item	Min	Max	Number of items
(1)	(2)	(3)	(4)	(5)
Upstander	On a scale of 1–4, how likely are you to speak up for someone, even if it will result in ridicule?	1	4	4
Civil Disobedience	If a law existed that you believed to be unjust or immoral, would you obey or disobey the law if it resulted in arrest, imprisonment, or criminal charges?	1	4	10
Empathy Efficacy	On a scale of 1–4, how strongly do you agree with the following statement?: Compassion for others’ feelings is an important consideration for my actions	1	4	5
Empathic Behaviors	On a scale of 1–5, when you are angry at someone, how often do you try to “put yourself in his or her shoes”?	1	5	7
Tolerance	On a scale of 1–4, how comfortable would you be living next door to someone of a different religious faith?	1	4	17
Knowledge
Q1–Q10	Which of the following was NOT a targeted victim group during the Holocaust?	0	1	10
Q11–Q20	Who among the following was a doctor who performed deadly human experiments on prisoners?	0	1	10

Instrumentation

Knowledge

We include several questions to measure students’ knowledge of the Holocaust. For the first cohort, we include 10 questions (e.g., “Which of the following was NOT a targeted victim group during the Holocaust?”). We find that these questions were subject to ceiling effects as nine students answered all 10 questions correctly before the conference (µ = 7.40), and 12 students answered all 10 questions correctly after the conference (µ = 7.50). To address the problem of range restriction, in addition to the 10 original questions, we introduce 10 more difficult questions to the second cohort (e.g., “Who among the following was a doctor who performed deadly human experiments on prisoners?”). In the second cohort, only one student answered all 10 questions correctly before the conference (µ = 5.44) and no students answered all 10 questions correctly after the conference (µ = 5.48). For both years, we wrote the questions to test general knowledge about the Holocaust prior to learning the conference themes. Students took the same knowledge questions in both the pre- and post-survey.

Upstander Efficacy

Previous studies examine the role of a bystander in helping to prevent violence. Slaby and colleagues developed the Slaby Bystander Efficacy Scale to measure participants’ beliefs that violent behaviors could be prevented (Slaby et al., 1994b, 1994a). Similarly, in their evaluation of the Holocaust Museum Houston, Bowen & Kisida, 2020 ask eight questions measuring participants’ belief that they can make a difference in helping to prevent violence. Finally, Banyard and colleagues argue that prosocial bystander behavior can help reduce the incidence of sexual and intimate partner violence (Banyard et al., 2004; Banyard et al., 2007; Banyard, 2008).

Our Upstander Efficacy construct is a four-item scale in which participants reveal their willingness to intervene on behalf of a victim. Responses are based on a four-point Likert scale (1 = very unlikely and 4 = very likely), and a higher score indicates greater upstander efficacy. The scale demonstrates strong construct validity in the combined cohort ( $α$ = 0.80).¹

Civil Disobedience

A common plea of Nazi officers during the Nuremberg trials is that they were simply following orders. In a series of experiments in which subjects administered shocks to a “learner,” former Yale psychologist Milgram (1974) finds that subjects demonstrated a surprisingly high willingness to continue administering shocks. Milgram (1974) reasons, “For a person to feel responsible for his actions, he must sense that the behavior has flowed from ‘the self.’ In the situation we have studied, subjects have precisely the opposite view of their actions—namely, they see them as originating in the motives of some other person.” We theorize that learning about the Holocaust should reduce the moral relevance of authority and increase students’ willingness to exercise civil disobedience.

Our Civil Disobedience construct is a 10-item scale in which participants reveal their willingness to disobey an authority if they believe that obeying an authority would violate their moral values. It is based on a scale developed by Graham et al. (2009), which measures the moral relevance of authority. In our scale, items vary in severity of consequence associated with disobeying, ranging from the mildest “Personal embarrassment” to more severe “Arrest, imprisonment, or criminal charges,” “Harm to family or friends,” or “Death.” Responses are based on a four-point Likert scale (1 = very unlikely or definitely obey to 4 = very likely or definitely disobey). In an effort to ensure survey fidelity and avoid careless answering patterns, one item is reverse-coded. Students are asked if they would “Obey a grown-up, even if what they are asking me to do is morally wrong.” For this item, a response of “Very unlikely” would receive the highest score. A higher overall score indicates a greater proclivity to civil disobedience. The scale demonstrates strong construct validity in the combined cohort ( $α$ = 0.88)

Empathy

We also expect that learning about the Holocaust will teach students to be more empathetic. Survivors such as Marian Turski have called for the world to respond with “empathy and compassion” (UN News, 2019). Researchers describe empathy as “an important component of social cognition that contributes to one’s ability to understand and respond adaptively to other’s emotions, succeed in emotional communication, and promote prosocial behavior” (Spreng et al., 2009, p. 62).

We use two scales based on the Toronto Empathy Questionnaire (Spreng et al., 2009). Our Empathy Efficacy scale is a five-item scale that prompts participants about how important they believe it is to understand the feelings of others. Responses are based on a four-point Likert scale (1 = very unlikely to 4 = very likely). A higher overall score indicates a stronger belief in the importance of empathy. The scale demonstrates strong construct validity in the combined cohort ( $α$ = 0.88). We also ask participants about their Empathic Behaviors. Participants respond to seven questions on a five-point Likert scale (1 = almost never to 5 = almost all the time) on how often they try to understand what someone else may be thinking. A higher overall score indicates that a participant engages in empathic behaviors more frequently. The scale demonstrates strong construct validity in the combined cohort ( $α$ = 0.91).

Tolerance

Another important civic value we consider is tolerance. The Holocaust was motivated by intolerant attitudes towards people of religious backgrounds, minority groups, and sexual orientations that the Nazi regime deemed undesirable. In contrast, one of the hallmarks of a liberal society is tolerance for people from different walks of life. One particular form of tolerance is religious tolerance. Despite “[v]itriolic rhetoric from both left-wing and right-wing media outlets,” findings from Gallup’s, 2004 Religious Tolerance Index reveals that the “vast majority of Americans are at least tolerant of other religious points of view” (Winseman, 2005).

Based on the Gallup Religious Tolerance Index (Winseman, 2005), our Tolerance measure is a 17-item scale in which participants reveal how strongly they agree with certain statements or how comfortable they would be exchanging ideas with, being friends with, or living next door to a person of a different political persuasion, religious faith, race, or sexual orientation. Responses are based on a four-point Likert scale (1 = strongly disagree or very uncomfortable to 4 = strongly agree or very comfortable). To encourage response fidelity and discourage careless answering, two items are reverse coded: “The only acceptable religious beliefs are mine” and “People of other religious faiths should not be allowed to make public speeches.” A higher overall score indicates a higher level of self-reported tolerance. The scale demonstrates strong construct validity in the combined cohort ( $α$ = 0.94).

Descriptive statistics for all survey instruments are summarized in Tables 3 –5. At baseline, treatment and control groups demonstrated balance on all self-reported constructs (see Table 3). Construct validity for each instrument is detailed in Table 4. Sample items are provided in Table 5.

Analytical Strategy

We use the following models in order to estimate the effect of being randomly assigned to attend the Arkansas Holocaust Education Conference on various student outcomes, estimating each multiple regression using Ordinary Least Squares

Y_{i} = β_{0} + β_{1} {treat}_{i} + β_{2} pre Y_{i} + ε_{i}

where:

• $Y_{i}$ represents our post-treatment outcomes of interest for student i;

• ${treat}_{i}$ is student i’s randomly-assigned treatment status and = 1 for treatment and = 0 for control;

• $pre Y_{i}$ is the baseline measure for student i, taken before the scheduled intervention; and

• $ε_{i}$ is the individual’s error term.

Our coefficient of interest is

β_{1}

, which captures the effect of being randomly assigned to attend the Holocaust conference on student outcomes. We do not cluster our standard errors as students are randomly assigned to treatment and control groups on an individual level (Abadie et al., 2017). In total, we run regressions on six outcomes in three samples (Cohort 1, Cohort 2, Combined Cohorts), and a regression on Knowledge (2019) for Cohort 2, a total of 19 regressions. Given the multiple comparisons in our analysis, we implement Benjamini-Hochberg corrections to account for the false discovery rate or FDR (Benjamini & Hochberg, 1995).² Benjamini-Hochberg adjusted p-values are presented below standard p-values in our main findings (see Table 6).

Table 6.

Impact Estimates of Being Assigned to Attend the Conference.

	2018 Cohort	2019 Cohort	Combined Cohorts	FDR Adjustments
	(1)	(2)	(3)	(4)
Upstander	0.26	0.55	0.37	Regressions: 3
SE	(0.18)	(0.19)	(0.13)	Significant p < 0.01: 2
p-value	0.141	0.006**	0.006**	Significant p < 0.05: 2
BH p-value	0.141	0.009**	0.017*
Civil Disobedience	0.06	0.05	0.06	Regressions: 3
SE	(0.17)	(0.19)	(0.13)	Significant p < 0.01: 0
p-value	0.711	0.791	0.638	Significant p < 0.05: 0
BH p-value	0.791	0.791	0.791
Empathy Efficacy	−0.06	−0.05	−0.07
SE	(0.17)	(0.18)	(0.12)
p-value	0.714	0.769	0.593	Regressions: 6
BH p-value	0.769	0.769	0.769	Significant p < 0.01: 0
Empathic Behaviors	0.31	0.15	0.24	Significant p < 0.05: 0
SE	(0.17)	(0.22)	(0.13)
p-value	0.064	0.508	0.073
BH p-value	0.219	0.769	0.219
Tolerance	0.12	0.34	0.20	Regressions: 3
SE	(0.19)	(0.20)	(0.14)	Significant p < 0.01: 0
p-value	0.510	0.101	0.150	Significant p < 0.05: 0
BH p-value	0.510	0.225	0.225
Knowledge
Q1–Q10	0.27	0.14	0.22
SE	(0.15)	(0.18)	(0.12)
p-value	0.075	0.436	0.057	Regressions: 4
BH p-value	0.100	0.436	0.100	Significant p < 0.01: 0
Q11–Q20		0.42		Significant p < 0.05: 0
SE		(0.22)
p-value		0.057
BH p-value		0.100
n	100	63	163

Notes. ** p < 0.01, * p < 0.05. Standard errors reported in parentheses. Benjamini-Hochberg adjusted p-values reported below standard p-values.

Results

Main Findings

We find that participants randomly assigned to attend the Arkansas Holocaust Education Conference increased in their upstander efficacy. Participants in the second cohort expressed a greater willingness to serve as an upstander on behalf of others, a difference of 0.55 standard deviations holding all else equal (p < 0.01). When pooling the two cohorts together, we find that the conference increased upstander efficacy by 0.37 standard deviations, holding all else equal (p < 0.01; p < 0.05 with FDR adjustments). Though statistically significant, this effect size may not be economically meaningful; treatment group students averaged 3.4 on the four Upstander items while control group students averaged 3.2 on the same construct items.

We fail to reject the null hypothesis that the conference would improve students’ historical knowledge of the Holocaust or any other civic attitudes. Our main findings are summarized in Table 6.

Limitations

Our study is subject to a few limitations. First is the question of generalizability. By volunteering for our study, participants already demonstrated a degree of interest in learning about the Holocaust, civic values, or both. The benefits of Holocaust education programming may not extend to students who express less interest in the topic. Though we leverage randomization as our identification strategy, our participants are drawn from local schools and may not be representative of the general population of U.S. students.

Second, our study depends on self-reports rather than actual changes in attitudes or behaviors. Participants may bias their response away from their true beliefs in order to provide more socially desirable answers. The finding that the conference increased participants’ frequency of engaging in empathetic behaviors illustrates this concern. It may be that participants randomly assigned to attend the conference were more willing to provide socially desirable answers than that they changed behaviors by behaving empathetically more frequently.

Conclusion

We find that Holocaust education programming is beneficial for participants’ upstander efficacy. In our study, consistent with our hypothesis, participants randomly assigned to attend the conference expressed greater willingness to intervene on behalf of others (0.37 standard deviations, p < 0.01). Inconsistent with our hypotheses, we do not find any evidence that the intervention increased participants’ knowledge, raised their proclivity to civil disobedience, made them more empathetic, or improved their tolerance.

By featuring sessions led by subject-matter experts as well as a keynote address by a Holocaust survivor, the design of the conference itself addresses general concerns about Holocaust education. Specifically, students who attended the conference were likely to receive accurate, contextualized information that could improve their knowledge and understanding of the Holocaust. However, the conference is only a one-day experience, which may be insufficient to produce measurable effects on students’ knowledge. The general absence of significant findings suggests a limited educational intervention like a one-day conference may not be efficacious for improving students’ knowledge or other civic outcomes. Our current study does not allow us to infer the specific design elements—like length of the intervention—that may make Holocaust education more or less effective. Ultimately, Holocaust education may be an intrinsically worthwhile course of study for America’s students, independent of the effects on those who study it. While this philosophical question cannot be addressed by empirical research, replication of this study in future years, with different student populations, and in different contexts may help us gain more clarity as to the most effective design elements and the benefits of Holocaust education for high school students.

Footnotes

Acknowledgments

We gratefully acknowledge the Arkansas Holocaust Education Committee for their cooperation in facilitating this study, Kathleen Wolf for editorial help and feedback on earlier drafts of this work, and for the teachers and administrators in the three schools with which we worked.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was supported by a generous grant from the University of Arkansas’ Chancellor’s Fund for Fine Arts and Humanities.

ORCID iD

Matthew H. Lee

Notes

Appendix

References

Abadie

Athey

Imbens

G. W.

, and Wooldridge

(2017). When should you adjust standard errors for clustering? NBER Working Paper 24003. National Bureau of Economic Research. https://doi.org/10.3386/w24003

Ain

(2020, January 22). Do museum trips actually prevent anti-Semitism? New study suggests yes. The New York Jewish Week. https://jewishweek.timesofisrael.com/do-school-trips-holocaust-museum-actually-prevent-anti-semitism/

Astor

(2018, April 12). Holocaust is fading from memory, survey finds. New York Times. https://www.nytimes.com/2018/04/12/us/holocaust-education.html

Banyard

V. L.

(2008). Measurement and correlates of prosocial bystander behavior: The case of interpersonal violence. Violence and Victims, 23(1), 83–97. https://doi.org/10.1891/0886-6708.23.1.83

Banyard

V. L.

Moynihan

M. M.

Plante

E. G.

(2007). Sexual violence prevention through bystander education: An experimental evaluation. Journal of Community Psychology, 35(4), 463–481. https://doi.org/10.1002/jcop.20159

Banyard

V. L.

Plante

E. G.

Moynihan

M. M.

(2004). Bystander education: Bringing a broader community perspective to sexual violence prevention. Journal of Community Psychology, 32(1), 61–79. https://doi.org/10.1002/jcop.10078

Benjamini

Hochberg

(1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. Journal of the Royal Statistical Society, 57(1), 289–300. https://doi.org/10.2307/2346101

Bickman

Hamner

K. M.

(1998). An evaluation of the Yad Vashem Holocaust Museum. Evaluation Review, 22(4), 435–446. https://doi.org/10.1177/0193841X9802200401

Bowen

D. H.

Greene

J. P.

Kisida

(2014). Learning to think critically: A visual art experiment. Educational Researcher, 43(1), 37–44. https://doi.org/10.3102/0013189X13512675

10.

Bowen

D. H.

Kisida

(2020). April). Never again: The impact of learning about the Holocaust on civic outcomes. Journal of Research on Educational Effectiveness, 13(1), 67-91. https://doi.org/10.1080/19345747.2019.1652712.

11.

Brabham

E. G.

(1997). Holocaust education: Legislation, practices, and literature for middle-school students. The Social Studies, 88(3), 139–142. https://doi.org/10.1080/00377999709603761

12.

Brown

J. B.

(2015, April). 29). Eighth graders’ flatline on NAEP U.S. history, civics, and geography tests. Education Week. http://blogs.edweek.org/edweek/curriculum/2015/04/student_achievement_in_us_hist_html.

13.

Bryant

A. N.

Gayles

J. G.

Davis

H. A.

(2012). The relationship between civic behavior and civic values: A conceptual model. Research in Higher Education, 53(1), 76–93. https://doi.org/10.1007/s11162-011-9218-3

14.

Carrington

Short

(1997). Holocaust education, anti-racism and citizenship. Educational Review, 49(3), 271-282. https://doi.org/10.1080/0013191970490306.

15.

Cowan

Maitles

(2005). Values and attitudes-positive and negative: A study of the impact of teaching the Holocaust on citizenship among Scottish 11-12 year olds. Scottish Educational Review, 37(2), 104.

16.

Cowan

Maitles

(2007). Does addressing prejudice and discrimination through Holocaust education produce better citizens? Educational Review, 59(2), 115–130. https://doi.org/10.1080/00131910701254858

17.

Crowe

D. M.

(1970). The holocaust: Roots, history, and aftermath. Routledge.

18.

Doering

Z. D.

Pekarik

A. J.

(1996). Assessment of informal education in Holocaust museums. Smithsonian Institute, Institutional Studies Office. https://repository.si.edu/bitstream/handle/10088/17201/opanda_96-2B-Education.pdf?sequence=1&isAllowed=y

19.

Downey

(2000). Findings and implications from an evaluation of school programs at the United States Holocaust Memorial Museum. Visitor Studies Today, 3(2), 13–16.

20.

Dwork

Van Pelt

R. J.

(2002). Holocaust: A history. Norton.

21.

Farkas

Duffett

A. M.

(2010). High school, civics, and citizenship: What social studies teachers think and do. American Enterprise Institute. https://www.aei.org/wp-content/uploads/2014/09/High-Schools-Civics-Citizenship-Full-Report.pdf

22.

Foster

S. J.

Pettigrew

Pearce

A. R.

Hale

Burgess

Salmons

Lenga

(2016). What do students know and understand about the holocaust? Evidence from English secondary schools. Centre for Holocaust Education, Institute of Education, University College London. http://discovery.ucl.ac.uk/1475816/

23.

Gilbert

(1987). The holocaust: A history of the jews of Europe during the second World War. Henry Holt and Company, LLC.

24.

Graham

Haidt

Nosek

B. A.

(2009). Liberals and conservatives rely on different sets of moral foundations. Journal of Personality and Social Psychology, 96(5), 1029–1046. https://doi.org/10.1037/a0015141

25.

Greene

J. P.

Erickson

H. H.

Watson

A. R.

Beck

M. I.

(2018). The play’s the thing: Experimentally examining the social and cognitive effects of school field trips to live theater performances. Educational Researcher, 47(4), 246–254. https://doi.org/10.3102/0013189X18761034

26.

Greene

J. P.

Kisida

Bowen

D. H.

(2014). The educational value of field trips. Education Next, 14(1), 78–86.

27.

Gutmann

(1987). Democratic education. Princeton University Press.

28.

Hess

F. M.

Schmitt

G. J.

Miller

Schuette

J. M.

(2010, September). Foreword to High schools, civics, and citizenship: What social studies teachers think and do. American Enterprise Institute. https://www.aei.org/wp-content/uploads/2014/09/High-Schools-Civics-Citizenship-Full-Report.pdf.

29.

Hirsch

E. D.

(2009). The making of Americans. Yale University Press.

30.

Hunter

Brisbin

R. A.

Jr. (2000). The impact of service learning on democratic and civic values. PS: Political Science and Politics, 33(3), 623–626. https://doi.org/10.2307/420868

31.

Jennings

L. B.

(2010). Challenges and possibilities of Holocaust education and critical citizenship: An ethnographic study of a fifth-grade bilingual class revisited. Prospects, 40(1), 35–56. https://doi.org/10.1007/s11125-010-9143-9

32.

Kisida

Bowen

D. H.

Greene

J. P.

(2016). Measuring critical thinking: Results from an art museum field trip experiment. Journal of Research on Educational Effectiveness, 9(1), 171–187. https://doi.org/10.1080/19345747.2015.1086915

33.

Landau

(1992). The Nazi holocaust. I. B. Tauris & Company Limited.

34.

Levinson

(2012). No citizen left behind. Harvard University Press.

35.

Lindquist

D. H.

(2006). Guidelines for teaching the Holocaust: Avoiding common pedagogical errors. The Social Studies, 97(5), 215–221. https://doi.org/10.3200/TSSS.97.5.215-221

36.

Lipstadt

D. E.

(1995). Not facing history. The New Republic, 212(10), 26–29.

37.

Longerich

(2010). Holocaust: The Nazi persecution and Murder of the jews. Oxford University Press.

38.

Marcus

(2018, May). 23). Why we need to rethink how to teach the Holocaust. The Associated Press. https://wtop.com/national/2018/05/why-we-need-to-rethink-how-to-teach-the-holocaust/.

39.

Milgram

(1974). The perils of obedience. Harper’s Magazine.

40.

Morgan

Streb

(2001). Building citizenship: How student voice in service-learning develops civic values. Social Science Quarterly, 82(1), 154–169. https://doi.org/10.1111/0038-4941.00014

41.

Peterson

P. E.

(2010). Saving schools: From Horace Mann to virtual learning. The Belknap Press of Harvard University Press.

42.

Pew Research Center . (2020). What Americans know about the Holocaust. https://www.pewforum.org/wp-content/uploads/sites/7/2020/01/PF.01.22.20_holocaust.knowledge-FULL.pdf

43.

Rich

(2020, January 20). Why legislation is needed to make Holocaust education more prominent in public schools: 5 questions answered. The Conversation. https://theconversation.com/why-legislation-is-needed-to-make-holocaust-education-more-prominent-in-public-schools-5-questions-answered-130681

44.

Riley

Totten

(2002). Understanding matters: Holocaust curricula and the social studies classroom. Theory & Research in Social Education, 30(4), 541–562. https://doi.org/10.1080/00933104.2002.10473210

45.

Russell

I. W.

(2005). Teaching about the Holocaust: A resource guide. The Social Studies, 96(2), 93–96. https://doi.org/10.3200/TSSS.96.2.93-96

46.

Schoen Consulting . (2018). Holocaust knowledge awareness study executive summary. http://www.claimscon.org/wp-content/uploads/2018/04/Holocaust-Knowledge-Awareness-Study_Executive-Summary-2018.pdf

47.

Schweber

(2003). Simulating survival. Curriculum Inquiry, 33(2), 139–188. https://doi.org/10.1111/1467-873X.00255

48.

Shapiro

Brown

(2018). February 21). The state of civics education. Center for American Progress. https://www.americanprogress.org/issues/education-k-12/reports/2018/02/21/446857/state-civics-education/.

49.

Shiman

D. A.

Fernekes

W. R.

(1999). The Holocaust, human rights, and democratic citizenship education. The Social Studies, 90(2), 53–62. https://doi.org/10.1080/00377999909602391

50.

Slaby

R. G.

Wilson-Brewer

Dash

(1994a). Aggressors, victims, and bystanders: Thinking and acting to prevent violence. Education Development Center.

51.

Slaby

Wilson-Brewer

DeVos

(1994b). Final report for aggressors, victims, and bystanders project. Education Development Center.

52.

Spreng

R. N.

McKinnon

M. C.

Mar

R. A.

Levine

(2009). The Toronto empathy questionnaire: Scale development and initial validation of a factor-analytic solution to multiple empathy measures. Journal of Personality Assessment, 91(1), 62–71. https://doi.org/10.1080/00223890802484381

53.

Starratt

G. K.

Fredotovic

Goodletty

Starratt

(2017). Holocaust knowledge and Holocaust education experiences predict citizenship values among U.S. adults. Journal of Moral Education, 46(2), 177–194. https://doi.org/10.1080/03057240.2017.1303465

54.

Tec

(1995). Altruism and the Holocaust. Social Education, 59(6), 348-353.

55.

Totten

(2012). Holocaust education. In Totten

Pederson

J. E.

(Eds.), Educating about social issues in the 20th and 21st centuries: A critical annotated bibliography (pp. 223–250). Information Age Publishing.

56.

Totten

(2013). Richard E. Gross: Addressing controversial issues in the classroom. In Totten

Pederson

J. E.

(Eds.), Educating about social issues in the 20th and 21st centuries: A critical annotated bibliography (Vol. 2, pp. 45–56). Information Age Publishing, Inc.

57.

UN News . (2019, January 28). ‘Humiliation was the worst’; Holocaust survivor at UN, asks world to act with ‘empathy and compassion’. https://news.un.org/en/story/2019/01/1031502

58.

United States Holocaust Memorial Museum . (2021). Where Holocaust education is required in the US. United States Holocaust Memorial Museum. https://www.ushmm.org/teach/fundamentals/where-holocaust-education-is-required-in-the-us

59.

Weinstein

Whitesell

E. R.

Schwartz

A. E.

(2014). Museums, zoos, and gardens: How formal-informal partnerships can impact urban students’ performance in science. Evaluation Review, 38(6), 514–545. https://doi.org/10.1177/0193841X14553299

60.

West

M. R.

(2007). Testing, learning, and teaching: The effects of test-based accountability on student achievement and instructional time in core academic subjects. In Finn

C. E.

Ravitch

(Eds.), Beyond the basics: Achieving a liberal education for all children (pp. 45–61). Fordham Institute.

61.

Whitesell

E. R.

(2016). A day at the museum: The impact of field trips on middle school science achievement. Journal of Research in Science Teaching, 53(7), 1036–1054. https://doi.org/10.1002/tea.21322

62.

Wieser

(2001). Instruction issues/strategies in teaching the Holocaust. In Totten

Feinberg

, Teaching and studying the Holocaust (pp. 62–80). Allyn & Bacon.

63.

Winseman

A. L.

(2005, March 15). Religious tolerance score edged up in 2004 . Gallup. https://news.gallup.com/poll/15253/religious-tolerance-score-edged-2004.aspx

64.

Ziv

(2017, May 1). Two new efforts launched to require Holocaust education nationwide, one triggered by Spicer. Newsweek. https://www.newsweek.com/two-new-efforts-launched-require-holocaust-education-nationwide-one-triggered-592671