Racial/Cultural Awareness Workshops and Post-College Civic Engagement

Abstract

Racial/cultural awareness workshops constitute a salient form of co-curricular diversity engagement in higher education. Although these workshops are generally quite short in duration (often no more than two hours), previous research suggests that workshop participation is associated with undergraduate civic growth. The current study uses multilevel propensity score matching analyses to explore whether racial/cultural awareness workshops during college are associated with a variety of civic outcomes six years after graduation. Using a 10-year longitudinal sample of 8,634 alumni from 229 institutions, diversity workshop participation is significantly and positively related to 10 post-college behaviors, attitudes/beliefs, and skills/tendencies. Moreover, these effects are consistent regardless of participants’ race/ethnicity, gender, and institutional affiliation.

Keywords

civic engagement college diversity experiences college students post-college outcomes racial/cultural awareness workshops

As colleges and universities work to support racially diverse student populations, a common question is how these schools can promote meaningful engagement around diversity-related issues. Such engagement is critical in order to reap the educational benefits of diversity (Gurin, 1999). Over the past couple of decades, universities have increasingly institutionalized diversity initiatives in curricular and co-curricular spaces (Smith, 2009). While fostering diversity through interpersonal interactions and the curriculum receives a great deal of attention, educators also seek to maximize learning outside of the classroom by fostering co-curricular opportunities for engagement with diversity issues. One such initiative found at many college campuses is the racial/cultural awareness workshop. These workshops vary in scope but generally include content or programming that helps broaden and challenge students’ understanding of issues related to race/ethnicity, diversity, inequality, and/or privilege, among other topics. While the content of workshops can be critiqued as overly simplistic (by providing a “one-stop shop” for students to learn about complex topics), numerous studies have found positive outcomes associated with workshop participation (e.g., Antonio, 2001; Astin, 1993; Cole & Zhou, 2014; Vogelgesang, 2001).

Although these studies document the potential benefits associated with racial/cultural awareness workshop participation during college, it is unknown whether the efficacy of these programs extends beyond the undergraduate years. Do these workshops constitute an experience that challenges students while in school but fades away with the passage of time? Or might there be effects that persist beyond graduation? As educators seek to understand how different curricular and co-curricular diversity opportunities enhance student learning, it is important to understand whether the impact of various diversity initiatives continues after college. Few data sets that contain information about student experiences during college also follow up with students after graduation. Therefore, this article examines the effect of participating in a racial/cultural awareness workshop on numerous civic outcomes six years beyond college.

Diversity Workshops in Higher Education

Given the increasing diversity of U.S. institutions and the United States as a whole, the majority of college campuses offer some type of diversity workshop for their students and/or faculty (McCauley, Wright, & Harris, 2000). Diversity workshops are “a small group situation in which a number of students and a group leader or facilitator discuss diversity-related issues” (McCauley et al., 2000, p. 102). With the goals of improving intergroup relations and reducing intergroup bias, diversity workshops are sometimes referred to as racial or cultural awareness workshops, prejudice reduction workshops, multicultural workshops, pluralism workshops, anti-bias workshops, or intergroup dialogues (McCauley et al., 2000; Schoem, Hurtado, Sevig, Chesler, & Sumida, 2001).

While diversity workshops appear to be present on most campuses, there is surprisingly little research about the prevalence of diversity workshops in higher education, when and how students might participate in them, and what they entail. One study by McCauley and colleagues (2000) has addressed these issues in depth. The authors conducted telephone interviews with almost 300 administrators from a random sample of over 350 U.S. four-year institutions (79% response rate). Although conducted a while ago, this study provides a snapshot of diversity workshops in higher education at the end of the past century; a summary of some of the key results is provided in Table 1. Even then, almost three-quarters (73%) of institutions reported currently offering diversity workshops on their campus, with the vast majority (81%) of institutions reported having offered diversity workshops on their campus either currently or in the past. Most diversity workshops were open to all students whenever the workshops were offered (61%); however, a substantial minority (36%) were available only to first-year students, typically during first-year orientation. Almost half (42%) of institutions that offered workshops required students to participate. The duration of the diversity workshops varied considerably, ranging from 45 minutes to 120 hours (median duration was 2 hours). The size of the workshop groups also varied, ranging from 4 to 1,080 students (median size was 25 students). Even for workshops that contained larger groups of students, institutions often reported using small group discussions as part of the larger session. Many institutions used more than one type of workshop facilitator; staff were the most common leaders (72%), but students (55%), faculty (41%), and outside consultants (32%) also served these roles at many schools.

Table 1

Summary of College Diversity Workshop Characteristics From McCauley, Wright, & Harris (2000)

Characteristic	Percentage
When are diversity workshops offered?
First-year orientation only	31
First year only	5
Open to all whenever given	61
Who leads the diversity workshop?
Staff	72
Students	55
Faculty	41
Outside consultants	32
What training methods are used in diversity workshops?
Participants sharing stories of their own experience with bias or discrimination	92
Group exercises for exploring ethnic differences	87
Written information on handouts	86
Personal contact with minority participants	82
Lectures	75
Discussion of actual campus incidents	73
Role playing or behavioral training	71
Videos	68
Skits	67
Self-discovery exercises	53
What is the length of the typical diversity workshop?
1 hour or less	19
1.25–1.75 hours	30
2.0–2.5 hours	29
3.0-3.5 hours	9
4 hours or more	12
What is the size of the typical workshop group?
14 or less	14
15–19	10
20–24	18
25–29	17
30–39	14
40–59	10
60 or more	17

There was considerable variation in the activities conducted in the workshops, with many institutions reporting using multiple activities. At least 80% of institutions reported that their workshop activities consisted of some form of written information or handouts along with activities to facilitate intergroup contact such as sharing personal experiences with bias or discrimination and/or group exercises for exploring racial/ethnic group differences. At least half of the institutions reported using other methods, such as role playing, videos, skits, and self-discovery exercises. Fewer than half of the institutions used case studies, and less than one-fifth of the institutions utilized stress reduction exercises or computer-based learning. While diversity workshops can cover a broad range of issues, the most frequent topics examined problems experienced by various minority groups, and these often covered more than one topic. The majority of the workshops explored challenges experienced by racial/ethnic/cultural minority groups, ranging from 61% that focused on Native Americans to 89% that focused on African Americans. Diversity workshops also explored issues for gays/lesbians/bisexuals (77%), Jewish students (72%), international students (70%), people with disabilities (67%), other religious groups (48%), women (44%), and general cross-cultural topics (28%).

Theory and Research on Racial/Cultural Awareness Workshops and Civic Outcomes

With the goals of improving intergroup relations and reducing intergroup bias, racial/cultural awareness workshops are narrowly tailored to promote civic growth at a critical developmental stage for college students. Drawing on a number of psychological frameworks (e.g., Piaget, 1971, 1975/1985; Ruble, 1994), Gurin, Dey, Hurtado, and Gurin (2002) posit that many college students are at a developmental stage in which they are forming their values and identities, so they are particularly open to growth associated with diversity experiences. Given that many students are often from racially and socioeconomically homogeneous backgrounds (Orfield, 2009; Orfield, Kucsera, & Siegel-Hawley, 2012), college campuses provide students with greater opportunities to be exposed to opinions and situations different from their own home environments, which are often incongruous with their preexisting stereotypes and worldviews. As a result, students typically seek to resolve this discrepancy, which they may do either by reconciling these interactions with their current beliefs and conceptions or changing their views to incorporate this new information.

Dovidio and colleagues (2004) provided a framework through which diversity-related activities, such as racial/cultural awareness workshops, reduce racial bias. They argue that diversity-related activities operate through exposing students to content-related knowledge and/or intergroup contact approaches. The purpose of exposing students to knowledge is to increase understanding and empathy toward others or to one’s own role and responsibilities in creating social change, whereas the purpose of intergroup contact is to provide students with structured interactions between majority- and minority-group members. These diversity activities stimulate important mediating processes, which include cognitive aspects (i.e., regarding the ways that students think about others) and emotional aspects (i.e., regarding the ways that students feel about others); in turn, these affect students’ civic development by increasing cultural awareness and reducing racial bias.

Some research has directly examined the link between racial/cultural awareness workshops and college civic outcomes. Astin (1993) examined the extent to which workshop participation predicts 82 different outcomes when controlling for pretests, precollege characteristics, and other variables within a large, multi-institutional sample. He found that workshop attendance was associated with a variety of civic outcomes, including commitment to promoting racial understanding, cultural awareness, recognition of racial discrimination, and agency for effecting social change. Other studies have shown that these workshops significantly predict greater awareness of racism (Katz & Ivey, 1977), cultural knowledge/understanding and leadership ability (Antonio, 2001), civic mindedness (Cole & Zhou, 2014), motivation to promote inclusion and social justice (Zuniga, Williams, & Berger, 2005), commitment to promoting racial understanding (Antony, 1993; Hyun, 1994; Milem, 1994; Vogelgesang, 2001), openness to diversity and challenge (Pascarella, Edison, Nora, Hagedorn, & Terenzini, 1996; Whitt, Edison, Pascarella, Terenzini, & Nora, 2001), and attitudes toward diversity (Springer, Palmer, Terenzini, Pascarella, & Nora, 1996).

Additional research has examined outcomes associated with co-curricular diversity activities in general; this broader construct typically includes not only racial/cultural awareness workshops but also other campus events that explore issues of culture and difference. Several longitudinal studies have shown that co-curricular diversity experiences are positively associated with civic attitudes, values, and behaviors (e.g., Denson & Bowman, 2013; Engberg, 2007; Gurin et al., 2002; Hurtado, 2005; Taylor, 1994). Moreover, quantitative meta-analyses have shown that co-curricular and curricular diversity experiences are associated with not only increased civic outcomes (Bowman, 2011) but also improved cognitive skills and tendencies (Bowman, 2010) and reduced racial bias (Denson, 2009). The size and consistency of these positive effects are noteworthy. Within a meta-analysis of college diversity experiences and civic engagement, Bowman (2011) found that the average effect sizes for co-curricular diversity, diversity coursework, and intergroup dialogue were virtually identical.

An understudied issue is the extent to which the relationship between co-curricular experiences and civic outcomes varies as a function of student or institutional characteristics. Providing some indirect evidence, Denson’s (2009) review found that studies with a higher proportion of students of color within the sample tended to have a weaker link between co-curricular/curricular activities and racial bias, which suggests that White students might benefit more than students of color. Moreover, the link between cross-racial interaction and civic growth also varies as a function of institutional diversity climate, such that this association is more positive at colleges and universities with poorer campus climates (Denson & Chang, 2009, 2015). Therefore, the potential presence of differential relationships between racial/cultural awareness workshops and civic outcomes also merits attention.

Sustained Effects of Short-Term College Experiences

Two additional studies have examined the extent to which racial/cultural awareness workshops during college predict outcomes well after graduation. Bowman, Brandenberger, Hill, and Lapsley (2011) found that undergraduate workshop participation was positively related to volunteer work 13 years beyond college among alumni from a single religiously affiliated university. Moreover, within a multi-institutional sample, Jayakumar (2008) found that workshop participation was positively and significantly associated with pluralistic orientation and leadership skills five years after college among White alumni (racial/ethnic minorities were not included within the study).

Is it really possible that this type of short experience—which may last only a couple of hours—can have such a lasting impact? Experimental evidence indeed suggests that other short-term interventions may have a sustained effect on college student outcomes. In a notable example, Walton and Cohen (2011) had Black and White first-year students take part in a one-hour laboratory session that was described as investigating the experiences and attitudes of first-year students. Participants who were randomly assigned to the treatment condition read survey results indicating that many students did not feel that they belong on campus at first but these concerns dissipated over time. Participants then wrote a short essay and gave a short speech (presumably to be given to future first-year students) about how their concerns about belonging had changed over time. The control group read results from a survey that was irrelevant to college sense of belonging (i.e., regarding changes in students’ social-political views) and provided a short essay and speech on that topic. This intervention was intended to be most influential for Black students, who typically have greater concerns and challenges with campus belonging than do White students. When assessed three years later, Black students in the treatment condition had higher GPAs and reported being happier and healthier than Black students in the control condition, whereas there were no significant effects for White students. Other experimental studies with brief interventions (which involve either affirming students’ deeply held values or changing their perceptions about the malleability of intelligence) have demonstrated achievement effects over the course of a college semester (Aronson, Fried, & Good, 2002; Harackiewicz, Canning, Tibbetts, Giffen, & Hyde, 2014; Miyake et al., 2010) and among K–12 students two or three years later (G. L. Cohen, Garcia, Purdie-Vaughns, Apfel, & Brzustoski, 2009; Sherman et al., 2013).

Short-term interventions may also have a sustained impact on civic attitudes and values. Brannon and Walton (2013) conducted a laboratory study in which they created a sense of social connection between a non-Latino participant and a Mexican American confederate (i.e., a student who was posing as another participant but was actually part of the study). Six months after this experiment, participants who had worked with the Mexican American student on a freely chosen Mexican cultural task were more interested in engaging with Mexican American peers and had more favorable attitudes toward undocumented Mexican immigrants than did participants who were randomly assigned to the control groups. Although this one-time experience (which combined intergroup contact with learning about another culture) lasted only about 15 minutes, its impact persisted through the end of the academic year.

How do these brief interventions maintain their long-term impact, especially when most participants forget that the intervention ever occurred and believe that it had no effect (Walton & Cohen, 2011)? According to multiple literature syntheses (G. L. Cohen & Sherman, 2014; Yeager & Walton, 2011), these approaches target recursive processes that lead to accumulated effects over time. That is, these interventions are designed so that students reattribute failure to unstable causes, believe that intelligence and academic performance are malleable (not innate), or perceive that difficulties with college adjustment and belonging are normal (and eventually reduced or eliminated). If the interventions are successful, then students are likely to develop better study skills and habits, greater self-efficacy for high achievement through hard work, and closer relationships with fellow students and faculty. Students benefit in the short term by thinking differently about—and usually undergoing immediate improvements in—academic success and social belonging. Moreover, they may then draw on these psychological and interpersonal resources throughout their college years. In sum, these interventions are effective because they are specifically designed to influence mechanisms and processes that shape student success. Similarly, racial/cultural awareness workshops may change students’ perceptions, attitudes, and behaviors regarding diversity and other civic issues in the short term, which may then lead to psychological and behavioral tendencies that persist in the long term.

Limitations of College Diversity Research

However, given the available evidence, there may be no true causal relationship between college diversity experiences and post-college outcomes. The vast majority of studies on college diversity experiences and student outcomes are nonexperimental; as a result, it is unclear whether engaging in a racial/cultural awareness workshop (or other forms of engagement) actually causes civic engagement during the college years or beyond. Moreover, most of the analyses in Jayakumar (2008) and Bowman et al. (2011) did not use pretests for the outcome variables upon entering college, so the observed relationships do not reflect changes in civic engagement over time. It is extremely difficult to conduct a large-scale, real-world experiment in which some students are randomly assigned (and then required) to participate in diversity workshops, whereas other students are randomly assigned not to participate (and then prevented from attending even if they want to do so). Given that randomized experiments constitute the “gold standard” of causal inference (e.g., Campbell & Stanley, 1966; Shadish, Cook, & Campbell, 2002), research that falls short of that standard may yield different results than does experimental research. Indeed, in their meta-analysis of hundreds of studies on intergroup contact and intergroup bias, Pettigrew and Tropp (2006) observed differential effects for experimental versus nonexperimental studies. Self-selection into college diversity experiences may constitute a significant problem for drawing strong causal conclusions. In their seminal review, Feldman and Newcomb (1969) observed that students often choose college experiences that accentuate their preexisting attributes. In a recent diversity-related example, Bowman and Brandenberger (2012) showed that students who entered a service-learning course with higher levels of equality and social responsibility (ESR) orientation were more likely to have positive experiences with diversity during the course, which were then associated with increases in ESR orientation. In other words, a preexisting student characteristic (in this case, the pretest) was substantially related to both the experience and the outcome, which can lead to challenges for drawing causal conclusions from those observational data.

To help address this difficulty, educational researchers have increasingly begun to use quasi-experimental methods to provide a more rigorous examination of the causal impact of an intervention (see Shadish et al., 2002; Thyer, 2012). One quasi-experimental approach designed to address self-selection into experiences is propensity score matching (PSM; see Guo & Fraser, 2015; Holmes, 2013). The basic idea of PSM is to determine the likelihood that students (or others) will participate in a given experience (i.e., treatment) and then to “match” these students with other students who are equally likely to participate. If students in the treatment condition fare better on the outcome variable than their matched counterparts in the control condition (who did not participate in the experience), then one can be more confident that the treatment influenced the outcome. The strength of causal inference from a propensity score analysis depends on having variables that contribute both to selection into the treatment and to the level of the outcome. For example, a PSM analysis of the impact of a bachelor’s degree on future earnings should include precollege socioeconomic status as one of the variables within the propensity score, since this precollege attribute is very likely related to degree attainment and future earnings.

Present Study

The purpose of this study is to examine the long-term effects of college participation in racial/cultural awareness workshops. The present study used PSM analyses to examine the relationships between participation in these workshops during college and numerous civic outcomes six years after graduation. Key constructs associated with the treatment and outcomes were used to create the propensity score, including pretests and proxy pretests for the outcomes, race-related attitudes, political orientation, intended college behavior, demographics, and institutional characteristics. All of these variables were collected when students entered college, so those attributes were precursors to both workshop participation and post-college outcomes. The use of a three-wave longitudinal study—with surveys administered upon entering college, end of the senior year, and six years after college—also ensures that students are more likely to remember their college experiences (as opposed to recalling their college experiences on an alumni survey). In addition to the quasi-experimental and longitudinal design, this study also improves on previous research by examining alumni behaviors (whereas most studies have focused on attitudes and self-perceptions), predicting civic outcomes that are both diversity related and not diversity related, considering conditional effects of workshop participation (i.e., whether these relationships vary by race/ethnicity, gender, or institution), using a large multi-institutional sample, and assessing post-college outcomes (as opposed to exploring gains during college and assuming that these will persist beyond graduation). Additional analyses also explored whether workshop participation predicts outcomes above and beyond interpersonal and curricular diversity engagement.

Method

Data Source and Participants

The data set was collected through the Cooperative Institutional Research Program, which is housed within the Higher Education Research Institute (HERI) at the University of California, Los Angeles. Three waves of data collection were included from students’ entry into college in 1994 (Freshman Survey), end of their senior year in 1998 (College Senior Survey), and six years after college in 2004 (Civic Engagement Survey). The Freshman Survey contained information about students’ demographics, attitudes, precollege experiences, and college intentions. The College Senior Survey asked about students’ college experiences and other indicators (e.g., college satisfaction), and institutional data were linked to student records. Finally, the Civic Engagement Survey contained information about post-college experiences, attitudes, and behaviors. A total of 8,634 participants who graduated from 229 institutions completed surveys at all three timepoints. Because each college or university in the sample was responsible for administering the Freshman and Senior Surveys to its own students, the overall initial response rate and first retest response rate are unknown. In contrast, HERI directly administered the Civic Engagement Survey, which had a retest response rate of 50%. This sample was 67.6% female, 90.7% White, 3.5% Asian American/Pacific Islander, 3.1% Black/African American, 2.6% Latino/Hispanic/Chicano, 1.2% American Indian/Alaska Native, and 1.8% other race/ethnicity (the race/ethnicity figures add up to slightly more than 100% because participants were allowed to select multiple categories). Thirty-seven percent of participants reported having attended a racial/cultural awareness workshop in college. The institutional sample included five Historically Black Colleges and Universities (HBCUs) and 23 public institutions. The average institutional SAT score (verbal plus math) ranged from 615 to 1410, with a mean of 987.

Measures

Dependent Variables

A variety of post-college civic behaviors, attitudes/beliefs, and skills/tendencies were included as outcomes. For behaviors, community leadership was assessed with a three-item index (Cronbach’s alpha = .76) regarding activities in which participants took a key civic role (e.g., “played a leadership role in improving your community”; 1 = never, to 4 = frequently). This variable was highly skewed such that about half of participants reported never having engaged in any of these behaviors. Since no transformation could yield even an approximately normal distribution, this variable was dichotomized into participants who had never participated in a community leadership experience since college (i.e., they had a mean of 1.0) and those with a mean greater than 1.0 (preliminary analyses showed that using different cut points yielded similar substantive results). Volunteer work was indicated with a single item regarding the number of hours per week spent volunteering during the past year (1 = none, to 8 = over 20 hours). Because this variable was also skewed, it was natural log transformed to yield an approximately normal distribution. Donating money to nonprofit organizations or political causes was indicated with a five-item index (1 = never, to 4 = frequently; α = .62). An 11-item index was used to measure the frequency of accessing various news sources through print, online, television, and radio formats (1 = none, to 4 = 5+ times per week; α = .65). To allow for a more meaningful interpretation of effect size, the three continuous variables (news consumption, donating money, and volunteer work) were standardized with a mean of zero and a standard deviation of one. As a result, unstandardized multivariate coefficients predicting these outcomes can be interpreted as indicating the effect of racial/cultural workshop participation in standard deviation units (J. Cohen, Cohen, West, & Aiken, 2003). In addition, discussing racial issues and socializing with someone of another racial/ethnic group during the past year were each indicated with a single item (1 = not at all, to 3 = frequently).

For attitudes and beliefs, participants’ personal value for keeping up to date on politics was measured (1 = not important, to 4 = essential). In addition, an item about how individuals can do little to change society (1 = disagree strongly, to 4 = agree strongly) was reverse-coded so that higher values indicate greater agency for effecting social change. Participants also reported the extent to which everyone in the United States has essentially the same opportunity to be successful (1 = disagree strongly, to 4 = agree strongly); this item was also reverse-coded so that higher values reflect a greater recognition of the presence of an unequal opportunity for success. Finally, for skills and tendencies, participants rated their leadership ability and cooperativeness relative to the average person their age (1 = lowest 10%, to 5 = highest 10%).

Independent Variables

Participation in a racial/cultural awareness workshop during college was measured on the College Senior Survey (0 = no, 1 = yes). All variables used to construct the propensity score were taken from the Freshman Survey; this decision is consistent with the conception of PSM as a technique that measures participants’ preexisting proclivity to participate (or not) in a given intervention. Moreover, the use of variables from an earlier point in time ensures that these attributes may be causally related to participation in the intervention (as opposed to using a cross-sectional design, in which the values for the variables could be influenced by participation in the intervention). Our choice of measures to create the propensity score was influenced by conceptual considerations and previous research. We could only identify one study that examined racial/cultural awareness workshop participation as an outcome (Springer et al., 1996), and this study contained only a small number of predictors. According to their logistic regression analysis, political conservatism, degree aspirations, and parental education were all significantly related to workshop participation. We were not surprised by the significant effect of political orientation, since political orientation and affiliation predict the perceived importance of race within society (e.g., Sniderman, Crosby, & Howell, 2000). Therefore, we included political orientation (1 = far right, to 5 = far left) for creating the propensity score for this study. However, we were not sure why parental education or degree aspirations should logically be related to participation. Preliminary analyses showed that neither of these attributes significantly predicted workshop participation within the present data set; therefore, for both conceptual and empirical reasons, we did not include parental education or degree aspirations.

We chose the other propensity score variables through conceptual considerations and previous research on interpersonal interactions and friendships with diverse peers (e.g., Bowman & Park, 2014; Chang, Astin, & Kim, 2004; Milem, Umbach, & Liang, 2004; Saenz, 2010; Saenz, Ngai, & Hurtado, 2007). The pretest for the outcome variable is quite useful for creating a propensity score, since this variable is often substantially related to both the treatment and the outcome (Pascarella, Salisbury, & Blaich, 2013; Steiner, Cook, Shadish, & Clark, 2010). Therefore, several pretests and proxy pretests were included. The direct pretests were leadership ability, cooperativeness, agency for effecting social change, and time spent volunteering (also log transformed); the proxy pretests were discussing politics (1 = not at all, to 3 = frequently), belief that racial discrimination is a problem (1 = strongly disagree, to 4 = strongly agree), and a seven-item index (α = .80) measuring pluralistic orientation (e.g., importance placed on promoting racial understanding, being a community leader; 1 = not important, to 4 = essential). One of the items within this pluralistic orientation index was actually a direct pretest of an outcome variable (importance of keeping up to date with political affairs). Moreover, as another indicator of proclivity for civic engagement, students’ anticipated likelihood of performing volunteer work during college was also included (1 = no chance, to 4 = very good chance).

Key demographics and precollege experiences that often predict college diversity engagement were used. Race/ethnicity was measured with several dummy variables (American Indian/Alaska Native, Asian American/Pacific Islander, Black/African American, Latino/Hispanic/Chicano, and other race/ethnicity, with White/Caucasian as the referent group), and sex was included (0 = male, 1 = female). Racial/ethnic difference in the precollege environment was indicated with a two-item index (α = .85) that indicated the proportion of high school classmates and neighbors who were from a different racial/ethnic group (1 = none, to 5 = all); we expected that students with more precollege diversity exposure would be more likely to participate in a racial/cultural awareness workshop.

Several institutional characteristics were also used; support for the selection of these variables is provided from studies that not only predict diversity interaction but also student satisfaction with campus diversity (Park, 2009) and diversity advocacy among faculty members (Park & Denson, 2009). The representation of faculty of color and female faculty were both expected to be positively related to workshop participation since faculty from these minority groups tend to perceive a greater need for diversity advocacy (Park & Denson, 2009). Because the distribution of faculty of color was notably skewed, a natural log transformation was conducted on the percentage of faculty of color. HBCUs were indicated with a dichotomous variable (1 = HBCU, 0 = non-HBCU) because both faculty and students at these institutions may be more cognizant of and concerned with racial issues. Given previous studies that showed institutional selectivity is positively related to diversity engagement (Bowman, Park, & Denson, 2015; Chang et al., 2004) yet negatively related to satisfaction with diversity (Park, 2009), the average total SAT score (verbal plus math or equivalent ACT composite score) was divided by 100 and included in the analyses.

However, we ultimately decided to exclude two other institutional variables that did not significantly predict workshop participation within the data set: racial diversity of the student body and institutional control. We anticipated that the structural racial diversity would be positively related to workshop participation, but this was not the case. Moreover, relative to public four-year schools, private schools have faculty who are more likely to advocate for diversity (Park & Denson, 2009), while students who attend private schools are less satisfied with diversity on campus (Park, 2009). Nonetheless, institutional control was not significantly associated with workshop participation within the present data set. Table 2 provides the descriptive statistics for all variables used within the study.

Table 2

Descriptive Statistics

	Mean	SD	Minimum	Maximum
Outcome variables
Community leadership	0.529	0.499	0.000	1.000
Discuss racial issues	2.146	0.591	1.000	3.000
Socialize across race	2.564	0.548	1.000	3.000
Donate money	2.147	0.666	1.000	4.000
News consumption	2.132	0.465	1.000	4.000
Volunteer work	0.641	0.533	0.000	2.079
Agency for effecting social change	3.067	0.777	1.000	4.000
Importance of keeping up to date on politics	2.368	0.912	1.000	4.000
Perceive unequal opportunity for success	2.756	0.903	1.000	4.000
Cooperativeness	3.936	0.708	1.000	5.000
Leadership skills	3.661	0.841	1.000	5.000
Key independent variable
Racial/cultural awareness workshop	0.370	0.483	0.000	1.000
Control variables for select propensity score analyses
Positive cross-racial interaction	1.906	0.468	1.000	3.000
Ethnic studies coursework	0.451	0.498	0.000	1.000
Women’s studies coursework	0.275	0.446	0.000	1.000
Student characteristics used to create the propensity score
Female	0.676	0.468	0.000	1.000
American Indian/Native American	0.012	0.109	0.000	1.000
Asian American/Pacific Islander	0.035	0.184	0.000	1.000
Black/African American	0.031	0.173	0.000	1.000
Latino/Hispanic/Chicano	0.026	0.159	0.000	1.000
Other race/ethnicity	0.018	0.132	0.000	1.000
Racial difference in precollege environment	2.121	0.968	1.000	5.000
Cooperativeness	3.972	0.709	1.000	5.000
Leadership ability	3.695	0.874	1.000	5.000
Agency for effecting social change	3.109	0.833	1.000	4.000
Time spent volunteering	0.816	0.569	0.000	2.079
Intentions for college volunteering	3.136	0.795	1.000	4.000
Pluralistic orientation	2.301	0.572	1.000	4.000
Believe racial discrimination is a problem	3.316	0.733	1.000	4.000
Discussed politics	2.035	0.641	1.000	3.000
Liberal political orientation	2.925	0.808	1.000	5.000
Institutional characteristics used to create the propensity score
Institutional selectivity	9.870	1.300	6.150	14.100
Historically Black College or University	0.022	0.146	0.000	1.000
Representation of faculty of color	1.943	0.696	0.000	4.407
Representation of female faculty	0.360	0.121	0.100	0.830

Note. The continuous outcomes (donate money, news consumption, and volunteer work) were subsequently standardized for inclusion in the multilevel analyses.

Although propensity scores should only be created with variables that occur before the treatment, control variables can be modeled independent of the PSM adjustment when predicting the outcomes (see Holmes, 2013). Therefore, we included three college diversity experiences as control variables. Positive cross-racial interaction was indicated with a four-item index of the frequency with which participants studied, shared a meal, interacted in class, or dated someone from a different racial/ethnic group (1 = not at all, to 3 = frequently; α = .75). These forms of meaningful engagement across difference are more strongly related to college outcomes than is the mere frequency of cross-racial interaction (Denson & Chang, 2015; Hurtado, 2005; Nelson Laird, 2005), and interpersonal interactions are more strongly related to college civic and cognitive outcomes than are co-curricular and curricular diversity experiences (Bowman, 2010, 2011). As a result, including this interpersonal measure provides a strong test of the alternative hypothesis that the results for racial/cultural workshops can be explained by other forms of diversity engagement. Moreover, two dichotomous variables indicated taking at least one ethnic studies course and at least one women’s studies course (0 = no, 1 = yes); such diversity coursework measures are consistently and positively related to civic outcomes in previous research (see Bowman, 2011; Denson & Bowman, in press).

Analyses

Because the current sample contained students/alumni nested within institutions, hierarchical linear modeling (HLM) analyses were used. This nesting violates a key assumption of ordinary least squares (OLS) multiple regression; HLM accounts for this issue by partitioning the variance within institutions (at Level 1) and between institutions (at Level 2) and adjusting standard errors accordingly (Raudenbush & Bryk, 2002). Intraclass correlation coefficients (ICCs) were examined to determine the proportion of variance that occurs between institutions; these can be computed for outcomes that are continuous (e.g., news consumption), ordinal (e.g., agency for effecting social change), and dichotomous (community leadership; for relevant formulas, see Snijders & Bosker, 2012). Although some scholars recommend that multilevel modeling be used if the ICC is at least 5% (Heck & Thomas, 2008), others do not provide a specific minimum ICC value (Luke, 2004; Raudenbush & Bryk, 2002), and single-level analyses cannot be performed on multilevel data if the predictor variables occur at both the individual and institutional levels (Thomas & Heck, 2001), which is the case here. In addition, multilevel analyses are simply considered unnecessary—not incorrect or inappropriate—if the ICC is low (Heck & Thomas, 2008). The ICCs for the post-college civic outcomes ranged from 1% (for cooperativeness and agency for effecting social change) to 7% (for perceiving unequal opportunity for success). As described later in this section, participation in a racial/cultural awareness workshop is also used as an outcome when creating the propensity score and determining whether this score sufficiently accounts for self-selection bias. The ICC for workshop participation was 12%. Listwise deletion was used to examine incomplete data. Participants with missing data on any variable (who comprised 14% of the total sample) did not differ significantly from those with complete data in terms of race/ethnicity, gender, attending a racial/cultural workshop, and 10 of the 11 post-college outcomes.

To select variables for the propensity score, each precollege variable was entered at the appropriate level (student or institution) as the lone predictor of workshop participation. Only significant predictors at p < .05 were used to create the propensity score; as noted earlier, structural racial diversity, institutional control, degree aspirations, and parental education did not meet this criterion. American Indian/Alaska Native was a nonsignificant predictor, but it was retained so that White/Caucasian would continue to serve as the racial/ethnic referent group. The remaining variables were entered simultaneously as predictors of workshop participation: sex, race/ethnicity, racial difference in the precollege environment, cooperativeness, leadership ability, agency for effecting social change, time spent volunteering, anticipated college volunteering, pluralistic orientation, belief that racial discrimination is a problem, discussing politics, and political orientation were modeled at Level 1, while institutional selectivity, HBCU, representation of faculty of color, and representation of female faculty were modeled at Level 2. The logit for each predictor was used to compute a single, linear propensity score for participation in a racial/cultural awareness workshop.

Consistent with the multilevel PSM analyses of Hong and Raudenbush (2005, 2006), stratification was used to match students with similar propensities to participate in the treatment. Stratification is one technique used for comparing the outcomes of students in the treatment and control conditions who are very similar in their predisposition to engage in the treatment. Alternative PSM approaches include matching each individual participant in the treatment group with one (or more) participants in the control group (see Guo & Fraser, 2015; Holmes, 2013; Rosenbaum, 2002). The linear propensity score variable was divided into five equal strata with 20% of participants included per stratum (Cochran, 1968; Guo & Fraser, 2015). This approach is designed to equate participants on the propensity score within each stratum; if this balancing is sufficient, then a two-way analysis of variance (ANOVA) predicting the linear propensity score with strata and treatment condition (workshop vs. no workshop) as independent variables should yield no significant main effect of treatment condition. However, there was a highly significant effect of workshop participation, indicating that workshop participants and nonparticipants did not have the same average propensity score within each stratum. Therefore, greater numbers of strata were tested to further reduce bias (Akers, 2010). A 15-strata solution performed much better, but it still exhibited a main effect of workshop participation and a significant Workshop × Strata interaction, Fs > 8.00, ps < .001. A visual inspection of the means suggested that the data were sufficiently balanced except for the lowest and the highest strata; the lowest stratum contained a large majority of non-workshop participants (some of whom had very low propensity scores), and the highest stratum contained a large majority of workshop participants. Figure 1 provides an overall histogram of propensity scores by treatment group, which illustrates some of this mismatch within the distribution tails. Therefore, all participants from these lowest and highest strata were removed from the final analyses (preliminary analyses showed that this decision produced a somewhat more modest estimate of the effect of workshop participation than including these alumni). The removal of participants from these two strata resulted in a total reduction of 1,033 participants (13%); specifically, 517 and 516 participants were within the lowest and highest stratum, respectively. Within this truncated sample with participants from the 13 remaining strata, the two-way ANOVA found no main effect of workshop participation, F(1, 6697) = 2.205, p = .138, and no significant Workshop × Strata interaction, F(12, 6697) = .997, p = .449.

Figure 1.

Propensity score frequency distributions for students who did and did not participate in a racial/cultural awareness workshop.

Another test of the effectiveness of the propensity score balancing examines whether each variable used to create the propensity score significantly predicts workshop participation when including the propensity score adjustment. If the propensity score succeeds in removing self-selection bias, then the precollege variable should not significantly predict workshop participation when performing the propensity score adjustment (this adjustment occurs by including dummy variables for all but one stratum—leaving one out as a referent group—as additional independent variables within the HLM analysis). Table 3 provides a summary of these multilevel tests; the left-hand columns contain results when entering only the precollege predictor (as described earlier), and the right-hand columns display results that also include the strata within the models. None of the independent variables significantly predicted workshop participation when the PSM adjustment occurred (ps > .05), which indicates that the propensity score successfully removed bias associated with those variables.

Table 3

Hierarchical Generalized Linear Modeling Analyses Predicting Participation in a Racial/Cultural Awareness Workshop Before and After Propensity Score Balancing

	Before Balancing			After Balancing
Predictor	B	SE	p	B	SE	p
Level 1 (student characteristics)
Female	0.365	.050	<.001	.014	.061	.815
American Indian/Native American	0.111	.204	.586	−.043	.258	.867
Asian American/Pacific Islander	0.356	.138	.010	.096	.171	.575
Black/African American	1.334	.179	<.001	.207	.291	.477
Latino/Hispanic/Chicano	0.670	.146	<.001	−.184	.197	.348
Other race/ethnicity	0.475	.170	.005	.139	.228	.542
Racial difference in precollege environment	0.118	.028	<.001	−.005	.032	.879
Cooperativeness	0.114	.031	<.001	−.008	.037	.840
Leadership ability	0.282	.028	<.001	.043	.034	.203
Agency for effecting social change	0.239	.029	<.001	.021	.035	.549
Time spent volunteering	0.529	.040	<.001	.046	.053	.393
Intentions for college volunteering	0.493	.034	<.001	.073	.044	.096
Pluralistic orientation	0.677	.039	<.001	.091	.057	.108
Believe racial discrimination is a problem	0.201	.031	<.001	.026	.036	.472
Discussed politics	0.364	.043	<.001	.039	.050	.430
Liberal political orientation	0.222	.033	<.001	.032	.035	.374
Level 2 (institutional characteristics)
Institutional selectivity	0.143	.044	.001	.019	.042	.651
Historically Black College or University	0.993	.290	<.001	.292	.374	.436
Representation of faculty of color	0.336	.071	<.001	.042	.079	.594
Representation of female faculty	2.009	.543	<.001	.552	.542	.310

Note. All predictors were measured upon entering college at Time 1.

The primary analyses predicted each of the 11 civic outcomes with workshop participation and the PSM strata entered as predictors. Volunteer work, donating money, and news consumption were treated as continuous outcomes through HLM analyses. Hierarchical generalized linear modeling (HGLM) analyses were used to predict the dichotomous outcome (community leadership) and the ordinal outcomes (discuss racial issues, perceive unequal opportunity for success, socialize across race, agency for effecting social change, importance of keeping up to date on politics, cooperativeness, and leadership ability). To explore whether the effect of racial/cultural awareness workshops varies across groups, additional analyses included interaction terms between workshop participation and several variables (gender and each racial/ethnic group in the model). Furthermore, to explore whether the effect of workshops might differ across institutions, the slope for workshop participation predicting each outcome was allowed to vary (i.e., it was not fixed to be identical across schools), and the potential presence of significant variance in slopes was examined.

Moreover, as a form of sensitivity analysis, positive cross-racial interaction, ethnic studies coursework, and women’s studies coursework were added as control variables within propensity score analyses predicting the 11 post-college outcomes. This approach could possibly provide a skewed estimate of the impact of workshops, since all diversity experiences were measured at the same time. Thus, it is unclear whether these additional diversity experiences actually occurred before workshop participation (and therefore would constitute pretreatment control variables) or that the workshops caused students to engage in subsequent diversity experiences (and therefore the inclusion of these variables might obscure the relationship between participation and subsequent outcomes). Nonetheless, if the results of these analyses corroborate the findings from the analyses without control variables, then conclusions about the potential impact of racial/cultural awareness workshops would be strengthened, since these PSM analyses likely provide a more rigorous examination of the link between workshop participation and post-college outcomes.

Limitations

Some limitations should be noted. First, Whites/Caucasians, women, and private institutions were overrepresented within this data set relative to national norms (see The Chronicle of Higher Education, 2013). More than three-fourths of U.S. four-year colleges and universities are privately controlled (Council of Christian Colleges and Universities, 2010), so the overrepresentation of private schools is not as severe as one might think. In addition, moderation analyses determined whether the effect of workshops differs by race/ethnicity, gender, and institution. If the effects of workshop participation on civic outcomes are similar across students and institutions, then these sample characteristics are likely not a concern because they would not alter the relationships of interest. Thus, nonresponse bias would only be a problem for this study if the students and alumni who did not respond are affected differently by racial/cultural workshop participation than those who did respond. Second, the sample consisted entirely of four-year institutions, so it is unclear whether racial/cultural workshops at two-year colleges would yield similar findings. Third, most of the outcome measures consist of single items rather than multi-item indices, which are generally considered to be preferable. However, the range restriction of these single items may mean that the relationships observed in this study constitute underestimates of the “true” effects of racial/cultural workshops. Clearly, considering the pattern of effects across these continuous, ordinal, and dichotomous outcomes will yield a clearer picture of the link between these workshops and post-college civic engagement.

Fourth, students who reported participating in a racial/cultural awareness workshop may have attended more than one workshop while in college. As a result, the analyses here reflect a comparison of students who participated in at least one workshop versus those who did not participate in any workshops. Fifth, participants simply reported whether they had engaged in this co-curricular activity, so the actual content and structure of each workshop is unknown. As a result, the analyses cannot determine whether certain types of workshops might be more effective at promoting long-term outcomes, which may be relevant as such practices have likely evolved since the college experiences in this data set were assessed. Finally, as discussed in more detail in a subsequent section, workshop participation may cause students to engage in other diversity-related activities while in college, so some of the relationships reported here may be indirect. Because all college experiences were assessed in the senior year, it is unclear in what order a student participated in various diversity experiences, so mediating processes cannot be explored directly in this study.

Results

Table 4 displays the results for HLM analyses predicting each dependent variable without the PSM adjustment (left-hand columns) and with the PSM adjustment (right-hand columns). In the unadjusted analyses, workshop participation is a significant, positive predictor of all 11 civic outcomes (ps < .001). When propensity score adjustments are made, the HLM coefficients are smaller for all analyses; this reduction ranges from less than one-fourth (for perceiving unequal opportunity for success) to more than two-thirds (for importance of keeping up to date on politics). However, workshop participation is still a significant predictor for 10 out of 11 civic outcomes (the effect for keeping up to date on politics is positive but nonsignificant). These post-college outcomes cover a range of behaviors (e.g., community leadership, charitable giving), attitudes and beliefs (e.g., agency for social change, perceiving inequality), and skills and tendencies (e.g., leadership, cooperativeness) that are essential for the functioning of an effective society and workforce. The continuous dependent variables (volunteer work, donate money, and news consumption) were standardized with a mean of zero and a standard deviation of one, so the HLM coefficients can be interpreted as the adjusted Cohen’s d for workshop participation. Because community leadership was a dichotomous outcome, a delta-p value can be calculated to determine the predicted change in probability of taking a community leadership role for alumni who did and did not participate in a racial/cultural awareness workshop (Cruce, 2009; Petersen, 1985). Using the workshop participation mean as the initial probability level, workshop participation is associated with a .107 increase in the probability of taking a community leadership role six years after graduation.

Table 4

Results of Multilevel Analyses for Participation in a Racial/Cultural Awareness Workshop Predicting Civic Engagement Outcomes Six Years After College Graduation

	No PSM Adjustment			PSM Adjustment
Outcome variable	B	SE	Odds Ratio	B	SE	Odds Ratio
Behaviors
Community leadership	.588***	.056	1.800	.444***	.061	1.559
Discuss racial issues	.648***	.049	1.911	.398***	.055	1.489
Socialize across race	.483***	.050	1.621	.327***	.057	1.387
Donate money	.229***	.025		.135***	.027
News consumption	.117***	.025		.075*	.030
Volunteer work	.275***	.022		.174***	.024
Attitudes/beliefs
Agency for effecting social change	.407***	.045	1.502	.139**	.051	1.149
Importance of keeping up to date on politics	.302***	.043	1.352	.092	.051	1.096
Perceive unequal opportunity for success	.432***	.044	1.540	.331***	.050	1.392
Skills/tendencies
Cooperativeness	.142***	.041	1.152	.094*	.047	1.098
Leadership skills	.337***	.045	1.401	.153**	.053	1.165

Note. Volunteer work, donate money, and news consumption were continuous outcomes examined with hierarchical linear modeling analyses, and all other variables were examined with hierarchical generalized linear modeling analyses (community leadership was a dichotomous outcome, whereas the remaining outcomes were ordinal). PSM = propensity score matching.

p < .05. **p < .01. ***p < .001.

Results from propensity score analyses that include interpersonal and curricular diversity experiences as control variables are provided in Table 5. Even with the addition of these well-established predictors of civic outcomes, workshop participation is positively and significantly related to 9 of the 11 post-college outcomes (only cooperativeness became nonsignificant when adding the control variables). Moreover, the reduction in the effect size when incorporating these control variables is typically modest; the median decrease in HLM coefficients across all outcomes is 14.7%, and two relationships actually become slightly larger when including control variables.

Table 5

Results of Multilevel Propensity Score Analyses for Participation in a Racial/Cultural Awareness Workshop Predicting Civic Engagement Outcomes Six Years After College Graduation (With Control Variables Added)

Outcome Variable	B	SE	Odds Ratio
Behaviors
Community leadership	.382***	.065	1.466
Discuss racial issues	.270***	.054	1.310
Socialize across race	.199**	.061	1.221
Donate money	.136***	.028
News consumption	.064*	.031
Volunteer work	.167***	.026
Attitudes/beliefs
Agency for effecting social change	.119*	.053	1.127
Importance of keeping up to date on politics	.054	.051	1.056
Perceive unequal opportunity for success	.232***	.050	1.262
Skills/tendencies
Cooperativeness	.070	.049	1.073
Leadership skills	.168**	.055	1.183

Note. Positive cross-racial interaction, ethnic studies coursework, and women’s studies coursework during college were used as control variables in addition to the propensity score matching. Volunteer work, donate money, and news consumption were continuous outcomes examined with hierarchical linear modeling analyses, and all other variables were examined with hierarchical generalized linear modeling analyses (community leadership was a dichotomous outcome, whereas the remaining outcomes were ordinal).

p < .05. **p < .01. ***p < .001.

Additional analyses examined the extent to which the effect of racial/cultural awareness workshops might vary across students and institutions. First, the slope for workshop participation was allowed to vary across institutions to determine whether these workshops might be more effective at promoting post-college civic outcomes at some schools than at others. However, no significant variation in slopes was observed for any of the 11 analyses (ps > .18), which means that this effect is consistent across institutions. Second, additional models were examined to determine whether workshops might be more influential for some students than for others. All such analyses contained the following predictors: workshop participation, PSM strata, six demographic variables (American Indian/Alaska Native, Asian American/Pacific Islander, Black/African American, Latino/Hispanic/Chicano, other race/ethnicity, and female), and the interaction between workshop participation and each of the demographics. Across the 66 interaction terms included in these 11 moderation analyses, only 3 were statistically significant at p < .05, which is exactly what one would expect by random chance if no group differences were present in the population. As a result, these results are not presented in detail here.

Discussion

In summary, participation in a racial/cultural awareness workshop during college is significantly associated with numerous civic outcomes six years after graduation. This pattern is consistent for civic behaviors, attitudes/beliefs, and skills/tendencies; it occurs regardless of whether the outcome is diversity related or ostensibly unrelated to diversity, and it persists even in propensity score analyses that control for other forms of diversity engagement. When considering both the timing of the outcome assessment (6–10 years after the workshop) as well as the brevity of many of these workshops (which often last for no more than two hours), the magnitude of these effects is noteworthy. For instance, alumni who participated in a racial/cultural awareness workshop during college are over 10 percentage points more likely to take a leadership role in their post-college communities. Moreover, alumni who participate in these workshops fare about .15 standard deviations higher in terms of volunteer work and charitable/political donations than those who do not participate. The latter figure is considered “small” according to both J. Cohen’s (1988) widely cited guidelines for social science research and recent guidelines for college student research (Mayhew, Rockenbach, Bowman, Seifert, & Wolniak, 2016). However, it is certainly not trivial when considered in context, as these short workshops are one of many potential ways to bolster long-term civic engagement among college students. This contextual consideration is consistent with J. Cohen’s recommended use of these figures in light of relevant factors, which could include the substantial length of time between treatment and outcome. Moreover, effect sizes are often smaller in educational contexts than in other social science settings (Valentine & Cooper, 2003).

An important contribution of this study is its use of quasi-experimental design to more accurately estimate the effects of participating in a racial/cultural awareness workshop. Our findings show that employing PSM substantially reduces the estimates for all analyses (sometimes by more than half), which suggests that this method accounts for considerable self-selection bias. That is, students who choose to participate in a racial/cultural awareness workshop tend to have different attitudes, goals, personal attributes, and behaviors from those students who have chosen not to participate in these workshops. Even when accounting for self-selection bias, the attenuated effects of participating in racial/cultural awareness workshops have lasting effects six years after college graduation. The largest disparities between adjusted and unadjusted results occur within analyses predicting post-college attitudes and skills (not behaviors). Because most research on college diversity and civic engagement has examined nonbehavioral outcomes (Bowman, 2011), PSM may constitute a particularly important form of analysis for this line of inquiry and should be utilized when possible in future research.

Although the use of propensity score matching constitutes an improvement upon existing research, this quasi-experimental approach is generally perceived to be weaker than some others in terms of drawing causal conclusions (Shadish et al., 2002). Other quasi-experimental techniques are designed to deal with different means of selection into experiences (e.g., regression discontinuity is useful when the treatment assignment is determined by a cutoff score, such as remedial coursework or need-based financial aid). However, PSM is the best available approach for exploring the impact of many student experiences within large, multi-institutional data sets. Indeed, higher education researchers have begun to utilize PSM in recent years to control for self-selection bias in examining the impact of various college experiences on student outcomes (e.g., Clark & Cundiff, 2011; Malcom & Dowd, 2012; Melguizo, Kienzl, & Alfonso, 2011).

Another noteworthy aspect of these findings is that the observed effects are similar regardless of alumni’s race/ethnicity, gender, and institution attended. Some previous evidence suggests that the potential impact of racial/cultural awareness workshops may differ as a function of students’ race/ethnicity. For instance, Hyun (1994) found a larger effect of workshop participation on the commitment to promoting racial understanding for White students than for African American students. In addition, Engberg and Hurtado (2011) observed a larger direct effect of co-curricular diversity experiences on pluralistic orientation for Whites than for Asians but a larger indirect effect for Asians than for Latinos. However, other studies that have examined co-curricular diversity and civic outcomes have obtained similar effect sizes across racial/ethnic groups (Gurin et al., 2002; Kotori, 2009; Vogelgesang, 2001). Any similarity in the magnitude of these relationships does not imply that the processes through which these impacts occur are identical across groups, as the examination of direct effects in this study does not shed light on the specific mechanisms involved. Moreover, the lack of variation across more than 200 institutions in this sample may be the most surprising moderation finding; this result suggests that colleges and universities are about equally effective in providing diversity workshops that contribute to long-term civic outcomes. It is not necessarily the case that the length or content of these workshops are similar across schools; instead, institutions may be adept at designing workshops that are useful for the particular student populations that they serve.

An interesting issue that could not be explored directly within this study is whether students’ motivation for workshop attendance might help shape their subsequent outcomes. Some students attend these workshops as part of a broader training program of which a racial/cultural awareness workshop is a required component (e.g., to become a resident assistant), whereas others attend these as a voluntary, stand-alone activity. Might the outcomes vary for these two types of students? Cognitive dissonance theory (Festinger, 1957) would suggest that students who choose to attend racial/cultural awareness workshops would then view their own participation as evidence that they care deeply about these issues, and their civic attitudes (and perhaps civic behaviors) would then change as a result. If students see themselves as being forced to participate, then these same benefits may not accrue. However, some evidence suggests that students’ apparent receptiveness to diversity may not actually shape these outcomes; specifically, Denson and Bowman (2013) found that the relationships between curricular/co-curricular diversity and college civic outcomes do not differ as a function of students’ openness to diversity or their precollege diversity exposure. Exploring possible differences in students’ motivation for participating (or not) in racial/cultural awareness workshops would prove a fruitful avenue for future research.

Conclusion

Overall, racial/cultural awareness workshops appear to affect various civic outcomes six years after college graduation. This study identified positive and significant relationships for 10 of the 11 civic outcomes through propensity score matching techniques, which provide a rigorous analytic test of the potential impact of this experience. Taken together, the results suggest that participation in such workshops have a noteworthy effect on numerous attitudes and activities that are characteristic of a well-functioning workforce and democracy, such as volunteer work, donating money, community leadership, cross-racial interaction, and discussion of racial issues. Importantly, these results extend beyond usual examinations of outcomes toward the end of college to the critical years following graduation in which students are establishing life patterns around civic mindedness and community engagement.

Keeping in mind that many college students grow up in racially and socioeconomically segregated neighborhoods (Orfield, 2009; Orfield et al., 2012), these workshops may serve as an important first step in exposing students to diverse others and ideas. In turn, these workshops can nurture students’ interest in diversity topics and subsequently enhance the proclivity of those exposed students to seek out more of those experiences or related knowledge. Interestingly, some evidence suggests that the apparent benefits of diversity experiences on various civic and intergroup outcomes are similar regardless of the difference in racial composition between students’ high school and college environments as well as students’ openness to diversity (Denson & Bowman, 2013). Thus, the significance of these workshops is even more compelling given that they may yield improvements even among students with little experience or interest in engaging across difference. This form of co-curricular involvement can provide students who have had limited precollege exposure to diversity a valuable opportunity for a brief but effective introduction to diversity, which can lead to more meaningful and sustained long-term interest in these issues. This approach would be consistent with Gurin et al.’s (2002) developmental perspective on college diversity experiences (which are often unusual and challenging for entering students) and subsequent outcomes.

While some workshops may be part of a broader and longer-term effort such as intergroup dialogue programs, this is often not the case. Thus, the results illustrate the positive and long-lasting role that brief interventions can play in enhancing civic and diversity-related outcomes. However, these workshops should not be a replacement for other critical curricular and co-curricular diversity activities that take place over a longer period of time, such as ethnic studies classes, intergroup dialogue programs, and student organizations that foster discussions around social issues. They can instead supplement and complement such efforts by enhancing student experiences around diversity with an influence that stretches beyond the college years. Racial/cultural awareness workshops may be an effective means of supporting student development because they can function as a bridge between the curricular and co-curricular spaces. Such workshops often contain information derived from academic content, so they may provide a space where material comes “alive” for students outside of the classroom, engaging students in a different way to enhance awareness of issues related to race, culture, and diversity. Workshops can also reach students who may not heavily engage with diversity issues in their major-related coursework.

Moreover, workshops may provide an effective impetus for students to engage in future interpersonal, curricular, and co-curricular diversity experiences. That is, attending these workshops may cause students to become more interested in and comfortable with issues of difference, which then leads them to engage more frequently and substantively with diversity. If this is true, then some of the relationships observed in this study may occur indirectly through these additional diversity experiences. Since few studies have examined the extent to which diversity experiences predict changes in other diversity experiences (see Bowman, 2012; Nelson Laird, Engberg, & Hurtado, 2005), this catalyzing potential of workshops merits attention in future research.

As a whole, colleges and universities are increasingly under pressure to show the worth of a postsecondary degree to the public. Although many consider classroom-based learning to be of utmost importance, this study suggests that co-curricular activities such as diversity workshops also have a marked influence on behaviors that are critical to societal flourishing and the functioning of a healthy diverse democracy. Therefore, as institutions seek to allocate their limited resources to maximize student learning outcomes, well-designed co-curricular activities may constitute a prudent (yet sometimes overlooked) investment. Universities often tout the impact their alumni have on society, and these workshops appear to be a key means of seeding the ground for social change and civic engagement during the post-college years.

Our findings are illuminating given recent events that magnify the continuing significance of race in society. During the writing of this article, the events in Ferguson sparked a national conversation about the persistent and systemic racial injustices that continue to permeate the country. More recently, at the University of Oklahoma, fraternity members were expelled for singing an egregiously racist song, showing how colleges still struggle to effectively engage students in understanding their responsibility and role in a diverse democracy. While racial/cultural awareness workshops as isolated events are not the sole remedy to alleviating these problems that plague college campuses and beyond, they can play a role in engaging students to think critically about the world around them. We recommend that workshops are accompanied by other curricular and co-curricular opportunities for students to spur active learning around these issues.

Footnotes

NICHOLAS A. BOWMAN is an associate professor in the Department of Educational Policy and Leadership Studies as well as the director of the Center for Research on Undergraduate Education at the University of Iowa, N491 Lindquist Center, University of Iowa, Iowa City, IA, 52242; e-mail: nick-bowman@uiowa.edu . He uses a social psychological lens to examine various topics in higher education, including college diversity, student success, college admissions and prestige, and methodological issues for studying college impact.

NIDA DENSON is a senior research fellow in the School of Social Sciences and Psychology at Western Sydney University in Australia. Her research interests include diversity and diversity-related initiatives, racism, educational contexts, campus climates, and faculty work-life balance.

JULIE J. PARK is assistant professor in the Department of Counseling, Higher Education, and Special Education at the University of Maryland, College Park. Her work examines race, diversity, and equity in higher education, with a particular interest in Asian American college students. Other areas of focus include social capital, affirmative action, and the educational benefits of diversity.

Notes

References

Akers

(2010). Determination of the optimal number of strata for bias reduction in propensity score matching. Dissertation Abstracts International, 71 (08), 58A. (UMI No. 3417726)

Antonio

A. L.

(2001). The role of interracial interaction in the development of leadership skills and cultural knowledge and understanding. Research in Higher Education, 42, 593–617.

Antony

(1993, November). Can we all get along? How college impacts students’ sense of the importance of promoting racial understanding. Paper presented at the annual meeting of the Association for the Study of Higher Education, Pittsburgh, PA. (ERIC Document Reproduction Service No. ED 365174).

Aronson

Fried

Good

(2002). Reducing the effects of stereotype threat on African American college students by shaping theories of intelligence. Journal of Experimental Social Psychology, 38, 113–125.

Astin

A. W.

(1993). Diversity and multiculturalism on the campus: How are students affected? Change, 25(2), 44–49.

Bowman

N. A.

(2010). College diversity experiences and cognitive development: A meta-analysis. Review of Educational Research, 80, 4–33.

Bowman

N. A.

(2011). Promoting participation in a diverse democracy: A meta-analysis of college diversity experiences and civic engagement. Review of Educational Research, 81, 29–68.

Bowman

N. A.

(2012). Promoting sustained engagement with diversity: The reciprocal relationships between informal and formal college diversity experiences. The Review of Higher Education, 36(1), 1–24.

Bowman

N. A.

Brandenberger

J. W.

(2012). Experiencing the unexpected: Toward a model of college diversity experiences and attitude change. Review of Higher Education, 35, 179–205.

10.

Bowman

N. A.

Brandenberger

J. W.

Hill

P. L.

Lapsley

D. K.

(2011). The long-term effects of college diversity experiences: Well-being and social concerns 13 years after graduation. Journal of College Student Development, 52(6), 729–239.

11.

Bowman

N. A.

Park

J. J.

(2014). Interracial contact on college campuses: Comparing and contrasting predictors of cross-racial interaction and interracial friendship. Journal of Higher Education, 85, 660–690.

12.

Bowman

N. A.

Park

J. J.

Denson

(2015). Student involvement in ethnic student organizations: Examining civic outcomes six years after graduation. Research in Higher Education, 56, 127–145.

13.

Brannon

T. N.

Walton

G. M.

(2013). Enacting cultural interests: How intergroup contact reduces prejudice by sparking interest in an out-group’s future. Psychological Science, 24, 1947–1957.

14.

Campbell

D. T.

Stanley

J. C.

(1966). Experimental and quasi-experimental designs for research. Boston, MA: Houghton Mifflin.

15.

Chang

M. J.

Astin

A. W.

Kim

(2004). Cross-racial interaction among undergraduates: Some consequences, causes, and patterns. Research in Higher Education, 45, 529–553.

16.

The Chronicle of Higher Education. (2013, August 19). Almanac issue 2013–14.

17.

Clark

M. H.

Cundiff

N. L.

(2011). Assessing the effectiveness of a college freshman seminar using propensity score adjustments. Research in Higher Education, 52(6), 616–639.

18.

Cochran

W. G.

(1968). The effectiveness of adjustment by subclassification in removing bias in observational studies. Biometrics, 24, 295–313.

19.

Cohen

G. L.

Garcia

Purdie-Vaughns

Apfel

Brzustoski

(2009). Recursive processes in self-affirmation: Intervening to close the minority achievement gap. Science, 324, 400–403.

20.

Cohen

G. L.

Sherman

D. K.

(2014). The psychology of change: Self-affirmation and social psychological intervention. Annual Review of Psychology, 65, 333–371.

21.

Cohen

(1988). Statistical power analysis for the behavioral sciences (2nd ed.). Mahwah, NJ: Lawrence Erlbaum.

22.

Cohen

West

S. G.

Aiken

L. S.

(2003). Applied multiple regression/correlation analysis for the behavioral sciences (3rd ed.). Mahwah, NJ: Lawrence Erlbaum.

23.

Cole

Zhou

(2014). Do diversity experiences help college students become more civically minded? Applying Banks’ multicultural education framework. Innovative Higher Education, 39, 109-121.

24.

Council for Christian Colleges and Universities. (2010). Profile of post-secondary education. Retrieved from https://www.cccu.org/filefolder/Profile_US_Post-Secondary_Education-updated2010.pdf

25.

Cruce

T. M.

(2009). A note on the calculation and interpretation of the delta-p statistic for categorical independent variables. Research in Higher Education, 50(6), 608–622.

26.

Denson

(2009). Do curricular and co-curricular diversity activities influence racial bias? A meta-analysis. Review of Educational Research, 79, 805–838.

27.

Denson

Bowman

N. A.

(2013). University diversity and preparation for a global society: The role of diversity in shaping intergroup attitudes and civic outcomes. Studies in Higher Education, 38, 555–570.

28.

Denson

Bowman

N. A.

(in press). Do diversity courses make a difference? A critical examination of college diversity coursework and student outcomes. In Paulsen

M. B.

(Ed.), Higher education: Handbook of theory and research (Vol. 32). New York, NY: Springer.

29.

Denson

Chang

M. J.

(2009). Racial diversity matters: The impact of diversity-related student engagement and institutional context. American Educational Research Journal, 46, 322–353.

30.

Denson

Chang

M. J.

(2015). Dynamic relationships: Identifying moderators that maximize benefits associated with diversity. Journal of Higher Education, 86, 1–37.

31.

Dovidio

J. F.

Gaertner

S. L.

Stewart

T. L.

Esses

V. M.

Vergert

Hodson

(2004). From intervention to outcome: Processes in the reduction of bias. In Stephan

W. G.

Vogt

W. P.

(Eds.), Education programs for improving intergroup relations (pp. 243–265). New York, NY: Teachers College Press.

32.

Engberg

M. E.

(2007). Educating the workforce for the 21st century: A cross-disciplinary analysis of the impact of the undergraduate experience on students’ development of a pluralistic orientation. Research in Higher Education, 48, 283–317.

33.

Engberg

M. E.

Hurtado

(2011). Developing pluralistic skills and dispositions in college: Examining racial/ethnic group differences. The Journal of Higher Education, 82, 416–443.

34.

Feldman

K. A.

Newcomb

T. M.

(1969). The impact of college on students. San Francisco, CA: Jossey-Bass.

35.

Festinger

(1957). A theory of cognitive dissonance. Stanford, CA: Stanford University Press.

36.

Guo

Fraser

M. W.

(2015). Propensity score analysis: Statistical methods and applications (2nd ed.). Los Angeles, CA: Sage.

37.

Gurin

(1999). Expert report. “Gratz et al. v. Bollinger, et al.” No. 97-75321 (E.D. Mich.); “Grutter, et al. v. Bollinger, et al.” No. 97-75928 (E.D. Mich.). Equity & Excellence in Education, 32(2), 36–62.

38.

Gurin

Dey

E. L.

Hurtado

Gurin

(2002). Diversity and higher education: Theory and impact on educational outcomes. Harvard Educational Review, 72(3), 330–367.

39.

Harackiewicz

J. M.

Canning

E. A.

Tibbetts

Giffen

C. J.

Hyde

J. S.

(2014). Closing the social class achievement gap for first-generation students in undergraduate biology. Journal of Educational Psychology, 106, 375–389.

40.

Heck

R. H.

Thomas

S. L.

(2008). An introduction to multilevel modeling techniques (2nd ed.). New York. NY: Routledge.

41.

Holmes

W. M.

(2013). Using propensity scores in quasi-experimental designs. Los Angeles, CA: Sage.

42.

Hong

Raudenbush

S. W.

(2005). Effects of kindergarten retention policy on children’s cognitive growth in reading and mathematics. Educational Evaluation and Policy Analysis, 27, 205–224.

43.

Hong

Raudenbush

S. W.

(2006). Evaluating kindergarten retention policy: A case study of causal inference for multilevel observational data. Journal of the American Statistical Association, 101, 901–910.

44.

Hurtado

(2005). The next generation of diversity and intergroup relations research. Journal of Social Issues, 61, 595–610.

45.

Hyun

(1994). Helping to promote racial understanding: Does it matter if you’re Black or White? Paper presented at the annual meeting of Association of the Study of Higher Education, Tucson, AZ.

46.

Jayakumar

U. M.

(2008). Can higher education meet the needs of an increasingly diverse and global society? Campus diversity and cross-cultural work competencies. Harvard Educational Review, 78, 615–651.

47.

Katz

J. H.

Ivey

(1977). White awareness: The frontier of racism awareness training. Personnel and Guidance Journal (April), 485–489.

48.

Kotori

(2009). College impact on civic attitudes of Asian American and White undergraduate students: A comparative study. Dissertation Abstracts International, 69(07), 263A. (UMI No. 3315512)

49.

Luke

D. A.

(2004). Multilevel modeling. Thousand Oaks, CA: Sage.

50.

Malcom

L. E.

Dowd

A. C.

(2012). The impact of undergraduate debt on the graduate school enrollment of STEM baccalaureates. The Review of Higher Education, 35(2), 265–305.

51.

Mayhew

M. J.

Rockenbach

A. N.

Bowman

N. A.

Seifert

T. A.

Wolniak

G. C.

with Pascarella

E. T.

Terenzini

P. T.

(2016). How college affects students (Vol. 3): 21st century evidence that higher education works. San Francisco, CA: Jossey-Bass.

52.

McCauley

Wright

Harris

M. E.

(2000). Diversity workshops on campus: A survey of current practice at U.S. colleges and universities. The College Student Journal, 34(1), 100–114.

53.

Melguizo

Kienzl

G. S.

Alfonso

(2011). Comparing the educational attainment of community college transfer students and four-year college rising juniors using propensity score matching methods. The Journal of Higher Education, 82(3), 265–291.

54.

Milem

J. E.

(1994). College, students, and racial understanding. Thought and Action, 9(2), 51–92.

55.

Milem

J. F.

Umbach

P. D.

Liang

C. T. H.

(2004). Exploring the perpetuation hypothesis: The role of colleges and universities in desegregating society. Journal of College Student Development, 45, 688–700.

56.

Miyake

Kost-Smoth

L. E.

Finkelstein

N. D.

Pollock

S. J.

Cohen

G. L.

Ito

(2010). Reducing the gender achievement gap in college science: A classroom study of values affirmation. Science, 330, 1234–1237.

57.

Nelson Laird

T. F

. (2005). College students’ experiences with diversity and their effects on academic self-confidence, social agency, and disposition toward critical thinking. Research in Higher Education, 46, 365–387.

58.

Nelson Laird

T. F.

Engberg

M. E.

Hurtado

. (2005). Modeling accentuation effects: Enrolling in a diversity course and the importance of social engagement. Journal of Higher Education, 76, 448–476.

59.

Orfield

(2009). Reviving the goal of an integrated society: A 21st century challenge. Los Angeles, CA: The Civil Rights Project/Proyecto Derechos Civiles at UCLA.

60.

Orfield

Kucsera

Siegel-Hawley

(2012). E pluribus. . . separation: Deepening double segregation for more students. Los Angeles, CA: The Civil Rights Project/Proyecto Derechos Civiles at UCLA.

61.

Park

J. J.

(2009). Are we satisfied? A look at student satisfaction with diversity at traditionally White institutions. The Review of Higher Education, 32(3), 291–320.

62.

Park

J. J.

Denson

(2009). Attitudes and advocacy: Understanding faculty views on racial/ethnic diversity. Journal of Higher Education, 80(4), 415–438.

63.

Pascarella

Edison

Nora

Hagedorn

Terenzini

(1996). Influences on students’ openness to diversity and challenge in the first year of college. Journal of Higher Education, 67, 174–195.

64.

Pascarella

E. T.

Salisbury

M. H.

Blaich

(2013). Design and analysis in college impact research: Which counts more? Journal of College Student Development, 54, 329–335.

65.

Petersen

(1985). A comment on presenting results from logit and probit models. American Sociological Review, 50, 130–131.

66.

Pettigrew

T. F.

Tropp

L. R.

(2006). A meta-analytic test of intergroup contact theory. Journal of Personality and Social Psychology, 90, 751–783.

67.

Piaget

(1971). The theory of stages in cognitive development. In Green

D. R.

Ford

M. P.

Flamer

G. B.

(Eds.), Measurement and Piaget (pp. 1–111). New York, NY: McGraw-Hill.

68.

Piaget

(1985). The equilibrium of cognitive structures: The central problem of intellectual development. Chicago, IL: University of Chicago Press. (Original work published 1975)

69.

Raudenbush

S. W.

Bryk

A. S.

(2002). Hierarchical linear models: Applications and data analysis methods (2nd ed.). Newbury Park, CA: Sage.

70.

Rosenbaum

P. R.

(2002). Observational studies (2nd ed.). New York, NY: Springer.

71.

Ruble

D. N.

(1994). A phase model of transitions: Cognitive and motivational consequences. In Zanna

M. P.

(Ed.), Advances in experimental social psychology (Vol. 26, pp. 163–214). San Diego, CA: Academic Press.

72.

Saenz

V. B.

(2010). Breaking the segregation cycle: Examining students’ precollege racial environments and college diversity experiences. Review of Higher Education, 34, 1–37.

73.

Saenz

V. B.

Ngai

H. N.

Hurtado

(2007). Factors influencing positive interactions across race for African American, Asian American, Latino, and White college students. Research in Higher Education, 48, 1–38.

74.

Schoem

Hurtado

Sevig

Chesler

Sumida

S. H.

(2001). Intergroup dialogue: Democracy at work in theory and practice. In Schoem

Hurtado

(Eds.), Intergroup dialogue: Deliberative democracy in school, college, community, and workplace (pp. 1–21). Ann Arbor, MI: The University of Michigan Press.

75.

Shadish

W. R.

Cook

T. D.

Campbell

D. T.

(2002). Experimental and quasi-experimental designs for generalized causal inference (2nd ed.). Boston, MA: Houghton Mifflin.

76.

Sherman

D. K.

Hartson

K. A.

Binning

K. R.

Purdie-Vaughns

Garcia

Taborsky-Barba

. . . Cohen

G. L

. (2013). Deflecting the trajectory and changing the narrative: How self-affirmation affects academic performance and motivation under identity threat. Journal of Personality and Social Psychology, 104, 591–618.

77.

Smith

(2009). Diversity’s promise for higher education: Making it work. Baltimore, MD: Johns Hopkins University Press.

78.

Sniderman

P. M.

Crosby

G. C.

Howell

W. C.

(2000). The politics of race. In Sears

D. O.

Sidanius

Bobo

(Eds.), Racialized politics: The debate about racism in America (pp. 236–279). Chicago, IL: University of Chicago.

79.

Snijders

T. A. B.

Bosker

(2012). Multilevel analysis: An introduction to basic and advanced multilevel modeling (2nd ed.). Thousand Oaks, CA: Sage.

80.

Springer

Palmer

Terenzini

P. T.

Pascarella

E. T.

Nora

(1996). Attitudes toward campus diversity: Participation in a racial or cultural awareness workshop. The Review of Higher Education, 20(1), 53–68.

81.

Steiner

P. M.

Cook

T. D.

Shadish

W. R.

Clark

M. H.

(2010). The importance of covariate selection in controlling for selection bias in observational studies. Psychological Methods, 15, 250–267.

82.

Taylor

S. H.

(1994). Enhancing tolerance: The confluence of moral development with the college experience. Dissertation Abstracts International, 56(01), 114A. (UMI No. 9513291)

83.

Thomas

S. L.

Heck

R. H.

(2001). Analysis of large-scale secondary data in higher education research: Potential perils associated with complex sampling designs. Research in Higher Education, 42, 517–540.

84.

Thyer

B. A.

(2012). Quasi-experimental research designs. New York, NY: Oxford University Press.

85.

Valentine

J. C.

Cooper

(2003). Effect size substantive interpretation guidelines: Issues in the interpretation of effect sizes. Washington, DC: What Works Clearinghouse, U.S. Department of Education.

86.

Vogelgesang

L. J.

(2001). The impact of college on the development of civic values: How do race and gender matter? Paper presented at the annual meeting of the American Educational Research Association, Seattle, WA.

87.

Walton

G. M.

Cohen

G. L.

(2011). A brief social-belonging intervention improves academic and health outcomes among minority students. Science, 331, 1447–1451.

88.

Whitt

E. J.

Edison

M. I.

Pascarella

E. T.

Terenzini

P. T.

Nora

(2001). Influences on students’ openness to diversity in the second and third years of college. Journal of Higher Education, 72, 172–204.

89.

Yeager

D. S.

Walton

G. M.

(2011). Social-psychological interventions in education: They’re not magic. Review of Educational Research, 81, 267–301.

90.

Zuniga

Williams

E. A.

Berger

J. B.

(2005). Action-oriented democratic outcomes: The impact of student involvement with campus diversity. Journal of College Student Development, 46, 660–678.