The Impact of Learning Communities for Students in Developmental Education

Abstract

Community colleges play a vital role in postsecondary education, enrolling more than one in every three postsecondary students. While their importance has grown over the past 50 years, their students’ success rates remain low. Consequently, community college stakeholders are searching with mounting urgency for approaches that increase students’ success rates. “Learning communities” are one popular strategy. This article describes the results of one of the largest (n ≈ 7,000) independent randomized trials in higher education history—an evaluation of the effectiveness of learning communities for students in developmental education at six community colleges throughout the United States. The estimated overall average effects of learning communities are positive, although quite modest.

Keywords

learning community learning communities community college randomized trial program evaluation

Over the last five decades, community colleges have played an increasingly large role in American postsecondary education. In 1963, community colleges enrolled 740,000 students, representing 15% of all fall enrollment in postsecondary institutions (Provasnik & Planty, 2008). By 2011, enrollment had increased by nearly 1,000%, to 7.1 million students, representing 34% of all fall enrollment (Snyder & Dillow, 2013). This shift in the postsecondary landscape is due in part to the fact that community colleges are generally open-access institutions and are typically lower cost than 4-year colleges and universities. Tuition and fees at community colleges are around one third of their cost at public 4-year colleges and universities, and around one tenth of their cost at private 4-year colleges and universities (The College Board, 2012).

While community colleges have provided increased access to many students, particularly nontraditional, low-income, and minority students, the completion rates at these institutions leave much room for improvement. Among students beginning at public 2-year colleges, fewer than half earn a credential or transfer to a 4-year institution within 6 years after their initial enrollment (Radford, Berkner, Wheeless, & Shepherd, 2010).

One factor contributing to these low success rates is that many students arrive on campus academically unprepared for college-level coursework (Conley, 2010; Long & Boatman, 2013). One estimate of the prevalence of the need for developmental courses suggests that among students whose first institution attended was a community college, more than 60% took a remedial course at a postsecondary institution (Adelman, 2004). Unsurprisingly, degree or certificate attainment rates among students who need developmental education are even lower than that of the general population (Attewell, Lavin, Domina, & Levey, 2006). Students in need of developmental education are not just failing to earn a credential—the majority never complete the developmental course sequence to which they are referred (Bailey, Jeong, & Cho, 2010).

Prompted by these statistics, community college stakeholders are searching for approaches that increase students’ chances of earning a college credential—particularly for students referred to developmental education courses. “Learning communities” are one of the most popular strategies to help such students.¹

Learning Communities

Although definitions of learning communities vary, there is much agreement about their key elements. Learning communities involve the coenrollment of a cohort of students into two or more courses. Typically, the curricula of these courses are intentionally linked or integrated, sometimes around a theme. For example, a learning community with an overarching theme and title called “Poverty and Inequality” might link a developmental English course and a college-level sociology course. The courses could require that students learn various elements of essay writing and argumentative rhetoric in their English course and then use those techniques to write a paper in their sociology course exploring the relationship between income status and race. Learning communities tend to require instructor collaboration, which enables teachers to communicate about their shared students and to integrate the curriculum across courses through, for example, joint assignments and readings. Some definitions of learning communities also include a pedagogical component, usually focusing on “active engagement,” “active pedagogy,” and/or “collaborative learning.” Finally, some learning communities include add-on student supports, such as enhanced counseling, tutoring, or other student services (Minkler, 2002; Richburg-Hayes, Visher, & Bloom, 2008; Smith, 2001; Smith, MacGregor, Matthews, & Gabelnick, 2004).

In practice, learning communities vary from extremely basic (e.g., simply coenrolling students in two or more classes) to multifaceted (e.g., coenrollment, tightly integrated curriculum including joint assignments and assessments, strong faculty collaboration, and additional student support services). While there is a significant gap between idealized learning communities and many enacted learning communities, some form of learning communities operate at an estimated 400 or 500 colleges and universities throughout the United States (Smith, 2001; Smith et al., 2004). At community colleges, a national study reported that around half of the 288 colleges surveyed were operating at least one learning community (Center for Community College Student Engagement, 2012).

Why Learning Communities Are Expected to Work

Learning communities at community colleges have the potential to improve student outcomes through several mechanisms described briefly here. In Tinto’s (1975) framework on dropout from higher education, he posits,

It is the person’s normative and structural integration into the academic and social systems that lead to new levels of commitment. Other things being equal, the higher the degree of integration of the individual into the college systems, the greater will be his commitment to the specific institution and to the goal of college completion. (p. 96)

Yet, community colleges are attended largely by commuter students, many of whom contend with the competing demands of work, family obligations, and school (Brock & LeBlanc, 2005; Minkler, 2002). Consequently, for many students the only time they are on campus is when they attend class, limiting opportunities for integration. Some critics of Tinto’s model in the community college context claim that for 2-year-college students, particularly commuting students, demographics and external circumstances are the primary factors in dropout, rather than integration on campus (Bean & Metzner, 1985; Braxton, Sullivan, & Johnson, 1997). Critics note that for community college students, particularly students of color, Tinto’s idea of disconnecting from a previous community and connecting to a new college community is not applicable, as community college students typically attend while still in their home communities (Guiffrida, 2006). Therefore, the benefits of learning communities may accrue differently for students of different backgrounds. Recent work, however, finds that community college students, including students of color, experience a type of interconnected social and academic integration, which, while distinct from that of undergraduates at 4-year, residential universities, does play a role in students’ persistence and success (Deil-Amen, 2011; Karp, Hughes, & O’Gara, 2010).

By coenrolling students into two or more courses, learning communities are intended to foster stronger connections among students (Smith et al., 2004). Through sustained academic relationships among students and faculty, students are expected to feel more integrated into a community of peers and college life, leading to a greater level of commitment. In addition, integrating course materials may help students understand connections between disciplines and between what they are learning in school and their personal lives, and in doing so, engage students more deeply in learning. Curricular integration may be particularly effective when a developmental-level course and college-level course are paired in a learning community as this allows students to use the basic skills being taught in developmental classes in their college-level coursework. Broadly speaking, learning communities are theorized to lead to improved academic outcomes by fostering stronger connections among students and between students and faculty, integrating students into campus life, and providing a more engaging academic environment. This is hypothesized to improve students’ academic attainment and increase their likelihood of persisting in school.

Learning communities are a popular strategy for teaching students in need of developmental education and increasing their likelihood of moving on to college-level coursework. The social integration encouraged by coenrollment in multiple classes can be extremely important for academically underprepared students, who may be more marginalized from the campus community and have less campus knowledge (Person, Rosenbaum, & Deil-Amen, 2006). Moreover, the connection between linked developmental courses and college-level courses may bolster learning in each course (Rutschow & Schneider, 2011).

Evidence of the Effectiveness of Learning Communities

Over the last two decades, there has been a significant amount of research suggesting that learning communities have positive effects on students. Among the first of the studies published in a peer-reviewed journal was a study by Tinto and Russo (1994) of a Coordinated Studies Program (CSP) that found that students who participated in the CSP increased involvement in academic and social activities, reported more positive views of college, and persisted to the following semester at a higher rate.

Subsequent to Tinto and Russo’s findings, multiple studies of a variety of different learning communities tended to support the finding that learning communities have a positive effect on student engagement, retention, and academic progress (Engstrom & Tinto, 2008; Goldberg & Finkelstein, 2002; Gordon, Young, & Kalianov, 2001; Stassen, 2003; Taylor, Moore, MacGregor, & Lindblad, 2003; Tinto, 1997, 1998; Wilmer, 2009; Zhao & Kuh, 2004). These results were generally quite promising; however, the studies all had major limitations with respect to their designs’ ability to make causal claims about the effectiveness of learning communities.

A key limitation in all but one of these studies was the way in which a comparison group was selected. Generally, comparisons were made between students who were in learning communities and those who were not, despite the fact that students typically self-select into learning communities and are unlikely equivalent. Two studies controlled for a limited set of measured background characteristics in their statistical models, such as gender, race, and high school SAT/ACT scores; however, the students who chose to enroll in learning communities may still have differed from those who did not in other unmeasured ways, such as tenacity and motivation. With respect to establishing the causal effect of learning communities, the designs used in these past studies were quite limited, and none conducted analyses to determine the sensitivity of their findings to selection bias.

To our best knowledge, the first study not subject to selection bias was a random assignment evaluation of learning communities by Goldberg and Finkelstein (2002). They reported on an evaluation that randomly assigned 25 full-time Electronic Technician Certificate students—16 were randomly assigned to a team-taught learning community and 9 were randomly assigned to individually taught, unlinked classes. The study found positive impacts on student perceptions of academic and social integration but no impacts on behavioral outcomes, such as grade point averages or persistence to the second semester. This could be a result of lack of program effects, but it could also easily be a result of the lack of statistical power to detect meaningful impacts with such a small sample size.

The first large sample random assignment evaluation of learning communities was conducted by MDRC in 2003 to 2005 to evaluate the effectiveness of Kingsborough Community College’s (KCC) learning community program. Evaluators randomly assigned more than 1,500 students to either a program group, which had the opportunity to participate in the Opening Doors Learning Community, or a control group, which could enroll in the colleges’ usual unlinked courses. More than two thirds of the sample required developmental English at the start of the study (and the learning communities to which they were assigned included a developmental English course as one of the linked courses). Overall, the program had positive effects on students’ engagement, performance in the targeted developmental English course, and total number of credits earned 2 years after the study began. The evidence of positive impacts on persistence was less promising, with no increased persistence in the main semester after the learning community ended but some evidence of an effect three semesters after the program ended (Richburg-Hayes et al., 2008; Scrivener et al., 2008).

In part as a result of the positive early findings from the evaluation at KCC, MDRC launched the Learning Communities Demonstration—a large-scale evaluation funded through the National Center for Postsecondary Research of the effectiveness of six learning community models, each operated at a community college in the United States.² The demonstration was designed to obtain separate unbiased impact estimates at each college, each with enough precision that relatively small impacts could be detected with high confidence (sample sizes were more than 1,000 per college, yielding minimum detectable effect sizes below .20 at each college). This article synthesizes the results from this group of experiments, focusing on the effectiveness of learning community models that target students in developmental education. This synthesis represents one of the largest (n ≈ 7,000) independent random assignment evaluations in higher education history.

Objectives

This article examines the pooled overall average effect of six learning community programs on academic progress. The study participants were all academically underprepared for college-level coursework in either English or math. This means that they scored below a cut score, set by each college, on the colleges’ placement exam. In addition, the study investigates whether the effects of the six learning community programs vary significantly. That is, were the effects of these programs fairly homogeneous or were they different from college to college? As described in more detail below (and in past reports), the six colleges operated varying learning community models, they varied in their implementation of those models, they varied in their contexts, and they varied with respect to the types of students they serve; consequently, it is possible that the effects of learning communities might be large at some colleges and small or null at other colleges.

Because the six programs all operated in real-world school-based settings, compliance with the treatment was imperfect. While almost no control group members crossed over and experienced learning communities (less than 1%), around 29% of program group members were no-shows, that is, they were not enrolled in a learning community at the deadline for students to add or drop courses, despite being randomly assigned the opportunity to enroll in a learning community. This article focuses on the effect of the intent-to-treat (ITT), which includes these no-shows and thus provides an estimate of the effect of offering learning communities in a real-world school-based setting, where compliance is imperfect. Sensitivity analyses, which more closely approximate the effect of treatment-on-the-treated (TOT), are also described.

Method

Sample

The studied learning community programs are located at six community colleges throughout the continental United States: The Community College of Baltimore County (CCBC) in Maryland, Hillsborough Community College (Hillsborough) in Florida, Houston Community College (Houston) in Texas, KCC in New York, Merced College (Merced) in California, and Queensborough Community College (QCC) in New York. The institutions were selected by MDRC after a national search for community colleges that would reflect a range of experiences among students enrolled in developmental education learning communities across the country. Candidates were identified through networks and individuals very familiar with community colleges, including professional associations, professional development providers, and research organizations. MDRC conducted dozens of phone meetings and site visits to candidate colleges to assess whether colleges met the key criteria for selection. Criteria for selection included the requirement that each college exhibit a reasonably strong contrast between learning communities and standard developmental education classes; offered learning communities that coenrolled students in at least two courses and included at least some cross-curricular instruction; offered a core developmental subject (e.g., math or English) across the learning communities; demonstrated sufficient instructor and student interest in the program to meet study intake goals and to make random assignment possible; and agreed to cooperate with the researchers to facilitate data collection and study procedures. The original intent was to select a purposive sample among colleges that met these criteria, representing the range of developmental education learning communities in operation in community colleges, which tend to vary from what Stassen (2003) referred to as “least coordinated” to “most connected.” While individual examples of advanced learning communities were found within some colleges, it proved difficult to identify colleges that offered programs where this was the norm across most learning communities. Ultimately, the six colleges presented in this study represent a range of developmental education learning communities. None of the colleges had only learning communities of the “most connected” type, although all of the colleges had at least one or two. This article provides some information about the recruitment effort, the specific colleges, their learning community models, and their implementation; for more detailed accounts, see D. Bloom and Sommo (2005); Richburg-Hayes et al. (2008); Scrivener et al. (2008); Visher, Schneider, Wathington, and Collado (2010); Visher and Teres (2011); Weiss, Visher, and Wathington (2010); Weissman et al. (2011); and Weissman, Cullinan, Cerna, Safran, and Richman (2012).

At five of the six colleges described in this article, the studied learning community programs targeted students in need of developmental education only, rather than the colleges’ entire student body, and provided students with learning communities that included both developmental and college-level courses. Only at KCC were “college-ready” students, defined as those needing no developmental reading/writing courses, also included in the eligible population. This synthesis focuses on the effectiveness of learning communities that include a developmental course as part of the learning community; consequently, in this article KCC’s sample includes only those students with developmental English needs.³ Four of the six programs targeted students in need of developmental English (CCBC, Hillsborough, KCC, and Merced), whereas the other two programs targeted students in need of developmental math (Houston and QCC). A brief overview of each program model is provided in the first column of Table 1. A description of students’ characteristics is provided in the following section.

Table 1

Overview of Developmental Education Learning Communities, by College

Learning community program model	Summary of key implementation findings
The Community College of Baltimore County (Baltimore, MD)
Developmental reading or writing linked with a college-level course	There was a significant variation in implementation among the learning communities in the program, particularly in the areas of curricular integration and faculty collaboration. The Master Learner content also varied considerably by instructor
A faculty “Master Learner” sat in the college-level course and conducted a weekly, 1-hour, noncredit seminar on learning-to-learn in the context of that course
Hillsborough Community College (Tampa, FL)
Developmental reading linked with a student success course	Curricular integration was generally not emphasized by program leaders, although by the last semester of the evaluation, some instructors were attempting to align and integrate their courses
Success course focused on acclimation to college and study skills
Houston Community College (Houston, TX)
Developmental math linked with a student success course	Curricular integration was not initially emphasized by program leaders, but as the program evolved, most instructors worked with each other to create joint assignments
Success course focused on acclimation to college and study skills
Kingsborough Community College (Brooklyn, NY)
Developmental English linked with a college-level course in the student’s major and a one-credit freshman orientation course	The learning communities varied in their content, class sizes, and in the degree to which faculty integrated their courses. College leaders were consistently very supportive of the program
Program included enhanced advising, tutoring, and a textbook voucher
Merced College (Merced, CA)
Developmental writing linked with developmental reading or math, a college-level course, or a student success course	The program had a relatively high level of faculty collaboration, and many instructors created linked syllabi, visited each other’s classrooms, and discussed individual students’ progress. However, the degree of curricular integration varied among the learning communities
Links included cross-content themes and integrated assignments
Queensborough Community College (Queens, NY)
Developmental math linked with developmental or college-level English (fall 2007) or with a college-level course	Curricular integration varied, but by the second semester of the evaluation, more instructors collaborated on joint assignments and overarching themes

Source. Scrivener et al. (2008); Visher and Teres (2011); Weiss, Visher, and Wathington (2010); Weissman et al. (2011); and Weissman, Cullinan, Cerna, Safran, and Richman (2012).

Random Assignment Procedure and Student Characteristics

College staff invited students who had tested in developmental English or math to participate in the learning communities evaluation through letters, emails, and phone calls, as well as in person during the registration process. Students who attended an intake session on campus and agreed to take part in the study completed an informed consent form and a baseline survey (described below). After completing these forms, each student was assigned, at random, either to the program group, whose members had the opportunity to enroll in learning communities, or to the control group, whose members had the opportunity to receive the college’s standard services. The random assignment decision was controlled by MDRC. Each college’s research sample was comprised of three or four cohorts of students. Each cohort started at the beginning of subsequent semesters. For example, at Hillsborough, students in the first cohort were randomly assigned during the registration period for the fall 2007 semester, with the expectation that the program group would enroll in learning communities in fall 2007; the second cohort of students did so in spring 2008, and the third in fall 2008. In addition, three of the colleges included more than one campus in the study. The random assignment process occurred separately at each college (and campus, when applicable) for each cohort.

Table 2 provides descriptive background information on the study sample at baseline, by research group. Breaking the data up by research group shows the extent that the random assignment procedure yielded two groups that are statistically similar at baseline.

Table 2

Sample Members Baseline Characteristics by Research Group

	Program group	Control group	M difference	p value
Gender (%)				.5410
Male	43.5	42.8	0.7
Female	56.5	57.2	−0.7
Race/ethnicity^a (%)				.1030
Hispanic	34.6	34.9	−0.3
White	18.4	17.8	0.7
Black	33.5	31.8	1.7
Asian or Pacific Islander	6.6	7.2	−0.6
Other^b	3.0	4.0	−1.0
Missing	3.8	4.3	−0.5
Age (%)				.3800
20 years old and younger	71.6	70.1	1.4
21–25 years old	16.0	16.8	−0.8
26–30 years old	5.1	5.8	−0.7
31 and older	7.3	7.1	0.1
Single parent (%)				.3650
Yes	13.1	13.7	−0.6
No	69.7	70.2	−0.5
Missing	17.3	16.1	1.2
Has one or more children (%)				.6540
Yes	18.1	18.6	−0.5
No	78.6	77.7	0.8
Missing	3.4	3.7	−0.3
First person in family to attend college (%)				.6890
Yes	31.9	32.7	−0.7
No	62.3	61.9	0.4
Missing	5.7	5.4	0.4
Language other than English spoken regularly at home (%)				.3580
Yes	35.7	37.2	−1.5
No	61.1	59.5	1.7
Missing	3.1	3.3	−0.2
Financially dependent on parents (%)				.3690
Yes	39.7	41.1	−1.3
No	43.0	42.8	0.2
Missing	17.2	16.1	1.1
Highest grade completed (%)				.2470
11th grade or lower	12.7	13.5	−0.8
12th grade	83.1	81.7	1.4
Missing	4.2	4.8	−0.6
Diplomas/degrees earned^c (%)				.4300
General educational development (GED)	13.9	15.0	−1.1
High school diploma	79.2	77.4	1.8
Occupational/technical certificate	4.6	4.0	0.6
Two-year or higher degree	0.4	0.5	0.0
None of the above	2.1	2.5	−0.4
Missing	3.7	3.8	−0.1
Taken any college courses (%)				.4450
Yes	17.9	18.8	−1.0
No	78.4	77.2	1.3
Missing	3.7	4.0	−0.3
Sample size (total = 6,974)	3,983	2,991

Source. MDRC calculations from Baseline Survey data.

Note. Calculations for this table used all available data for the 6,974 sample members. Random assignment ratios vary across cohorts. Estimates are weighted to account for probability of being assigned to the treatment group. Distributions may not add to 100% because of rounding.

Respondents who said they are Hispanic and chose a race are included only in the Hispanic category. Respondents who said they are not Hispanic and chose more than one race are included only in the multiracial category.

“Other” race/ethnicity includes those who marked “Other,” more than one race, or Native American/Alaska Native.

Distributions do not add to 100% because categories are not mutually exclusive.

Mirroring national trends and each college’s student population, the majority of study participants were women. Study participants were primarily of traditional college age at baseline, and they were racially diverse—there was no racial majority. Most sample members did not have any children at the start of the study, and more than 3 in every 10 study participants reported being the first person in their family to attend college. Over a third of the overall sample spoke a language other than English regularly at home. As Table 2 shows, study participants came from a diverse set of backgrounds and experiences. It is worth noting that students’ background characteristics varied across the six colleges, which is one reason that it is interesting to examine whether program impacts varied among the colleges.

Table 2 also includes p values from a chi-square test of independence, which provides a sense of whether there were significant baseline differences in the characteristics of program and control group members. As Table 2 shows, the program and control group were very similar at the start of the study. Similar baseline tables are available for each individual college through past reports (Scrivener et al., 2008; Visher & Teres, 2011; Weiss et al., 2010; Weissman et al., 2011; Weissman et al., 2012). Across these six evaluations, random assignment generally did not yield systematic differences in observable baseline characteristics between program and control group members. As noted in Richburg-Hayes et al. (2008), Kingsborough’s sample had more statistically significant differences in baseline characteristics than one would expect to observe by chance alone; however, they conducted sensitivity analyses that yielded qualitatively similar findings and thus do not express concern with regard to potential bias in their impact estimator.

The precise population to which the sample of students and sample of colleges generalize is difficult to define due to the nonrandom sample selection at both the student and school levels; however, the institutions include a variety of programs, locations, and populations served, including nearly 7,000 students and 174 learning communities.

Measures

Student Background Data

As described above, students completed a short baseline survey on background characteristics immediately before being randomly assigned. These data were used to describe the sample, to examine the similarity between research groups at the onset of the study, and to identify students for subgroup analyses.

Student Records Data

Each college provided information on students’ academic outcomes (e.g., enrollment and credit accumulation) from student transcript records. This article presents transcript data outcomes during the “program” semester, when program group members were eligible to participate in learning communities, and in two subsequent “postprogram” semesters.

Description of Key Outcomes

We focus on three primary indicators of student academic progress: credits earned in a targeted subject area (English or mathematics), credits earned outside the targeted subject area, and total credits earned. The credit accumulation policies at the six colleges are generally comparable; at all six colleges, credits are granted for developmental courses, but these credits do not count toward a degree. For students to earn an associate’s degree at each college, they must complete at least 60 college-level credits.

Credits earned in the targeted subject area

The learning community programs all included at least one subject-specific developmental course, either mathematics or English, in their learning community link. The theory of change suggested that learning communities would help students progress through the targeted subject’s course sequence more quickly. To assess the program’s impact on this outcome, the number of credits attempted and earned in the targeted subject area was measured. For example, at those schools targeting developmental math, this measure sums the total number of math credits earned, including both developmental (which do not count toward a degree) and college-level credits (meaning credits that are degree-applicable).

Credits earned outside the targeted subject area

Focusing efforts on a specific developmental subject may also lead to improved progress in courses outside that subject area—for example, the skills learned in developmental English may lead to success in history or sociology courses. Conversely, focusing efforts on a targeted subject may slow progress outside the targeted subject if students (a) put off other required courses (substituting the targeted developmental course for a course they otherwise would have taken), or (b) focus their time and energy on the targeted subject to the detriment of performance in other subject areas. To assess the program’s effects on progress outside the targeted subject, the total number of credits earned (developmental and college level) outside the targeted subject is measured.

Total credits earned

Ultimately, the most important question we seek to address is whether learning communities improve students’ overall progress toward a degree. There is no perfect measure of progress toward a degree, only useful proxies. The proxy used here is calculated by summing together credits earned inside and outside the targeted subject area, including developmental and college-level credits. For students who place into developmental coursework, developmental courses may be required before taking certain required college-level courses, so these students must complete more than 60 credits to earn a degree. Generally speaking, earning more credits brings a student closer to earning a degree, so total credit accumulation serves as a very good indicator of overall progress toward a degree.

Readers may wonder why persistence (enrollment in subsequent semesters) is not listed as a primary/confirmatory indicator of the program’s effectiveness as it is a key mechanism by which learning communities are theorized to work. While we consider enrollment a key potential mediating variable due to its importance as it relates to the theory of change of learning communities, we posit that enrollment without credit accumulation does not necessarily indicate progress. Moreover, it is likely that a program that increases enrollment will also increase credit accumulation by changing unenrolled students into positive credit earners. Consequently, by examining credit accumulation we account for the realized observable benefits of persistence. The programs’ estimated impacts on enrollment are explored, but the main purpose of these analyses is to better understand one mechanism through which learning communities might improve students’ academic progress as measured by credit accumulation.

Implementation of the Intervention

Implementation research was conducted at all six colleges using a variety of observations, interviews, and other methods to collect data. College-specific findings from the implementation research can be found in detail in previously published reports (Richburg-Hayes et al., 2008; Scrivener et al., 2008; Visher et al., 2010; Visher & Teres, 2011; Weiss et al., 2010; Weissman et al., 2011; Weissman et al., 2012), and are concisely described in column 2 of Table 1.

To summarize, all six colleges successfully linked a targeted developmental subject course with another course (and in KCC’s case, two other courses) and enrolled a group of students in these linked courses. Although several of the colleges experienced initial challenges in course scheduling, setting up enrollment processes, and so on, in the end, the coenrollment of students into two or more courses was very well implemented across all colleges. Throughout the evaluations (even for the final cohorts), there was a significant variation in the amount of faculty collaboration within the colleges—often ranging from almost none to routine, weekly (and on occasion, daily) check-ins. Likewise, efforts to integrate course content by aligning syllabi and assignments across the courses varied dramatically among teaching teams within each college, although there is some evidence that this component became stronger over time at several of the colleges. Curricular alignment was the most challenging component of learning communities to implement consistently, and the one that was met with the least success. Finally, extra student supports were generally offered as intended, although support other than that provided in linked student success courses was fairly minimal (with the exception of KCC’s enhanced student services and book voucher).

Results

The main analyses presented here are ITT—comparing all program group students with all control group students, regardless of compliance.⁴ The estimates reflect the effect of being offered the opportunity to participate in learning communities. This is not necessarily the same as the effect of experiencing a learning community as only 71% of program group members participated in a learning community. A brief discussion of sensitivity analyses that were conducted among a subgroup of students with higher rates of compliance is provided at the end of the this section. The appendix describes the impact model used for analyses.

The results are divided into four sections. The first section describes the pooled overall average effect of the opportunity to participate in learning communities. The second section examines whether the programs’ effects vary among the six colleges studied (as well as other groupings). The third section discusses the sensitivity of results to program participation. The final section summarizes the results and discusses limitations.

Pooled Overall Average Effects

Overall, the learning communities had a half-credit impact (positive) on students’ total credit accumulation. This impact was driven by credits earned in the targeted subject, with no discernible effect on credits earned outside the targeted subject. The effect was maintained for three semesters, although it did not change after the first (program) semester. Table 3 presents the overall average effects of learning communities pooled across the six colleges.

Table 3

Pooled Academic Outcomes

	Program group		Control group
Outcome	M	SD	M	SD	M difference	p value
In the targeted subject^a
Cumulative credits attempted
First semester	4.13	3.05	3.57	3.17	0.56	.0000
Second semester	5.83	4.28	5.23	4.30	0.60	.0010
Third semester	6.72	4.96	6.19	5.04	0.53	.0060
Cumulative credits earned
First semester	2.72	3.08	2.20	3.04	0.52	.0000
Second semester	4.02	4.46	3.52	4.42	0.50	.0020
Third semester	4.69	5.06	4.25	5.12	0.44	.0110
Outside the targeted subject
Cumulative credits attempted
First semester	6.04	4.04	6.31	4.62	−0.27	.0440
Second semester	11.14	7.72	11.37	8.40	−0.23	.3220
Third semester	15.44	11.60	15.60	12.31	−0.17	.6150
Cumulative credits earned
First semester	4.19	3.92	4.19	4.33	0.00	.9940
Second semester	7.88	7.53	7.84	8.05	0.05	.8340
Third semester	10.87	11.00	10.85	11.67	0.01	.9660
Total credits
Cumulative credits attempted
First semester	10.18	5.67	9.88	6.02	0.29	.0990
Second semester	16.99	10.26	16.60	10.68	0.38	.2130
Third semester	22.18	14.74	21.81	15.36	0.37	.3920
Cumulative credits earned
First semester	6.91	5.74	6.38	5.81	0.53	.0020
Second semester	11.92	10.34	11.36	10.61	0.56	.0580
Third semester	15.58	14.31	15.10	14.9	0.47	.2280
Sample size (total = 6,974)	3,983		2,991

Source. MDRC calculations from the Community College of Baltimore County, Hillsborough Community College, Houston Community College, Kingsborough Community College, Merced College, and Queensborough Community College transcript data.

Note. “Targeted subject” credit measures and “outside the targeted subject” credit measures may not sum to “total” credit measures because Hillsborough Community College includes vocational credits in total credit measures. These vocational credits are not included in targeted subject credit measures or in outside the targeted subject credit measures. Cumulative measures exclude repeated courses. Estimates are adjusted by campus and cohort. Cluster-robust standard errors are used. A two-tailed test was applied to estimated differences between research groups.

Courses in the targeted subject for Kingsborough Community College, Hillsborough Community College, the Community College of Baltimore County, and Merced College are English courses and include both college-level and developmental credits. Courses in the targeted subject for Houston Community College and Queensborough Community College are math courses and include both college-level and developmental credits.

Credits Earned in the Targeted Subject Area

Learning communities had a positive effect on credit accumulation in the targeted subject (mathematics or English). During the first semester, program group students earned an average of 2.72 credits in the targeted subject, compared with 2.20 credits for control group students. These credits were almost entirely developmental. The 0.52 credit impact represents a 24% increase over the control group base (0.52 / 2.20). For context, most developmental courses in this study carried three or four credits, so the estimated impact represents around one sixth or one eighth of a typical developmental course.

The learning communities’ estimated effect on credits earned in the targeted subject during the first semester is fairly small in magnitude. Nonetheless, it is conceivable that even a small boost early on could grow into a larger impact over time. For this reason, students’ outcomes on this measure were tracked for an additional year after the program was completed to assess whether the impact grew, decreased, or remained the same. The effect was maintained through the third semester of study (two semesters after most program group students participated in learning communities), but it did not increase in magnitude.

Credits Earned Outside the Targeted Subject Area

During the first semester, learning community students attempted fewer credits (−0.27) outside the targeted subject than their control group counterparts, suggesting a small amount of “substitution” in students’ course enrollments. That is, learning communities induced students to attempt more credits in a targeted subject area; however, in doing so, on average, students attempted slightly fewer credits in untargeted subjects. Despite this substitution, the average number of untargeted credits earned was strikingly similar between program and control group students during the first semester. This occurred because program group students passed their untargeted credits at a slightly higher rate than their control group counterparts. This implies that the substitution did not hold students back with respect to their academic progress outside the targeted subject. Notably, while most of the courses in the targeted subject were at the developmental level, most of the untargeted courses were at the college level. Given the null finding on untargeted credits earned, it is unsurprising that the program also had no discernible effect on college-level credits alone (not shown in tables).

After the first semester, program and control group students attempted and earned untargeted credits at a similar pace. Despite the small positive effect on students’ progress in a targeted subject, generally speaking, the learning community programs had no discernible effect (negative or positive) on students’ progress outside the targeted subject.

Total Credits Earned

Summing together credits earned in the targeted and untargeted subjects yields students’ total credits earned. During the first semester, learning community students earned, on average, 0.53 total credits more than their control group counterparts. In the absence of learning communities, students earned an average of 6.38 total credits, so the increase of 0.53 represents an improvement of 8%. This impact on total credits earned was a result of students earning more credits in the targeted subject.

Over the two postprogram semesters, the magnitude of the estimated impact on total credits earned remained stable. The programs’ impact on total credits earned occurred during the first semester and the advantage was largely maintained through two postprogram semesters (although the cumulative effect estimate was no longer statistically significant at the end of three semesters).

Persistence

Although not considered a primary indicator of program success, we provide the programs’ estimated effect on persistence (as measured by enrollment in each postprogram semester) as part of the theory of change underlying learning communities is that they will increase students’ likelihood of persisting, and, in doing so, improve future academic outcomes such as credit accumulation. We do not find evidence of discernible effects on enrollment during the first three semesters of study.⁵ This finding helps explain why the impact of learning communities does not grow beyond the program semester.

In sum, during the first semester of study, learning communities helped students earn an additional half credit in their targeted subject. This benefit did not come at the expense of progress outside the targeted subject. After the first semester was complete, learning community students maintain the edge they gained during that semester, but otherwise their academic trajectory remains the same. After three semesters, students enrolled in learning communities were, on average, one half of a developmental credit ahead of students offered their colleges’ regular services.

Variation in Impacts by College

The pooled average impact of learning communities provides an overall estimate of the average effectiveness of the six programs studied. However, overall average impacts can mask differences in impacts across colleges, types of students, and so on. It is therefore useful to formally test for evidence that the learning community programs’ effects were heterogeneous. Policymakers and other decision makers have become increasingly interested not only in understanding what works but for whom and under what circumstances. A program whose effects vary substantially and unpredictably may be a risky option for decision makers. A program whose effects vary predictably may have implications for targeting services to identifiable subpopulations (Weiss, Bloom, & Brock, 2014).

As the random assignment of students occurred within each college, it is possible to obtain an unbiased estimate of the effect of learning communities at each college. Comparing the impact findings among colleges requires prudence—it is not possible to pin down the exact factor(s) that cause variation in impacts among colleges. Differences may be a result of a variety of factors, including student characteristics, context, the service contrast, or a combination of any of these. In addition, some variation in observed impact estimates is always expected due to estimation error. The below analyses examine whether the observed variation in impacts among colleges is greater than what is expected due to chance, if the average impacts were homogeneous among the colleges. If there is no strong evidence that the impacts vary across colleges, then the pooled results described above may provide a reasonable description of the average effect of learning communities at the six colleges in this study. If there is evidence that the programs’ impacts vary across the six colleges, then there is a good reason to believe that this type of program is more effective at some colleges and less effective at others, and it would be important to try to understand the reasons behind those differences.

Figures 1 to 3 display the estimated impacts on the three key outcomes of interest, for each college (labeled with black squares) as well as pooled across all six colleges (labeled with a white diamond). These figures provide a visual representation of the observed variation in the estimated impacts of learning communities at the six colleges.

Figure 1.

ITT impact estimates on credits earned in the targeted subject at the end of the first (program) semester, by college.

Figure 2.

ITT impact estimates on credits earned outside the targeted subject at the end of the first (program) semester, by college.

Figure 3.

ITT impact estimates on total credits earned at the end of the first (program) semester, by college.

Credits Earned in the Targeted Subject Area

Figure 1 displays each college’s estimated impact and 95% confidence interval on progress through the targeted subject during the first semester. For example, at Houston, the estimated impact is 0.43 credits, and the confidence interval indicates 95% confidence that the true impact lies between 0.22 and 0.64 credits.

With respect to credits earned in the targeted subject, there is clear evidence that the effect of learning communities varied by college (a joint F test yields p < .01). While the pooled average impact estimate is around 0.52, there is more to the story than the pooled impact estimate. At KCC, for example, the learning communities estimated impact is 1.49 credits, nearly 3 times the pooled average estimate; at two colleges (CCBC and Hillsborough), the estimated effects are significantly below the pooled average. This suggests that, with respect to credit accumulation in a targeted subject, learning communities have the potential to have a significantly larger or smaller impact than the pooled estimate. Signs of impact variation among colleges on cumulative credits earned in the targeted subject remain evident through three semesters (a joint F test yields p = .04). A discussion of the unique elements of KCC’s program, which may explain why KCC’s program appears particularly effective, is provided later in the “Discussion” section.

The six evaluations discussed in this article were not designed to disentangle competing explanations for cross-college impact variation. That noted, across the six colleges there is a striking relationship between the programs’ estimated impact on credits attempted (not shown in figures) and credits earned in the targeted subject—programs with larger impacts on credits attempted have larger impacts on credits earned (r > .95). With only six colleges, establishing a pattern is difficult; but there is clearly suggestive evidence, which aligns with the pooled analyses, that an important mechanism through which learning communities impact credit accumulation in the targeted subject is by leading students to attempt more credits in the targeted subject.

Credits Earned Outside the Targeted Subject Area

Figure 2 displays each college’s estimated impact and 95% confidence interval on credits earned outside the targeted subject during the first semester. There is little observed variation in estimated impacts across colleges on this outcome—the amount of observed variation is no greater than what is expected by chance (a joint F test yields p = .26). Across the six colleges, learning communities had a fairly homogeneous average effect on students’ progress outside the targeted subject. In other words, the pooled impact estimate basically tells the full story—there is no strong evidence that these programs help (or harm) students with respect to credit accumulation outside the targeted subject.

Total Credits Earned

Figure 3 displays each college’s estimated impact and 95% confidence interval on total credits earned during the first semester. This outcome is our proxy for students’ overall progress. During the first semester, students offered the opportunity to participate in learning communities earned an average of 0.53 credits more than similar students without that opportunity. In Figure 3, there is some appearance that the impact estimates vary by college; however, the amount of observed variation in impacts is not greater than what is expected by chance if the true impacts were the same across all colleges (a joint F test yields p = .19).

Given that there is no strong evidence that the programs’ effects vary across colleges (and each college’s estimate is fairly precise), the pooled average effect provides a useful estimate of the average effect of learning communities at these six colleges—masking limited differences in average impacts among the colleges. It is important to be aware that the programs’ effects still may vary significantly within colleges based on student characteristics, fidelity of implementation, control group services received, and so on; however, this analysis suggests that among these six colleges the programs’ average effects on total credits earned were fairly homogeneous.

Additional Analyses of Impact Variation

We also investigated whether the effects of learning communities varied by students’ background characteristics. Two primary subgroups were specified prior to conducting analyses for this article: race by gender and recent high school graduate. These variables were selected chiefly because of their known associations with academic outcomes (Green & Radwin, 2012), rather than because of a strong theory why learning communities are expected to work better for one group compared with another. While these background characteristics did correlate with indicators of academic success, evidence suggests that the effects of learning communities did not vary significantly by these characteristics.

Additional exploratory subgroup analyses were conducted on four subgroups that were not prespecified, but were probed for hypothesis generation: first in family to attend college, usually speaks a language other than English at home, single parent, and parents pay for more than half of expenses. Across these subgroups, there was little evidence of discernible impact variation, with the exception that the program appeared more effective for students who were financially dependent on their parents (i.e., students who reported that their parents pay for more than half their expenses). This result is intriguing, although we are hesitant to draw too much from this finding, given the number of exploratory subgroups we examined. This is a discovery that future researchers of learning communities are encouraged to explore further.

Finally, we note that although all six colleges had conducted learning communities prior to the evaluation, every college (including those with the most prior experience) needed to scale up their developmental learning communities and offer more sections to participate in the study. As one of the main implementation findings at several colleges was that their developmental learning communities improved over the course of scaling up during the evaluation, we explored the hypothesis that the effects of learning communities varied within colleges between cohorts. If, broadly speaking, better implemented programs yield larger impacts, impact variation might be observed among cohorts as the quality of program implementation changed over time. Across the three key outcomes in this study, there was no strong evidence supporting the hypothesis that as the programs matured, the estimated impacts improved. This provides some indication that the fairly modest estimated impacts are likely not a result of programs being studied too early in their implementation and scaling cycle.

Sensitivity of Results to Participation Rates

An important consideration when examining program effects from a randomized experiment is the level of program participation. Across the six colleges, 71% of program group members were coenrolled in classes that made up a learning community at the colleges’ add/drop deadline (suggesting that they received a significant dose of the program). The main analyses presented in this article are ITT, so they do not necessarily reflect the impact of experiencing the program. Many readers may be interested in the effect of the program on those who participate, or the treatment-on-treated. While researchers often estimate TOT (or local average treatment effect [LATE]) using instrumental variables (IVs) analysis, the assumptions required for such analyses are likely violated in this study (Angrist, Imbens, & Rubin, 1996; H. S. Bloom, 1984). Specifically, such analyses require the assumption that the treatment has no impact on the “no-shows” (or “never-takers”), those randomly assigned to the program who did not experience learning communities. In this study, this is a strong assumption because program participation is defined several weeks into the semester, at a point when the program may already have affected some student’s enrollment decisions or academic outcomes. In addition, our best measure of program participation is defined as coenrolling in a learning community link, but some colleges offered additional program services, which a student might benefit from even if not coenrolled in a learning community link. In other words, the “exclusion restriction” may not hold, so such analyses would rely on major assumptions.

To come closer to estimating the effect of learning communities on those students who experienced learning communities, without such strong assumptions, analyses were conducted on a subset of the full sample of students—those who were enrolled in at least one class at the add/drop deadline of the first semester. The percentage of program and control group members enrolled at the add/drop deadline is statistically indistinguishable, so it is reasonable to expect that these groups were similar at the outset of the study (the weaker assumption required for these analyses to be internally valid). This subset of students is of interest because they had a significantly higher rate of program take-up (84%, compared with 71% for the full sample). As a result, analyses using this subset of students provide a closer estimate of the effect of the TOT. These sensitivity analyses confirm the pooled overall findings and result in almost no substantive changes in the magnitude of the pooled overall average impact estimates. This provides some evidence that the pooled overall analyses presented in this report are not too different than the effect of learning communities on those who participate in them.

Unlike the ITT analyses that suggested the programs’ effect on total credits earned were fairly homogeneous across colleges, the sensitivity analyses suggest that the programs’ effects do vary across colleges on total credits earned during the first semester. However, by the end of the third semester, there is again not meaningful evidence of variation in impacts among colleges on total credits earned.

Limitations

Before discussing the implications of the findings, we would like to point out a few important limitations to this research.

Limited External Validity

This study examines six learning community programs implemented at six community colleges. The randomized design offers a major improvement over past studies with respect to internal validity (for a definition, see Campbell & Stanley, 1963). We attempted to select a diverse group of colleges, operating in different contexts, serving diverse types of students, with programs representing a range of semester-long developmental education learning communities. However, the study was conducted on a nonrandom sample of six community colleges, so inferences regarding the impacts of learning communities at other colleges require speculation. Also, developmental learning communities at the colleges were offered with somewhat limited schedules; for example, very few learning communities were offered as night courses, so students only available for courses at these hours were unlikely to be represented in this sample.

Presence of Teacher Effects

In this study, random assignment was conducted at the student level, creating two similar groups of students for future comparisons. Due to practical restrictions, instructors were not randomly assigned; thus, the selection of instructors may have influenced the estimated effects of learning communities for better, for worse, or not at all. The presented effect estimates represent the combined effect of learning communities and the types of instructors delivering those services, or teacher effects, whereas we are most interested in the effects of learning communities alone (Weiss, 2010). To some extent, this concern is mitigated as many learning community instructors also taught stand-alone versions of these classes, sometimes to control group students. However, it is unclear how frequently control group students took classes with instructors who also taught the program group.

To the extent that the instructor selection mechanisms varied across colleges and cohorts and yet the impact estimates remained fairly constant, this may increase confidence that the effects described are primarily caused by the learning communities. Nonetheless, this limitation is real and noteworthy. It is also noteworthy that this concern exists in all past evaluations of learning communities, so the elimination of student selection bias through the randomized design presented in this article is a major improvement over past attempts to estimate the effects of learning communities, even with this limitation.

Learning Outcomes Not Measured

Finally, another limitation of this study is that we do not have any standardized measures (e.g., achievement tests) of student learning apart from course completion. This is a common limitation in most postsecondary education research. Nevertheless, as college certificates and degrees are awarded based on credit accumulation, the academic progress indicators used in this study are outcomes that are of critical importance to college administrators, policymakers, and researchers.

Discussion

Learning communities have become a popular strategy to improve the educational trajectories of students who are referred to developmental education. This article addresses the question of whether this intervention leads to better academic progress when compared with “business-as-usual” for students with developmental needs.

On average, the single-semester learning communities operated in the six community colleges in this study had a small positive effect on credit accumulation. This impact was achieved in the semester in which students enrolled in the learning communities, and it was driven by the impact of learning communities on credits accumulated in a targeted developmental subject area. In contrast to prior nonexperimental studies and the learning communities’ theory of change, this study found that these programs did not have a discernible effect on persistence.

The null findings of learning communities’ effect on persistence may give readers pause regarding the relationship between integration and retention in community colleges. MDRC’s learning communities’ evaluations were not designed to definitively answer questions about causal mediation, which may be more difficult to answer than is generally acknowledged (Bullock, Green, & Ha, 2010). It is possible that the learning communities in this study increased students’ feelings of integration, but that increased integration alone may not be sufficient to meaningfully change student success rates; this seems reasonable given the multiple barriers faced by many community college students (e.g., financial, balancing the demands of school, work, and family). Alternatively, these single-semester learning communities may not have provided the additional sense of integration needed to make a difference in community college students’ likelihood of persisting.

Because of the large sample size, we are able to examine whether program effects varied across colleges. The results of these analyses were mixed—there is clear evidence that the ITT impact estimates vary on targeted credits earned (with the impact at KCC being the largest), no evidence of impact variation on untargeted credits earned, and no evidence of discernible impact variation on total credit accumulation. Sensitivity analyses on a subset of students with higher participation uncover evidence of short-term variation in impacts among colleges on total credits earned, where again KCC has the largest estimated impact.

While long-term follow-up (beyond three semesters) data are unavailable for five of the six programs presented here, intriguingly, a recently released report on the effects of the program at KCC found that the early effects were sustained through 6 and 7 years (Sommo, Mayer, Rudd, & Cullinan, 2012; Weiss, Mayer, et al., 2014).

Given that there is no discernible evidence of cross-college impact variation on cumulative total credits earned after three semesters, it is debatable what to make of the appearance that KCC’s program has larger short-term effects, and the knowledge that KCC’s program has long-term impacts. The qualitative research conducted across these six colleges pointed out several unique features of the KCC program, which could explain the larger estimated effects at KCC. These include the fact that (a) KCC’s model included more enhancements in student services (e.g., counseling, tutoring); (b) students in KCC’s program were actively encouraged to enroll in 6-week “intersession” (i.e., winter/summer), which accounted for around one fourth of the 6-year impact on credits accumulated at KCC; (c) the sample at KCC was somewhat unusual for community colleges—students were more likely to be full-time and financially dependent on their parents—that is, they were more “traditional.” These types of students may benefit most from the extra support and structured cohort experience; this extra scaffolding of the transition into college may have served KCC’s student population particularly well; (d) the KCC program linked three courses rather than two; and (e) KCC’s program enjoyed unusually strong support from college administrators. Caution is required when trying to link these program features with the larger estimated impacts. The evidence is inconclusive on whether the effects truly vary across colleges—KCC’s larger impact estimate could simply be a result of estimation error. In addition, even if KCC’s program truly has larger effects than the other colleges, there are many plausible explanations for such differences; the above listed explanations are among them.

Conclusion

It is encouraging that, on average, the learning community programs benefited students in developmental education in the six community colleges studied. However, the impacts were small by most standards, and there was no discernible evidence that learning communities improved persistence—a key mechanism through which learning communities are hypothesized to benefit students. These findings may raise questions about the viability of learning communities as a key strategy for improving student outcomes at community colleges.

Advocates of learning communities may rightly point out that the learning community programs that were studied here were not the “ideal” learning communities where every faculty pair collaborates on an ongoing basis and the curricula are tightly integrated around a well-developed theme—it is possible (although untested) that such learning communities are very effective. Detractors of learning communities may rightly point out that this study examined the average effects of a variety of learning community models implemented across a diverse set of students, and the study found limited evidence that the programs’ effects varied, so perhaps the small estimated impacts are what can be expected by this type of intervention. Both of these views seem defensible, and we do not think they are inconsistent.

This research increases significantly the evidence which we hope college faculty and administrators, policymakers, and funders will use to improve student outcomes at community colleges. Evidence of the effectiveness of extremely well and consistently implemented learning community programs (i.e., unlike those studied here) remains limited, although through our implementation research it appears that achieving high levels of depth and breadth is a major challenge, and probably quite rare. Given the current evidence base, to the best of our knowledge learning community programs with well-implemented coenrollment and significant variation in implementation of the other key components have a small but positive benefit to students.

Footnotes

Appendix

To estimate the effect of being offered the opportunity to participate in learning communities, we use a generalized linear model including block dummy indicators (one for each unique college by cohort combination) as described by the following equation:

Y_{i j} = α_{j} + β T_{i j} + ε_{i j},

where $Y_{i j}$ is an outcome for student $i$ in random assignment block $j$ . College-by-cohort fixed effects are denoted by $α_{j}$ , and $T_{i j}$ is a treatment indicator set equal to 1 if the student was randomly assigned to the program group and 0 otherwise. Conservatively, cluster-robust standard errors are estimated to account for the potential lack of independence of errors caused by clustering in learning communities in the treatment arm (Weiss et al., 2010). Weights are used to account for varying random assignment ratios within college. Within each of the six colleges, the weights sum to the colleges’ total sample size, so that the pooled impact estimates are roughly weighted by the colleges’ sample size. Weights were obtained separately for each college and created to ensure that across blocks within colleges the random assignment ratio is constant. For ease of interpretation and presentation, tables and figures present the least squares means for program and control group members. College-by-college impact estimates were obtained through the use of treatment-by-college interaction terms. Similarly, subgroup analyses were conducted through the use of interaction terms.

Acknowledgements

The research described in this article could not have been conducted without considerable efforts of many people. Briefly, we thank the students and college staff who participated in these studies. In addition, we thank a long list of MDRC and Community College Research Center staff and the National Center for Postsecondary Research partnership, who served as senior advisors, draft reviewers, operations and qualitative researchers, data analysts, and resource managers.

Authors’ Note

The opinions expressed here are those of the authors and do not represent the views of the Institute for Educational Sciences or the U.S. Department of Education.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: the Institute for Educational Sciences, the Bill and Melinda Gates Foundation, the Ford Foundation, Lumina Foundation for Education, and the Robin Hood Foundation. Finally, the writing of this article was funded/supported by the Institute of Education Sciences, U.S. Department of Education, through Grant R305A100066-11 to MDRC.

1.

Portions of this article (in particular, the Introduction and background sections titled “Learning Communities,” “Why Learning Communities Are Expected to Work,” and “Evidence of the Effectiveness of Learning Communities”) are similar or identical to a recently published article sharing the same lead author: Weiss, Mayer, et al. (2014).

2.

The National Center for Postsecondary Research (NCPR) was established by a grant (R305A060010) from the Institute of Education Sciences of the U.S. Department of Education. The Learning Communities Demonstration was also supported by the Bill & Melinda Gates Foundation, the Ford Foundation, Lumina Foundation for Education, and the Robin Hood Foundation.

3.

In addition to scoring higher on the English placement test, the estimated effect of the program was larger for students deemed college-ready based on Kingsborough Community College’s (KCC) English placement test (Scrivener et al., 2008).

4.

Students who did not enroll are treated as having earned zero credits that semester. This could, theoretically, bias the impact estimator if program group members and control group members earned credits at different rates at nonstudy colleges. We believe that this concern is minimal because the programs did not have an effect on enrollment at the study colleges. Without an enrollment effect at the study colleges, a somewhat elaborate story must be concocted as to why the opportunity to participate in learning communities would nonetheless have a positive (or negative) effect on enrollment at other nonstudy colleges (i.e., Why would learning communities not effect enrollment at the initial college, yet have an effect on enrollment elsewhere?).

5.

Estimated effects on enrollment are 1.2, 1.3, and −0.3 percentage points in Semesters 1, 2, and 3, respectively. Control group enrollment rates are 84.1, 63.8, and 49.9, and standard error estimates are 1.2, 1.3, and 1.4, respectively.

Authors

MICHAEL J. WEISS, PhD, is a senior researcher at MDRC. His research work focuses on evaluating programs designed to improve community college students’ chances of achieving academic success. He is also deeply involved in methodological projects geared toward improving the quality of experimental evaluations.

MARY G. VISHER, PhD, is a senior researcher at MDRC. She has worked in public policy research and analysis for over 25 years. For the last 15 years, she has designed and conducted a range of evaluations of innovative reforms at the K–12 and postsecondary levels. She is particularly interested in the role of career-technical education and new approaches to preparing students for college and career.

EVAN WEISSMAN, BA, is an operations associate at MDRC. His research is focused in the area of postsecondary education, illuminating ways to improve college access, persistence, and success for low-income and underprepared students. He is currently leading Aid Like a Paycheck, a project testing the impacts of innovative changes to financial aid disbursements. He has also designed and launched evaluations, provided technical assistance, and conducted qualitative research in a range of other social policy areas, including maternal and infant well-being, early childhood education, and welfare employment programs.

HEATHER WATHINGTON, PhD, is a research assistant professor at the Curry School of Education at the University of Virginia. Her scholarship focuses on the academic achievement of low-income students and students of color in postsecondary education, with a specific focus on understanding the educational contexts, levers, and practices that promote greater academic success for these students.

References

Adelman

(2004). Principal indicators of student academic histories in postsecondary education, 1972–2000. Jessup, MD: ED Pubs.

Angrist

Imbens

Rubin

(1996). Identification of causal effects using instrumental variables. Journal of the American Statistical Association, 91, 444–455.

Attewell

P. A.

Lavin

D. E.

Domina

Levey

(2006). New evidence on college remediation. Journal of Higher Education, 77, 886–924.

Bailey

Jeong

D. W.

Cho

S.-W.

(2010). Referral, enrollment, and completion in developmental education sequences in community colleges. Economics of Education Review, 29, 255–270.

Bean

Metzner

(1985). A conceptual model of nontraditional undergraduate student attrition. Review of Education Research, 55, 485–540.

Bloom

Sommo

(2005). Building learning communities: Early results from the opening doors demonstration at Kingsborough Community College. New York, NY: MDRC.

Bloom

H. S.

(1984). Accounting for no-shows in experimental evaluation designs. Evaluation Review, 8, 225–246. doi:10.1177/0193841x8400800205

Braxton

J. M.

Sullivan

A. S.

Johnson

R. M.

(1997). Appraising Tinto’s theory of college student departure. In Smart

J. C.

(Ed.), Higher education: Handbook of theory and research (Vol. 12, pp. 107–164). New York, NY: Agathon.

Brock

LeBlanc

(2005). Promoting student success in community college and beyond: The opening doors demonstration. New York, NY: MDRC.

10.

Bullock

Green

(2010). Yes, but what’s the mechanism? (Don’t expect an easy answer). Journal of Personality and Social Psychology, 98, 550–558.

11.

Campbell

D. T.

Stanley

J. C.

(1963). Experimental and quasi-experimental designs for research on teaching. In Gage

N. L.

(Ed.), Handbook of research on teaching (pp. 171–247). Chicago, IL: Rand McNally.

12.

Center for Community College Student Engagement. (2012). A matter of degrees: Promising practices for community college student success (A first look). Austin: The University of Texas at Austin, Community College Leadership Program.

13.

The College Board. (2012). Trends in college pricing 2012. New York, NY: Author.

14.

Conley

D. T.

(2010). Eligible and ready for college. Principal Leadership, 11, 18–22.

15.

Deil-Amen

(2011). Socio-academic integrative moments: Rethinking academic and social integration among two-year college students in career-related programs. Journal of Higher Education, 82, 54–91.

16.

Engstrom

C. M.

Tinto

(2008). Learning better together: The impact of learning communities on the persistence of low-income students. Opportunity Matters, 1, 5–21.

17.

Goldberg

Finkelstein

(2002). Effects of a first-semester learning community on nontraditional technical students. Innovative Higher Education, 26, 235–249.

18.

Gordon

T. W.

Young

J. C.

Kalianov

C. J.

(2001). Connecting the freshman year experience through learning communities: Practical implications for academic and student affairs units. College Student Affairs Journal, 20, 37–47.

19.

Green

Radwin

(2012). Characteristics of associate’s degree attainers and time to associate’s degree (NCES 2012-271). Washington, DC: U.S. Department of Education.

20.

Guiffrida

D. A.

(2006). Toward a cultural advancement of Tinto’s theory. The Review of Higher Education, 29, 451–472.

21.

Karp

M. M.

Hughes

K. L.

O’Gara

(2010). An exploration of Tinto’s integration framework for community college students. Journal of College Student Retention: Research, Theory & Practice, 12, 69–86.

22.

Long

B. T.

Boatman

(2013). The role of remedial and developmental courses in access and persistence. In Perna

L. W.

Jones

A. P.

(Eds.), The state of college access and completion: Improving college success for students from underrepresented groups (pp. 77–95). New York, NY: Routledge.

23.

Minkler

(2002). ERIC review: Learning communities at the community college. Community College Review, 30(3), 46–63.

24.

Person

A. E.

Rosenbaum

J. E.

Deil-Amen

(2006). Student planning and information problems in different college structures. Teachers College Record, 108, 374–396.

25.

Provasnik

Planty

(2008). Community colleges: Special supplement to the condition of education 2008. Statistical analysis report (NCES 2008-033). National Center for Education Statistics. Retrieved from http://nces.ed.gov/help/orderinfo.asp

26.

Radford

A. W.

Berkner

Wheeless

S. C.

Shepherd

(2010). Persistence and attainment of 2003-04 beginning postsecondary students: After 6 years. First look (NCES 2011-151). National Center for Education Statistics. Available from http://nces.ed.gov/

27.

Richburg-Hayes

Visher

Bloom

(2008). Do learning communities effect academic outcomes? Evidence from an experiment in a community college. Journal of Research on Educational Effectiveness, 1, 33–65.

28.

Rutschow

E. Z.

Schneider

(2011). Unlocking the gate: What we know about improving developmental education. New York, NY: MDRC.

29.

Scrivener

Bloom

LeBlanc

Paxson

Rouse

C. E.

Sommo

(2008). A good start: Two-year effects of a freshmen learning community program at Kingsborough Community College. MDRC. Available from http://www.mdrc.org

30.

Smith

B. L.

(2001). The challenge of learning communities as a growing national movement. Peer Review, 3(4), 4–8.

31.

Smith

B. L.

MacGregor

Matthews

Gabelnick

(2004). Learning communities: Reforming undergraduate education. San Francisco, CA: Jossey-Bass.

32.

Snyder

T. D.

Dillow

S. A.

(2013). Digest of education statistics 2012. Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education.

33.

Sommo

Mayer

A. K.

Rudd

Cullinan

(2012). Commencement day: Six-year effects of a freshman learning community program at Kingsborough Community College. New York, NY: MDRC.

34.

Stassen

M. L. A.

(2003). Student outcomes: The impact of varying living-learning community models. Research in Higher Education, 44, 581–613.

35.

Taylor

Moore

W. S.

MacGregor

Lindblad

(2003). Learning community research and assessment: What we know now (National Learning Communities Project Monograph Series). Olympia: Washington Center, The Evergreen State College.

36.

Tinto

(1975). Dropout from higher education: A theoretical synthesis of recent research. Review of Education Research, 45, 89–125.

37.

Tinto

(1997). Classrooms as communities: Exploring the educational character of student persistence. Journal of Higher Education, 68, 599–623.

38.

Tinto

(1998). Learning communities and the reconstruction of remedial education in higher education. New York, NY: Ford Foundation and U.S. Department of Education.

39.

Tinto

Russo

(1994). Coordinated studies programs: Their effect on student involvement at a community college. Community College Review, 22(2), 16–25.

40.

Visher

Schneider

Wathington

Collado

(2010). Scaling up learning communities: The experience of six community colleges. National Center for Postsecondary Research. Available from http://www.postsecondaryresearch.org/

41.

Visher

Teres

(2011). Breaking new ground: An impact study of career-focused learning communities at Kingsborough Community College. National Center for Postsecondary Research. Available from http://www.postsecondaryresearch.org/

42.

Weiss

M. J.

(2010). The implications of teacher selection and the teacher effect in individually randomized group treatment trials. Journal of Research on Educational Effectiveness, 3, 381–405.

43.

Weiss

M. J.

Bloom

H. S.

Brock

(2014). A conceptual framework for studying the sources of variation in program effects. Journal of Policy Analysis and Management, 33, 778–808.

44.

Weiss

M. J.

Mayer

Cullinan

Ratledge

Sommo

Diamond

(2014). A random assignment evaluation of learning communities at Kingsborough Community College—Seven years later. Journal of Research on Educational Effectiveness. Advance online publication. doi:10.1080/19345747.2014.946634

45.

Weiss

M. J.

Visher

Wathington

(2010). Learning communities for students in developmental reading: An impact study at Hillsborough Community College. National Center for Postsecondary Research. Available from http://www.postsecondaryresearch.org/

46.

Weissman

Butcher

K. F.

Schneider

Teres

Collado

Greenberg

D. H.

(2011). Learning communities for students in developmental math: Impact studies at Queensborough and Houston Community Colleges. National Center for Postsecondary Research. Available from http://www.postsecondaryresearch.org/

47.

Weissman

Cullinan

Cerna

Safran

Richman

(2012). Learning communities for students in developmental English: Impact studies at Merced College and the Community College of Baltimore County. National Center for Postsecondary Research. Available from http://www.postsecondaryresearch.org/

48.

Wilmer

(2009). The influence of learning communities on the interaction levels of developmental English students. The Journal of the Virginia Community Colleges, 14, 55–67.

49.

Zhao

C.-M.

Kuh

G. D.

(2004). Adding value: Learning communities and student engagement. Research in Higher Education, 45, 115–138.