Should Students Assessed as Needing Remedial Mathematics Take College-Level Quantitative Courses Instead? A Randomized Controlled Trial

Abstract

Many college students never take, or do not pass, required remedial mathematics courses theorized to increase college-level performance. Some colleges and states are therefore instituting policies allowing students to take college-level courses without first taking remedial courses. However, no experiments have compared the effectiveness of these approaches, and other data are mixed. We randomly assigned 907 students to (a) remedial elementary algebra, (b) that course with workshops, or (c) college-level statistics with workshops (corequisite remediation). Students assigned to statistics passed at a rate 16 percentage points higher than those assigned to algebra (p < .001), and subsequently accumulated more credits. A majority of enrolled statistics students passed. Policies allowing students to take college-level instead of remedial quantitative courses can increase student success.

Keywords

higher education corequisite remediation mathematics randomized controlled trial

Colleges in the United States assess a total of about 60% of their new freshmen as unprepared for college-level work (Grubb et al., 2011), most often in mathematics (Attewell, Lavin, Domina, & Levey, 2006). College policies usually require such students to complete remedial courses prior to taking college-level courses in the remedial courses’ disciplines, based on the purported theory that students need to pass the remedial courses to be able to pass the college-level courses. However, the percentage of students successfully completing remedial courses is low (Bailey, Jeong, & Cho, 2010). For example, at The City University of New York (CUNY) in fall 2014, 76% of new community college freshmen were assessed as needing remedial mathematics (CUNY, Office of Institutional Research and Assessment, 2015b), and the pass rate in the highest level remedial mathematics course across the community colleges was 38% (CUNY, Office of Institutional Research and Assessment, 2015c). Furthermore, at CUNY and nationally, many students, though assigned to remedial courses, wait to take them or never take them, delaying or preventing graduation (Bailey et al., 2010). It is therefore not surprising that students who enter college needing any remedial courses are less likely to graduate than are students who enter college with no such need (7% vs. 28% after 3 years at CUNY for students who entered CUNY community colleges in 2011; CUNY, Office of Institutional Research and Assessment, 2015a). Successful completion of mathematics remediation may be the single largest barrier to increasing graduation rates (Attewell et al., 2006; Complete College America, 2012).

Addressing the low pass rates in remedial mathematics courses could not only help overall graduation rates but could also help close performance gaps. Students assessed as needing remediation are more likely to be members of underrepresented groups (Attewell et al., 2006). Therefore, low mathematics remediation pass rates contribute to the lower college attainment rates of members of underrepresented groups.

Various solutions to the low remedial course pass rates have been proposed at CUNY and nationwide. One alternative is having students address remedial needs in the summer before entering college. Although there is research supporting this type of approach (Douglas & Attewell, 2014), a randomized controlled trial found only modest positive effects in the first year following the summer program, and these positive effects did not persist (Barnett et al., 2012). Also, not all students can attend remedial courses the summer before college.

Another example is the CUNY Start program in which students with multiple remedial needs postpone initial matriculation for one semester while engaging in full-time remediation. However, this program is only for students with severe remedial needs; not every student can devote an entire semester to remediation; and, although CUNY Start’s initial results are promising, there has not yet been an experiment evaluating it (Office of Academic Affairs, 2013).

The Carnegie Foundation for the Advancement of Teaching has promoted the use of Statway, which combines remedial mathematics with introductory statistics. A recent rigorous analysis supports Statway as increasing student success (Yamada, 2014). However, Statway can require a full academic year to obtain credits for one college-level course and requires students to know much of elementary algebra. Furthermore, the effects on enrollment of students being assigned to such a course are unknown.

Alternatively, some practitioners have advocated streamlining the remedial mathematics curriculum so that students learn only the remedial mathematics that they need for subsequent courses. However, only descriptive data are available for evaluating such approaches (Kalamkarian, Raufman, & Edgecombe, 2015).

As a form of streamlining, some colleges and states are instituting policies in which students assessed as needing remedial courses take college-level courses such as statistics instead, sometimes with additional academic support (e.g., Hern, 2012; Smith, 2015). Several theories have been suggested regarding why such approaches should be effective. First, at least some students assessed as needing remediation should perform satisfactorily in college-level courses because placement mechanisms are sometimes inaccurate, assessing some students as needing remediation even though their skills are sufficient for college-level work (Scott-Clayton, Crosta, & Belfield, 2014). Second, assigning a student to a remedial course may decrease that student’s motivation due to college graduation being more distant and/or because the student already had an unpleasant experience with this course in high school and/or because of the stigma of being required to take a remedial course (see, for example, Bailey, 2009; Complete College America, 2011; Goldrick-Rab, 2007; Logue, 1995; Scott-Clayton & Rodriguez, 2012). Third, it has been proposed that students can pass college-level statistics more easily than remedial algebra because the former is less abstract and uses everyday examples (Burdman, 2013; Yamada, 2014).

There have been multiple attempts to compare the performance of students, assessed as needing remediation, who enroll first in remedial courses with the performance of students who enroll directly in college-level courses. Some of this research has used data obtained from naturally occurring variation in course placement, and some has used quasi-experimental methods such as propensity score matching and regression discontinuity. Results have been mixed. Some studies have found that students assessed as needing remediation perform better in college-level courses if they first take remedial courses (e.g., Bettinger & Long, 2009; Moss, Yeaton, & Lloyd, 2014). Others have found that such students do just as well or better in completing college if they skip remediation (e.g., Boatman, 2012; Calcagno & Long, 2008; Clotfelter, Ladd, Muschkin, & Vigdor, 2015; Jaggars, Hodara, Cho, & Xu, 2015; Martorell & McFarlin, 2011). Still others have found both types of results (e.g., Melguizo, Bos, & Prather, 2011; Wolfle & Williams, 2014).

The term mainstreaming has been used to describe placing students assessed as needing remediation directly into a college-level course (see, for example, Edgecombe, 2011; such students are not necessarily mixed within the classroom with other students, as occurs with mainstreaming in K–12 education). There have been several apparently successful programs for mainstreaming college students assessed as needing remediation, sometimes with additional instructional support (e.g., an English program at Community College Baltimore County, and a mathematics program at Austin Peay State University; Jones, 2014).

The concern with all of these studies is that, because none of them have used experimental methods (i.e., randomized controlled trials), there could have been uncontrolled, unmeasured differences in some variables across the groups of students exposed to different treatments (as in some propensity score matching studies) and/or the findings could be limited to a narrow range of students (as in some regression discontinuity studies). For example, student motivation, which is difficult to measure, may vary across groups of students who are not randomly assigned to remedial and college-level courses. Such differences could help explain the inconsistent results across studies.

Our research’s purpose was therefore to use a randomized controlled trial to examine a promising approach for overcoming the block to college progress posed by mathematics remediation: mainstreaming. The experiment compared academic performance (pass rates) in remedial elementary algebra with a college-level course (statistics) for students assessed as needing remedial elementary algebra. Most (55.69%) of the students who took the college-level course (statistics) passed that course. Furthermore, students assigned to statistics passed at a rate that was 16 percentage points greater, and subsequently accumulated more credits, than students assigned to elementary algebra. Students do not first have to pass remedial mathematics to pass college-level statistics, and policies placing students assessed as needing remedial mathematics directly into college-level quantitative courses can increase student success.

Design of Present Research

For purposes of sample size and generalizability, we conducted the experiment at three CUNY community colleges (Colleges A, B, and C), one each in the boroughs of the Bronx, Manhattan, and Queens. At all three, we randomly assigned students assessed as needing remedial elementary algebra to one of three fall 2013 course types: (a) traditional, remedial, noncredit, elementary algebra (Group EA); (b) that course with weekly workshops (Group EA-WS); or (c) college-level, credit-bearing statistics with weekly workshops (Group Stat-WS).

Additional academic support has been termed supplemental or corequisite instruction (Bueschel, 2009; Complete College America, 2016). The present experiment used it for three reasons: (a) Evidence suggests that such support tends to increase students’ grades (e.g., Bettinger & Baker, 2014; Bowles, McCoy, & Bates, 2008), (b) CUNY policy requires that students assessed as needing remediation be provided with an intervention addressing that need, and (c) the additional support helped allay concerns that placing students assessed as needing remedial elementary algebra directly into college-level statistics with no additional support would result in even lower pass rates than those for elementary algebra.

These three groups allowed us to examine (a) the effects of adding workshops to elementary algebra by comparing Groups EA and EA-WS (we could not assess the effects of adding workshops to statistics given that we could not offer statistics without workshops); (b) the effects of exposing students to statistics as opposed to elementary algebra, each with workshops (by comparing Groups EA-WS and Stat-WS); and (c) the effects of placing students into statistics with workshops as compared with a traditional remedial course (by comparing Groups EA and Stat-WS). We could also compare the performance of the three experimental groups with the performance of all students taking elementary algebra and statistics in fall 2012, allowing us to compare our students’ performance with typical norms.

We hypothesized that the EA group would pass at the typical elementary algebra rate (fall 2012, 37%), that the EA-WS group would pass at a higher rate due to the positive effects of the workshops, and that the Stat-WS group would pass at a rate at least as high as the EA group although lower than the typical rate for statistics (fall 2012, 69%; because the Stat-WS students would be taking a college-level quantitative course without the assumed benefits of first taking elementary algebra, but with the benefits of the workshops and of being assigned to a college-level course). We also hypothesized that a higher pass rate would be associated with more credits accumulated in the year following the experiment because students who passed would have an opportunity to take more credit-bearing courses.

Participant Recruitment

During the summer prior to the fall 2013 semester, all eligible students at each participating college were notified of the research via email and during in-person orientation sessions for new students. At the orientation sessions, potential participants were given a flyer and a consent form stating the requirements for study participation (Appendices A and B, available in the online version of the journal, contain the text of College A’s flyer and consent form): minimum age 18, first-time freshman, intending to major in disciplines that did not require college algebra, and assessed as needing elementary algebra.¹ Participants could obtain a US$40 Metrocard for New York City public transportation if they were enrolled in their assigned research sections after the end of the course drop period (73% of participants retrieved them), and a US$10 Metrocard after the semester ended (35% retrieved them). We instructed recruiters to be neutral when describing the different treatment conditions to potential participants. However, recruitment flyers did state, “Benefits [of participation] include: A one-in-three chance to skip remediation in math and go directly to an enhanced college-level mathematics course.”

A total of 907 eligible students consented to the experiment (see Appendix C for the relevant power analysis, available in the online version of the journal). As soon as the consent form was signed, research personnel randomly assigned these students to one of the three course types (Groups EA, EA-WS, and Stat-WS) using random number tables created with MS Excel and informed students of their assignments, including their course sections. Recruitment took place during the 3 months before the start of the semester. As of the official course census date (approximately 2 weeks after the start of the semester, the day after the end of the drop period), 717 of these consenting students were enrolled in their assigned research sections and were designated the experiment’s participants. Figure 1 and Tables 1 and 2 provide information about all of the students involved in the experiment.

Figure 1.

Flow of target students through recruitment, random assignment, and treatment.

Table 1

Means [95% CIs] of Characteristics of Participants, Noncompliers, and Nonconsenters Who Took Elementary Algebra in Fall 2013

Student characteristic	Participants	Noncompliers	Nonconsenters
Age (years)	21.04 [20.64, 21.44]	21.62 [20.85, 22.38]	20.62 [20.32, 20.91]
Age missing	0.00 [0.00, 0.00]	0.18 [0.12, 0.23]	0.00 [0.00, 0.00]
Compass z score (algebra)	−0.00 [−0.07, 0.07]	0.01 [−0.08, 0.10]	−0.01 [−0.07, 0.05]
Compass score missing	0.08 [0.06, 0.10]	0.66 [0.59, 0.73]	0.20 [0.17, 0.23]
Days to consent	77.10 [75.52, 78.67]	69.32 [65.24, 73.41]^a *	N/A
First language (English)	0.56 [0.52, 0.60]	0.57 [0.52, 0.61]	0.53 [0.50, 0.56]
First language missing	0.00 [0.00, 0.00]	0.58 [0.51, 0.65]	0.00 [0.00, 0.00]
Gender (female)	0.54 [0.50, 0.58]	0.57 [0.50, 0.64]	0.57 [0.54, 0.60]
Gender missing	0.00 [0.00, 0.00]	0.05 [0.02, 0.09]	0.00 [0.00, 0.00]
High school GPA z score	−0.02 [−0.10, 0.05]	0.08 [−0.06, 0.23]	0.02 [−0.04, 0.08]
High school GPA z score missing	0.31 [0.28, 0.35]	0.35 [0.28, 0.42]	0.16 [0.14, 0.18]
Race (underrepresented)	0.87 [0.84, 0.89]	0.84 [0.80, 0.87]	0.76 [0.73, 0.78]^b *
Race missing	0.00 [0.00, 0.00]	0.61 [0.54, 0.68]	0.00 [0.00, 0.00]
N	717	190	1,179

Note. CI = confidence interval; GPA = grade point average.

Participants and noncompliers different.

Participants and nonconsenters different.

p < .05.

Table 2

Means [95% CIs] of Characteristics of Participants

Student characteristic	EA	EA-WS	Stat-WS
Age (years)	21.16 [20.41, 21.92]	21.55 [20.73, 22.38]	20.45 [20.00, 20.91]
Age missing	0.04 [0.02, 0.06]	0.05 [0.03, 0.08]	0.02 [0.00, 0.03]
Compass z score (algebra)	−0.00 [−0.10, 0.10]	−0.05 [−0.15, 0.05]	0.06 [−0.05, 0.16]
Compass score missing	0.18 [0.14, 0.23]	0.22 [0.17, 0.26]	0.20 [0.16, 0.25]
Days to consent	77.28 [74.51, 80.05]	78.55 [75.87, 81.23]	75.58 [72.83, 78.32]
First language (English)	0.56 [0.51, 0.62]	0.56 [0.51, 0.61]	0.56 [0.50, 0.61]
First language missing	0.13 [0.09, 0.17]	0.16 [0.12, 0.20]	0.07 [0.04, 0.10]
Gender (female)	0.51 [0.46, 0.57]	0.58 [0.52, 0.63]	0.55 [0.49, 0.61]
Gender missing	0.02 [0.01, 0.04]	0.00 [−0.00, 0.01]	0.01 [−0.00, 0.02]
High school GPA z score	0.07 [−0.04, 0.18]	−0.06 [−0.18, 0.05]	−0.00 [−0.12, 0.11]
High school GPA missing	0.33 [0.28, 0.38]	0.33 [0.28, 0.39]	0.30 [0.25, 0.35]
Instructor experience (years)	12.37 [11.42, 13.31]	12.07 [11.12, 13.03]	13.00 [11.96, 14.01]
Instructor has taught statistics	0.77 [0.73, 0.82]	0.76 [0.72, 0.80]	0.77 [0.73, 0.82]
Instructor has tenure	0.37 [0.32, 0.42]	0.38 [0.34, 0.43]	0.42 [0.37, 0.47]
Race (underrepresented)	0.87 [0.83, 0.90]	0.88 [0.85, 0.91]	0.84 [0.80, 0.88]
Race missing	0.13 [0.09, 0.17]	0.16 [0.12, 0.20]	0.09 [0.06, 0.12]
N	297	313	297

Note. CI = confidence interval; EA = elementary algebra; GPA = grade point average.

Figure 1 shows the flow of target students through each stage of the experiment. There was an overall attrition rate of 21% (190 students) between when students were randomized and the semester’s course census date. Attrition was significantly higher in Group EA-WS than in Groups EA or Stat-WS (Table 3). A Tukey post hoc test comparing attrition in Groups EA and Stat-WS was not significant, but tests comparing attrition between Group EA-WS with Groups EA and Stat-WS were significant (p = .010 and p = .005, respectively). In contrast, there were no significant differences among the three groups in the percentages of students who withdrew during the semester. The relatively large attrition in Group EA-WS meant that we needed to consider the possibility that, although students were randomly assigned to Group EA-WS, the actual Group EA-WS participants did not constitute a random sample of those who consented. However, note that, as indicated by Figure 1 and Table 3, the attrition among the EA-WS students (28%) was nevertheless less than the percentage of nonconsenting students who, although assigned to elementary algebra, did not take it (40%; because they never enrolled at CUNY, because their mathematics placement level changed, because they did not attend orientation, or because they avoided taking elementary algebra).

Table 3

Attrition Following Random Assignment and Withdrawal During the Semester

Group	Attrition^a ***	Withdrawal^b
Group	Mean % [95% CI]	Mean % [95% CI]
EA	17.85 [13.47, 22.22]	15.13 [10.54, 19.71]
EA-WS	27.48 [22.50, 32.45]	16.67 [11.73, 21.61]
Stat-WS	17.17 [12.86, 21.49]	15.10 [10.59, 19.62]

Note. CI = confidence interval; EA = elementary algebra; WS = workshop.

F(2, 904) = 6.23, p = .002.

F(2, 702) = 0.14, p = .870.

***

p < .005.

Of the 190 students who signed the consent form but who were not enrolled in their research sections on the fall 2013 census date (“noncompliers”), 57.90% were not enrolled in any college—CUNY or non-CUNY—that semester (National Student Clearinghouse data; an example of what has been called “summer melt,” Castleman & Page, 2014). Consistent with the attrition data reported earlier, the largest proportion, 45.46%, of these 110 students consisted of students who had been randomly assigned to Group EA-WS.

A total of 34 noncompliers across the three groups enrolled in nonresearch sections of elementary algebra in the fall of the experiment. No student assigned to a research section attempted to attend a different research section. Although only research participants were supposed to enroll in research sections, five nonresearch students enrolled in research sections (four total in three EA-WS sections, and one in a Stat-WS section). We excluded these five students from all analyses.

Table 1 shows the variables for which we had data for both the 717 participants and the 190 noncompliers. There were no significant differences between these two groups except that, on average, noncompliers agreed to participate in the experiment significantly earlier than participants. These results are consistent with previous findings that students who agree early to participate in research are less likely to participate. Early consenting students may be more likely to encounter work or other time conflicts with scheduled research (Watanabe-Rose & Sturmey, 2008).

To examine whether the students who participated in the treatments were representative of all students assessed as needing elementary algebra, we also compared participants with nonconsenters who took nonresearch sections of elementary algebra during the same semester as the experiment (60% of all nonconsenters; see Table 1). The only significant difference between these two groups is in the proportion of underrepresented students (p < .001) although underrepresented students constitute a substantial majority of both groups. However, these two groups may have differed on other (unmeasured) variables given that one group consented to be in our experiment, an experiment that involved a class taught during the day, and the other group did not.

Participant Treatments

Research personnel recruited instructors and selected the course sections in which the participants would enroll. There were 12 instructors, 4 at each of the three colleges. The instructors had to be full-time, willing to teach two sections of elementary algebra and one of introductory statistics, and, preferably, have taught both subjects before (three of the 12 instructors had only taught elementary algebra before). To be able to assess instructor effects and to balance these effects across treatments, each instructor taught one section of each of the three course types: EA, EA-WS, and Stat-WS (Weiss, 2010). Thus, there were 12 sections each of EA, EA-WS, and Stat-WS. This meant that the instructors had to be informed about the basic structure of the experiment, including during a 6-hour orientation session that they attended prior to the experiment (Appendix D, available in the online version of the journal, provides an example of a faculty orientation agenda). The instructors were told that the researchers believed that “at least some students assessed as needing elementary algebra will successfully pass statistics without taking elementary algebra.” Faculty were not given the experiment’s research hypothesis and were never told that the researchers hoped that statistics would have at least the same pass rate as elementary algebra.

The instructors helped ensure that the research was conducted properly. For example, at each college, they ensured that all research sections of statistics used the same syllabus (there was already a departmental common syllabus for elementary algebra at each college). Each instructor also met monthly with research personnel and weekly with the workshop leaders of that instructor’s two sections that included workshops. During the weekly sessions, the instructors gave their workshop leaders assignments and exercises for the participants to work on during the workshops and as homework. Research personnel told the instructors to teach and grade the research sections as they would ordinarily. Each instructor was paid US$3,000 for his or her participation.

Research personnel recruited the workshop leaders. Qualifications included advanced undergraduate status at or recent graduation from CUNY, successful completion of the material to be covered in the leader’s workshops, a recommendation from a mathematics faculty member, and a satisfactory personal interview. A total of 21 workshop leaders were selected for the 24 research sections that had associated weekly workshops (three workshop leaders each led the workshops for two sections). They were paid at the rate of US$14 per hour. Before the experiment began, the workshop leaders had 10 hours of training concerning the experiment and how to conduct their workshops. During the experiment’s semester, the workshop leaders met monthly with research personnel and also discussed together on social media their concerns and suggestions about conducting their workshops. Workshop leaders attended their section’s regular class meetings.

Section size did not vary significantly by group: means and 95% confidence intervals (CIs) for Groups EA, EA-WS and Stat-WS were: 20.33 [17.51, 23.15], 18.92 [16.16, 21.67], and 20.50 [18.67, 22.3], respectively; F(2, 33) = 0.58, p = .56. Elementary algebra sections and any associated workshops covered topics such as linear equations, exponents, polynomials, and quadratic equations (Appendix E, available in the online version of the journal, provides a sample syllabus). Statistics sections and associated workshops covered topics such as probability, binomial probability distributions, normal distributions, confidence intervals, and hypothesis testing (Appendix F, available in the online version of the journal, provides a sample syllabus). If students in statistics sections needed to review certain algebra concepts to understand a particular statistics topic, such as using variables in equations and different types of graphs, the workshop leader would cover that topic in the workshop. Course sections lasted 3 to 6 hours per week, depending on the college.

All workshops occurred weekly, lasted 2 hours each, and had the same structure: 10 to 15 minutes of reflection by students on what they had learned recently in class and what they had found difficult, then approximately 100 minutes of individual and group work on topics students had found difficult, and a final 5 minutes of reflection by students on the workshop’s activities and whether the students’ difficulties had been addressed. Research personnel informed all students enrolled in research sections with workshops that they were required to attend the workshops and that if they missed more than three they would have to meet with the instructor. Only students in EA-WS and Stat-WS sections could attend those sections’ workshops.

At the end of the semester, EA and EA-WS participants took the required CUNY-wide elementary algebra final examination and received a final grade based on the CUNY-wide elementary algebra final grade rubric. Instructors graded their Stat-WS participants at their discretion using the common syllabus for that college. All outcomes other than a passing grade, including any type of withdrawal or a grade of incomplete, were categorized as not passing.

All participants who passed were exempt from any further remedial mathematics courses and were eligible to enroll in introductory, college-level (i.e., credit-bearing) quantitative courses and, in the case of Stat-WS participants, to enroll in courses for which introductory statistics is the prerequisite. A passing grade in statistics satisfied the quantitative category of the CUNY general education curriculum. Participants who did not pass had to enroll in traditional remedial elementary algebra and pass it before taking any college-level quantitative courses. Stat-WS participants were informed that if they did not pass, a failing grade would not be included in their grade point averages (GPAs).

To check course progress, research personnel observed three regular class meetings of each section, as well as at least three workshops for each section of Groups EA-WS and Stat-WS. Sections were 1 or 2 weeks behind the syllabus in 25.93% of the class meetings and 27.40% of the workshops observed. In such situations, research personnel reminded the relevant instructor or workshop leader to follow the syllabus as consistently as possible.

Participants completed a mathematics attitude survey at the semester’s start and end (based on Korey, 2000) and a student satisfaction survey at the semester’s end. These pencil-and-paper surveys primarily consisted of 7-point Likert-type scales. The mathematics attitude survey consisted of 17 questions covering the following four domains: perceived mathematical ability and confidence (“Ability”), interest and enjoyment in mathematics (“Interest”), the belief that mathematics contributes to personal growth (“Growth”), and the belief that mathematics contributes to career success and utility (“Utility”). The student satisfaction survey asked about a student’s activities during the semester, for example, whether the student had gone for tutoring (available to all students independent of the experiment) and about a student’s satisfaction with those activities.

Method of Analysis for Treatment Effects

Given that students were randomly assigned to treatments, simple comparisons of course outcomes for all 907 students randomized to the three groups can identify the relative treatment effects. Intent-to-treat (ITT) analysis compares mean outcomes of groups as randomized, without regard to attrition and other forms of deviation from protocol, thus providing an unbiased estimate of the treatment effect. We compared our two treatment groups, EA-WS and Stat-WS, with Group EA. We estimated the ITT effect using Equation 1:

\begin{array}{l} \ln (\frac{\overset{\land}{p}}{1 - \overset{\land}{p}}) i = δ + β_{1} \times {STATS}_{i} \\ + β_{2} \times {EAWORK}_{i} + ε_{i}, \end{array}

in which $\ln (\overset{\land}{p} / [1 - \overset{\land}{p}]) i$ is the log odds of a positive outcome for student i, δ is the equation constant, STATS represents whether the student was randomized into group Stat-WS, EAWORK whether the student was randomized into group EA-WS, β₁ and β₂ are coefficients, and ε_i is an error term. The outcomes of interest are, first, whether a student passed his or her assigned course and, second, the total number of credits that a student had earned by 1 calendar year following the experiment’s end. The latter analysis used an OLS regression in which Y_i was equal to credits earned.

To explore further the relationships between passing the assigned course and other variables, we also fit a model that included a vector of covariates (algebra placement test score, gender, high school GPA, number of days to consent, and controls for missing values). This vector of covariates is represented by X in Equation 2:

\begin{array}{l} \ln (\frac{\overset{\land}{p}}{1 - \overset{\land}{p}}) i = δ + β_{1} \times {STATS}_{i} \\ + β_{2} \times {EAWORK}_{i} + b X_{i} + ε_{i}, \end{array}

with terms defined as in Equation 1 plus addition of the coefficient b. We did not include the prealgebra (arithmetic) placement score as a covariate because it did not add any explanatory power. We incorporated additional control variables in a subsequent analysis of the 717 participants, but among all students randomized, we have only a limited set of covariates.

Given attrition varied by group, we also determined estimates of the effect of treatment on the treated (Treatment on Compliers, or TOC) by using Angrist, Imbens, and Rubin’s (1996) instrumental variables approach. Our design meets the assumptions necessary for this approach because (a) we randomized students into groups, (b) random assignment was highly correlated with receiving treatment, and (c) those assigned to the control group (Group EA) had no ability to enroll in a different group. Instrumental variables analysis has two steps: regressing random assignment on the actual receipt of the treatment, then using the predicted values from the first step in a second regression model predicting outcome variables (here, passing the assigned course). We estimated TOC effects with the same covariates used in the ITT analysis.²

Results of Analysis for Treatment Effects

ITT and TOC

Tables 4 (passing the assigned class) and 5 (total credits accumulated) report the results using ITT and TOC methods. Table 4’s ITT estimates with no covariates show that students in Group EA-WS were not significantly more likely to pass than those in Group EA (p = .48). Those in Group Stat-WS were significantly more likely to pass than those in Group EA by a margin of 16 percentage points, and than those in Group EA-WS by 13 percentage points. When we add covariates to the ITT equation (Equation 2), there is again no significant difference between groups EA and EA-WS (p = .14), but students in the Stat-WS group were significantly more likely to pass than EA students by 14 percentage points and than EA-WS students by 11 percentage points. TOC estimates show similar results.

Table 4

Estimates of Treatment Effects on Passing

	No covariates		With covariates		n
	ITT	TOC	ITT	TOC	n
Group means
EA	0.34	0.35	0.31	0.33	297
EA-WS	0.37	0.36	0.36	0.38	313
Stat-WS	0.50	0.53	0.49	0.53	297
Treatment effects
EA-WS vs. EA	0.03 [−0.05, 0.10]	0.04 [−0.07, 0.14]	0.05 [−0.02, 0.12]	0.07 [−0.02, 0.16]	610
Stat-WS vs. EA	0.16**** [0.08, 0.24]	0.19**** [0.09, 0.28]	0.16**** [0.09, 0.23]	0.18**** [0.10, 0.27]	594
Stat-WS vs. EA-WS	0.13** [0.05, 0.21]	0.16** [0.06, 0.25]	0.11** [0.04, 0.18]	0.13** [0.05, 0.22]	610

Note. For covariates see text. 95% CIs in brackets. ITT = intent to treat; TOC = treatment on compliers; EA = elementary algebra; WS = workshop; CI = confidence interval.

p < .01. ****p < .001.

Table 5 shows that the Stats-WS students’ enhanced academic success lasted beyond the experiment’s semester (beyond the grading of the experiment’s instructors), as evidenced by the Stat-WS students’ greater credit accumulation rates. ITT tests, both with and without covariates, and with and without statistics credits included, are significant (p < .001). One year after the end of the experiment, the Stat-WS participants had increased their mean total accumulated credit advantage from 2.38 (8.26 vs. 5.88) to 4.00 (21.71 vs. 17.71) in comparison with the EA participants. A higher percentage of the Stat-WS participants was enrolled (66%) than of the EA participants (62%) in fall 2014, but this difference is not significant.

Table 5

ITT Estimates of Treatment Effects on Total Credits Accumulated During Experiment’s Semester and the Year Following (N = 907)

	Total credits		Total credits not including statistics
	No covariates	Covariates	No covariates	Covariates	n
Group means
EA	15.78	15.53	15.78	15.53	297
EA-WS	14.42	14.66	14.41	14.66	313
Stat-WS	20.46	21.12	18.94	18.61	297
Treatment effects
EA-WS vs. EA	−1.37 [−3.57, 0.83]	−0.87 [−2.91, 1.17]	−1.37 [−3.57, 0.83]	−0.87 [−2.91, 1.17]	610
Stat-WS vs. EA	4.67**** [2.32, 7.03]	4.40**** [2.21, 6.59]	3.15** [0.87, 5.44]	2.87* [0.74, 5.02]	594
Stat-WS vs. EA-WS	6.04**** [3.72, 8.37]	5.38**** [3.22, 7.55]	4.52**** [2.26, 6.79]	3.88**** [1.76, 5.99]	610

Note. For covariates, see text. 95% CIs in brackets. ITT = intent to treat; EA = elementary algebra; WS = workshop; CI = confidence interval.

p < .05. **p < .01. ****p < .001.

We also explored the performance of the three groups in CUNY’s nine general education course categories through 1 calendar year after the end of the experiment (the end of fall 2014). Among all 907 randomly assigned students, as expected, the Stat-WS students were significantly more likely to have satisfied the quantitative category than students in the other two groups (0.48 [0.42, 0.54] compared with 0.22 [0.17, 0.27] and 0.21 [0.17, 0.26] for Groups EA and EA-WS, p < .001 for both comparisons) and as likely to have satisfied the two other science, technology, engineering and mathematics (STEM) and six non-STEM categories than had students in the other two groups (see Appendix G, available in the online version of the journal). Stat-WS students made progress in satisfying their general education requirements in science and non-STEM disciplines despite not having been assigned to elementary algebra.

Course Success Among Participants

Figure 2 shows the overall pass rates for each of the three groups of participants (EA, EA-WS, and Stat-WS) and compares them with the historical pass rates for these courses in fall 2012. The pass rate for Group EA-WS (44.93%), which was 5.59 percentage points higher than that of Group EA (39.34%), is also higher than that of students who took elementary algebra at the three colleges in fall 2012 (36.80%).³ In contrast, the pass rates for Group EA (39.34%) and for students who took elementary algebra in fall 2012 (36.80%) are similar. Group Stat-WS passed at a lower rate (55.69%) than did students who took introductory statistics at the three colleges in fall 2012 (68.99%). However, as demonstrated in Figure 2, if the Group Stat-WS sample is restricted to participants who received relatively high scores on the placement test, the mean pass rate (67.62%) is similar to that of the previous year’s statistics students (68.99%). Colleges can place into statistics students just below the cutoff for elementary algebra without any diminution in the typical statistics pass rate.

Figure 2.

Course pass rates.

Having established a significant effect of the treatment on passing using the ITT and TOC analyses, we utilized logistic regression to further investigate predictors of participants passing assigned courses. The main independent variable was a set of dummy variables indicating treatment status, with Group EA as the omitted reference group. For ease of interpretation, Table 6 reports the results of the logistic regression as average marginal effects rather than as odds ratios.⁴ The largest effect size was that of the treatment—being placed in Group Stat-WS. As noted above, we find no significant difference between Group EA and Group EA-WS in the probability of passing (p = .097), but the difference between Groups EA-WS and Stat-WS is significant (p = .031), as is the difference between Groups EA and Stat-WS (p < .001). When controlling for all other variables, there is a significant difference of almost 17 percentage points between students in Group Stat-WS and Group EA. Being enrolled in Stat-WS is associated with students’ probability of passing more than the increase associated with a one-standard-deviation increase in the Compass algebra score. Some covariates showed significant effects: Students with higher algebra placement scores and higher high school GPAs were more likely to pass, and students whose first language was English were less likely to pass.

Table 6

Logistic Regression Model Predicting Participants Passing (N = 717)

Variable	Mean marginal effect [95% CIs]
Treatment status (ref: Group EA)
Group EA-WS	0.07 [−0.01, 0.16]
Group Stat-WS	0.17 [0.08, 0.25]****
College (ref: College B)
College A	0.02 [−0.08, 0.11]
College C	0.015 [−0.07, 0.10]
Age (years)	0.00 [−0.01, 0.01]
Compass z score (algebra)	0.13 [0.09, 0.16]****
Compass score missing	0.02 [−0.10, 0.15]
Days to consent	−0.00 [−0.00, 0.00]
First language (English)	−0.09 [−0.16, 0−.02]*
Gender (female)	0.01 [−0.06, 0.08]
High school GPA z score	0.08 [0.04, 0.11]****
Instructor experience (years)	0.00 [−0.01, 0.01]
Instructor has taught statistics	−0.02 [−0.12, 0.07]
Instructor has tenure	0.03 [−0.06, 0.11]
Race (underrepresented)	−0.07 [−0.17, 0.04]

Note. CI = confidence interval; EA = elementary algebra; WS = workshop; GPA = grade point average.

p < .05. ****p < .001.

No interactions between student demographic variables and treatment status were significant. However, given that the purpose of placement tests is to place students in courses in which they are most likely to be successful, the widespread use of these tests, and their significant cost (Rodríguez, Bowden, Belfield, & Scott-Clayton, 2015), as well as the fact that we had placement test scores for most of the participants, we particularly examined the relationship between placement test score and passing. Table 6 shows that algebra placement test z score is a strong predictor of passing with all participants combined. Figure 3 shows, for each group separately, the relationship between these scores and the probability of passing (a version of this Figure based on a nonlinear—quadratic—form is shown in Appendix H, available in the online version of the journal). For this analysis, we focused only on those participants who had placement scores, and controlled for the same covariates included in the logistic regression model reported above in Table 6. Given that the line for Group Stat-WS is consistently higher than that for Group EA, students with any placement test algebra score are more likely to pass if they enroll in introductory statistics with a weekly workshop as opposed to traditional elementary algebra. It should also be noted that, in this sample, even students with average placement test scores have a better than 50% chance of passing statistics with a weekly workshop. Finally, the fact that the line for Group EA-WS is consistently higher than the line for Group EA supports the hypothesis that the workshops did help students to pass elementary algebra.

Figure 3.

Probability of passing as a function of Compass algebra z score with covariates.

Figure 4 summarizes the quantitative-skills and quantitative-course status of the 717 participants in the three groups as of 1 year after the end of the experiment: 57.32% of the Stat-WS students had passed a college-level quantitative course (all but two of them by passing statistics in the fall of 2013), while 37.80% still had remedial need. In contrast, only 15.98% of the EA students had passed a college-level quantitative course and 50.00% still had remedial need.

Figure 4.

Quantitative-course status of participants 1 year after experiment’s end.

Additional Effects

Neither an instructor’s tenure status nor experience was significant in the logistic regression (see Table 6). To test further instructors’ impact on course outcomes, we used a mixed-effects logit regression model with instructor as the random effect (see Appendix I, available in the online version of the journal).⁵ The results showed that instructor assignment affected students’ probability of passing. However, there was still an effect of treatment group, again indicating that, across classrooms and instructors, Stat-WS students were more likely to pass. A log likelihood test comparing a standard logistic regression with the mixed-effects model showed the latter to fit the model significantly better, χ²(1) = 5.33, p = .011. There was no differential effect of treatment status as a function of instructor demographic characteristics.

Being enrolled in Group Stat-WS may have particularly enhanced participants’ attitudes about mathematics. The three groups’ participants did not significantly differ on any of the four domains measured by the precourse student mathematics attitude survey. However, comparing this survey’s pre- and postcourse results among the 338 participants who completed both, Group Stat-WS participants showed significant increases by the end of the semester on the Interest, Growth, and Utility domains. In contrast, Groups EA and EA-WS participants showed increases only in the Interest domain (Table 7). However, conclusions should be made with caution given that the response rates were around 50% and that, across all three treatments combined, the pass rate of participants who filled out both the pre- and the postmathematics attitude surveys was 68%, as compared with 28% for other participants.

Table 7

Participant Mathematics Attitudes

Group	Measure	Domains
Group	Measure	Ability	Interest	Growth	Utility
EA	Presurvey M	18.49	14.10	15.12	14.26
	[95% CI]	[17.27, 19.71]	[13.13, 15.08]	[14.24, 16.01]	[13.45, 15.08]
	Postsurvey M	19.08	15.46	14.75	14.56
	[95% CI]	[17.84, 20.31]	[14.46, 16.47]	[13.69, 15.80]	[13.70, 15.41]
	t (df)	0.93 (105)	2.82 (105)***	−0.74 (105)	0.66 (105)
EA-WS	Presurvey M	17.52	13.57	15.35	14.33
	[95% CI]	[16.38, 18.66]	[12.57, 14.56]	[14.62, 16.08]	[13.49, 15.17]
	Postsurvey M	17.26	15.15	15.21	14.34
	[95% CI]	[16.11, 18.42]	[14.23, 16.07]	[14.24, 16.17]	[13.45, 15.23]
	t (df)	−0.47 (105)	3.14 (105)***	−0.27 (105)	0.019
Stat-WS	Presurvey M	18.38	13.44	14.87	13.33
	[95% CI]	[17.39, 19.37]	[12.56, 14.32]	[14.08, 15.67]	[12.58, 14.09]
	Postsurvey M	18.56	15.29	15.89	14.32
	[95% CI]	[17.48, 19.65]	[14.42, 16.17]	[14.98, 16.80]	[13.49, 15.15]
	t (df)	0.37 (125)	4.10 (125)***	2.14 (125)*	2.42 (125)**

Note. EA = elementary algebra; WS = workshop; CI = confidence interval.

p < .05. **p < .01. ***p < .005.

Two aspects of student behavior could help to explain the three groups’ pass rates differences. First, although there were no significant differences among the groups in terms of reported use of the tutoring available to all students, the postcourse student satisfaction survey showed that EA-WS and Stat-WS participants were more likely to participate in self-initiated study groups than were EA participants. The percentages of EA, EA-WS, and Stat-WS groups who engaged in a self-initiated study group were 41.98% [33.42, 50.55], 61.11% [52.48, 69.74], and 67.38% [59.54, 75.21], respectively; F(2, 395) = 9.97, p < .001. (Again, these results should be cautiously interpreted due to low survey response rates, around 50%.) Second, Stat-WS participants were more likely to attend their workshops (71.99%) than were EA-WS participants: 65.04%; 95% CI for the difference in the percentage of workshops attended by Stat-WS and EA-WS participants is [−12.13, −1.76]; t(471) = −2.63, p = .009. These behavior differences may be course effects, not a priori causes of higher pass rates for Groups EA-WS and Stat-WS, indicating the positive motivational effects of being assigned to a college-level course.

Another possible reason for the higher pass rates in Group Stat-WS is that, despite randomization, the groups might have differed in terms of student characteristics that can affect passing. However, there is no compelling evidence that this occurred. Table 2 shows no significant differences among the three groups on any of the measured variables.

Yet another possible reason for the higher pass rates in Group Stat-WS is that, due to the random assignment methods, the Stat-WS students may have felt that they won a lottery and therefore been more motivated. Any such effect, if it existed, would have had to have continued into the year after the experiment was over, when the credit gap between Groups EA and Stat-WS widened.

Still another possible reason for the higher pass rate in Group EA-WS than Group EA, and the highest pass rate in Group Stat-WS, is that, although each instructor taught one section of each course type, perhaps the instructors graded these courses differently. Given that elementary algebra and introductory statistics are qualitatively different courses, it is not possible to compare their grading directly. However, given that the experiment’s purpose was to compare student success rates in typical remedial mathematics and introductory statistics courses, a more useful question is whether the grading criteria used by the experiment’s instructors were similar to when these courses were usually taught. Nine pieces of evidence suggest that the results were due to group assignment and not due to changes in the instructors’ grading practices.

First, all 12 of the instructors had taught elementary algebra before, nine had previously taught introductory statistics, and all were told by the researchers to teach as they usually did. Second, there were no significant relationships between participants passing and instructors’ tenure status, total years of experience, or experience teaching statistics (Table 6). Third, as shown by the results of the mixed-effects logistic regression and Table 6, the stronger effect size was for group assignment, not instructor. Fourth, all sections of elementary algebra across CUNY are standardized in terms of topics, a common final exam, and a common final grade rubric. For introductory statistics, all sections at the community colleges cover the same topics, and the experiment’s instructors at each college taught from a common syllabus. Fifth, there were no significant differences in percentage passing by college (Table 6). Sixth, despite the use of a randomized controlled trial with monetary incentives for participation, as described previously, the pass rate for Group EA was similar to that of students who took elementary algebra 1 year earlier, and the pass rate of Group Stat-WS was lower than that of students who took statistics 1 year earlier (students who were either exempt from remedial mathematics, or who had previously passed remedial mathematics and were in at least their second semester at CUNY). Seventh, a subset of Stat-WS participants with Compass scores close to the criterion for mathematics remediation exemption passed statistics at a rate similar to that of statistics students with no remedial need (the two most right-hand bars in Figure 2). Eighth, as indicated earlier, the higher pass rates of Stat-WS students are consistent with indications that these students were more motivated. Ninth, as also discussed earlier, the Stat-WS students’ enhanced academic success lasted beyond the grading of the experiment’s instructors, with the Stat-WS students continuing to accumulate more credits than students in the other two groups during the calendar year following the experiment.

Discussion and Policy Implications

The results showed that the Stat-WS students passed statistics at not the hypothesized same rate as the elementary algebra students but at a significantly higher rate than did the EA and EA-WS students. The higher pass rate for statistics with additional support as compared with elementary algebra was robust across multiple types of analyses, colleges, and instructors. By 1 year after the experiment was over, the Stat-WS students had also satisfied the same number of general education science categories and had continued to accumulate more credits than the students in either of the other two groups. Students assessed as needing elementary algebra do not first need to pass that course to pass a college-level quantitative course and to be successful in college, at least not if that college-level course is introductory statistics with weekly workshops.

The mainstreaming/corequisite remediation approach has the potential to positively affect the academic progress of many thousands of college students. At CUNY in fall 2012 alone, 7,675 new students were initially assessed with a placement of remedial elementary algebra. To complete statistics within two semesters of entry, such students would have to pass elementary algebra in the fall (the actual pass rate is 37%), return in the spring (the overall retention rate for freshmen from fall to spring is 84%), and then take and pass statistics in the spring (CUNY statistics students who have previously passed elementary algebra have a 68% statistics pass rate). Even if we were to assume that all fall students who pass elementary algebra are retained for the spring, and that all such students take statistics that spring semester, the probability of completing statistics within two semesters for these students is therefore only .37 times .68, that is, .25. In contrast, 55.69% of the Stat-WS participants passed statistics in their first semester, and of those who did not, more might pass it in their second semester if permitted to attempt it again. This suggests⁶ that, of students entering CUNY just in fall 2012, at least 2,379 more students would pass statistics by the end of their second semester at CUNY if they took a statistics-with-workshops quantitative path rather than the traditional elementary algebra path (4,298 vs. 1,919 students completing statistics within two semesters). These benefits for thousands of students accrue with a 55.69% pass rate in statistics. Should that pass rate be increased, for example, by focusing the workshops on the most useful content, even more students could benefit in terms of satisfying CUNY’s general education quantitative requirement and making progress in college.

The degree to which these implications would apply nationally remains to be explored; we do not know precisely how representative the three participating colleges are of all colleges, or even community colleges, across the United States. Three different community colleges were involved in the present experiment, but all used the CUNY algebra placement cut score of 40 for exemption from (remedial) elementary algebra. In contrast, a national survey of colleges and universities (not just community colleges) found a mean cut score of 49 (Fields & Parsad, 2012). However, Figure 3 suggests that the remediation used here would benefit students with a wide range of Compass scores, and there have been many descriptions from around the United States of benefits of corequisite mathematics remediation (e.g., Palmer, 2016).

Our results demonstrate that mainstreaming/corequisite remediation can help to decrease performance gaps: Our student sample was diverse and our findings did not differ by race (Table 6). In addition, the statistics course pass-rate advantage in the present experiment exists for students with a wide variety of placement test scores (Figure 3). Whether their placement test scores accurately reflect their abilities or not, students placed into statistics with workshops rather than elementary algebra should be more likely to pass a college-level quantitative course and complete college. Furthermore, our data suggest that students whose placement is elementary algebra will not only be more likely to pass but will have a more positive attitude toward mathematics if they first take statistics than if they first take elementary algebra. Thus, taking statistics first might be appropriate for students intending to become STEM majors, not just other majors. Taking statistics first might encourage a student to remain, as well as to become, a STEM major.

Our findings are inconsistent with the purported theory underlying many colleges’ policies requiring students to pass remedial courses prior to taking college-level courses. Instead, the results support state and college policies, instituted to increase college graduation rates, which allow, or require, placement of students assessed as needing remedial mathematics instead into college-level quantitative courses.

Our data are consistent with some of the theories regarding why placement into college-level courses can enhance student performance. For example, the evidence supports the theory that the Stat-WS students would be more motivated than the other students. The higher workshop attendance rate, the self-reported higher study-group rate, and the greater increase in positive attitudes toward mathematics among the Stat-WS participants all support this theory.

Most participants in our EA and EA-WS groups ended the semester having spent 3 hours (or more) per week of course time, and the resulting tuition and financial aid, with no resulting progress toward their degrees, and still needing to pass a remedial course, but most of the participants in our Group Stat-WS ended the semester with three credits toward their degrees and satisfaction of their college’s general education quantitative requirement. Thus, students in Group Stat-WS who passed were two courses closer to their degrees than were students in any of the groups who failed. Such degree progress contributes to what has been termed academic momentum (Adelman, 2006; Attewell, Heil, & Reisel, 2012), in addition to decreasing time-to-degree (Bowen & McPherson, 2016), both of which have been described as critical to college completion. The increased total cumulated credits advantage of the Stat-WS compared with the EA participants 1 year after the experiment was over supports the hypothesis that mainstreaming increased the long-term academic momentum of the Stat-WS students.

Other findings from this experiment provide some guidance regarding the usefulness of workshops. The evidence regarding whether the workshops helped the elementary algebra students pass is mixed. The ITT and TOC comparisons of Groups EA and EA-WS were not significant (Table 4). However, the EA-WS participants consistently passed at a higher rate than the EA participants (Figures 2 and 3). Furthermore, in accordance with predictions, EA participants passed at a rate similar to that of elementary algebra students 1 year earlier, but EA-WS participants passed at a higher rate than did elementary algebra students 1 year earlier.

Prior research and the present Group EA versus Group EA-WS comparisons only suggest that the workshops improved the statistics pass rates. Due to CUNY policy, participants could not be placed into statistics without workshops. Therefore, it is not possible to determine from the present experiment how much of the greater pass rate by the Stat-WS students was due to the workshops and how much was due to the students being enrolled in a college-level statistics course.

We also do not know what the pass rate would have been had our participants been randomly assigned to college algebra with workshops. A direct comparison of student performance in such a course versus statistics with workshops could help determine whether the effect of mainstreaming with workshops is course specific.

An unanticipated and concerning finding is that there was differential attrition for the three groups (see Table 3), suggesting that assigning students to a time-consuming remedial course (here Group EA-WS) may discourage them from attending college altogether. This possibility should be examined in future research. This finding also indicates that, when comparing performance in different courses, researchers should carefully examine the precourse attrition rates and not just performance during the courses. Note that Group EA-WS students were required not only to attend class 5 to 6 hours per week just to address their remedial need, in comparison with 3 to 4 hours for Group EA, and 5 to 6 hours for Group Stat-WS to address their remedial need but also to receive three college-level general education credits. Thus, Group EA-WS was most distant in time from the goal of graduation, and was probably least motivated to attend the experiment’s assigned course and possibly also college.

Although the results of the present experiment show that students assigned to remedial mathematics can progress more quickly toward their degrees if they instead take introductory statistics with workshops, degree progression is not the only consideration in setting remediation policy. The participants in Group Stat-WS were only taught elementary algebra material to the extent that such material was needed to understand the statistics material. Whether students should be graduating from college having learned statistics but without having learned all of elementary algebra is one of the many decisions that a college must make regarding which particular areas of knowledge should be required for a college degree. Views can differ as to which quantitative subjects a college graduate should know. However, it is clear that passing elementary algebra is not necessary to pass introductory statistics with weekly workshops. Therefore, if a college or state deems introductory statistics to be necessary for a degree, it does not necessarily need to also require the usually precollege (i.e., remedial) course of elementary algebra. For a college or state to require all students to pass elementary algebra first, in addition to completing the credits needed for their degrees, can be an extra cost for students, colleges, and taxpayers; funds that could be spent on other college courses and programs. That extra cost, as well as educational goals, should be taken into account when higher education policy decisions are made. College communities, and our society, must decide whether the extra cost is worth the results.

Footnotes

Acknowledgements

We thank K. Kapp for invaluable management expertise; D. Allen, P. Attewell, B. Brown, C. Chellman, D. Crook, H. Everson, A. Horenstein, S. Jaggars, C. Kendrick, C. Littman, J. Lucariello, C. Mangino, A. Mayer, T. Meade, S. Ondrus, C. Orman, L. Richburg-Hayes, D. Rindskopf, S. Stigler, Z. Tang, and M. Weiss for advice and assistance with research design and data analysis, as well as for comments on a previous version of this article; Q. Chaughtai, L. Lambert, M. Nechayeva, L. Porte, and C. Wladis for assistance in conducting the research; the Presidents, Provosts, Deans, Mathematics Department Chairs, and staff of the three colleges for facilitating the research; CUNY Central Office members G. Dine, J. Jian, C. Kendrick, D. Linderman, R. Maruca, J. Mogulescu, J. Murphy, R. Spalter, G. Taylor, and J. Wrigley for helping to overcome administrative hurdles; and the faculty, workshop leaders, and hundreds of students who served as research participants.

Authors’ Note

Portions of the results reported here were presented at the National Conference on Acceleration in Developmental Education, June 2014; the National Urban League-Educational Testing Service Higher Education Convening, September 2014; the Annual Fall Conference of the Society for Research on Educational Effectiveness, September 2014; the Annual Meeting of the American Educational Research Association, April 2015; and the Annual Fall Research Conference of the Association for Public Policy and Management, November 2015.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was supported by a grant from The Spencer Foundation and by The City University of New York (CUNY).

Notes

Authors

A. W. LOGUE is a research professor in CASE (Center for Advanced Study in Education) of the Graduate Center of The City University of New York (CUNY), with particular responsibility for research and scholarship concerning college student success. She received her AB and PhD from Harvard University in experimental psychology. After 17 years as a faculty member in the Psychology Department of SUNY Stony Brook, specializing in the areas of learning and motivation, she served in various administrative roles in the New York metropolitan area, most recently as CUNY’s executive vice chancellor and university provost (chief academic officer) from 2008 to 2014.

MARI WATANABE-ROSE is director of Under-graduate Education Initiatives and Research in the CUNY Central Office of Academic Affairs. She received her BA in educational psychology from Meisei University in Tokyo, Japan; MA in psychology from Queens College, CUNY; and PhD in learning processes and behavior analysis from the Graduate Center, CUNY. Following eight years of teaching psychology as an adjunct instructor at Queens College and working as a board-certified behavior analyst with people with developmental disabilities, she joined CUNY Central in 2009.

DANIEL DOUGLAS is a PhD candidate in sociology at the Graduate Center of the City University of New York. His dissertation research focuses on teacher evaluation systems in K-12 education. His other research examines the connection between higher education and employment, specifically focusing on mathematics and other science, technology, engineering, and mathematics (STEM) disciplines.

References

Adelman

(2006). The toolbox revisited: Paths to degree completion from high school through college. Washington, DC: U.S. Department of Education. Retrieved from https://www2.ed.gov/rschstat/research/pubs/toolboxrevisit/toolbox.pdf

Angrist

J. D.

Imbens

G. W.

Rubin

D. B.

(1996). Identification of causal effects using instrumental variables. Journal of the American Statistical Association, 91, 444–455.

Attewell

Heil

Reisel

(2012). What is academic momentum? And does it matter? Educational Evaluation and Policy Analysis, 34, 27–44. doi:10.3102/0162373711421958

Attewell

Lavin

Domina

Levey

(2006). New evidence on college remediation. Journal of Higher Education, 77, 886–924.

Bailey

(2009). Challenge and opportunity: Rethinking the role and function of developmental education in community college. New Directions for Community Colleges, 145, 11–30. doi:10.1002/cc.352

Bailey

Jeong

D. W.

Cho

S.-W.

(2010). Referral, enrollment, and completion in developmental education sequences in community colleges. Economics of Education Review, 29, 255–270. doi:10.1016/j.econedurev.2009.09.002

Barnett

E. A.

Bork

R. H.

Mayer

A. K.

Pretlow

Wathington

H. D.

Weiss

M. J.

(2012, June). Bridging the gap: An impact study of eight developmental summer bridge programs in Texas. National Center for Postsecondary Research. Retrieved from http://www.postsecondaryresearch.org/i/a/document/22731_NCPR_TexasDSB_FullReport.pdf

Bettinger

E. P.

Baker

R. B.

(2014). The effects of student coaching: An evaluation of a randomized experiment in student advising. Educational Evaluation and Policy Analysis, 36, 3–19. doi:10.3102/0162373713500523

Bettinger

E. P.

Long

B. T.

(2009). Addressing the needs of underprepared students in higher education: Does college remediation work? Journal of Human Resources, 44, 736–771.

10.

Boatman

(2012, June). Evaluating institutional efforts to streamline postsecondary remediation: The causal effects of the Tennessee Development Course Redesign Initiative on early student academic success. New York, NY: National Center for Postsecondary Research. Retrieved from http://www.postsecondaryresearch.org/i/a/document/22651_BoatmanTNFINAL.pdf

11.

Bowen

W. G.

McPherson

M. S.

(2016). Lesson plan: An agenda for change in American higher education. Princeton, NJ: Princeton University Press.

12.

Bowles

T. J.

McCoy

A. C.

Bates

(2008). The effect of supplemental instruction on timely graduation. College Student Journal, 42, 853–859.

13.

Bueschel

A. C.

(2009). The landscape of policies and practices that support student preparation and success. New Directions for Community Colleges, 145, 1–10. doi:10.1002/cc.351

14.

Burdman

(2013). Changing equations: How community colleges are re-thinking college readiness in math. Oakland, CA: LearningWorks. Retrieved from http://www.learningworksca.org/wp-content/uploads/2013/10/LWBrief_ChangingEquations_WEB.pdf

15.

Calcagno

J. C.

Long

B. T.

(2008, July). The impact of postsecondary remediation using a regression discontinuity approach: Addressing endogenous sorting and noncompliance (NBER Working Paper No. 14194). Cambridge, MA: National Bureau of Economic Research. Retrieved from http://www.nber.org/papers/w14194.pdf

16.

Castleman

B. L.

Page

L. C.

(2014). Summer melt: Supporting low-income students through the transition to college. Cambridge, MA: Harvard Education Press.

17.

Clotfelter

C. T.

Ladd

H. F.

Muschkin

Vigdor

J. L.

(2015). Developmental education in North Carolina community colleges. Educational Evaluation and Policy Analysis, 37, 354–375. doi:10.3102/0162373714547267

18.

Complete College America. (2011, September). Time is the enemy. Indianapolis, IN: Author. Retrieved from http://completecollege.org/docs/Time_Is_the_Enemy.pdf

19.

Complete College America. (2012, April). Remediation: Higher education’s bridge to nowhere. Washington, DC: Author. Retrieved from http://completecollege.org/docs/CCA-Remediation-final.pdf

20.

Complete College America. (2016). Corequisite remediation: Spanning the completion divide. Washington, DC: Author. Retrieved from http://completecollege.org/spanningthedivide/#home

21.

CUNY, Office of Institutional Research and Assessment. (2015a). Graduation rates of students with remedial needs. Unpublished data report.

22.

CUNY, Office of Institutional Research and Assessment. (2015b). Initial remedial needs. Unpublished data report.

23.

CUNY, Office of Institutional Research and Assessment. (2015c). Pass rates of selected math courses: Fall 2014. Unpublished data report.

24.

Douglas

Attewell

(2014). The bridge and the troll underneath: Summer bridge programs and degree completion. American Journal of Education, 121, 87–109. doi:10.1086/677959

25.

Edgecombe

(2011, May). Accelerating the academic achievement of students referred to developmental education (CCRC Brief). New York, NY: Community College Research Center. Retrieved from http://ccrc.tc.columbia.edu/media/k2/attachments/accelerating-achievement-developmental-education-brief.pdf

26.

Fields

Parsad

(2012, November). Tests and cut scores used for student placement in postsecondary education: Fall 2011. Washington, DC: National Assessment Governing Board. Retrieved from https://www.nagb.org/content/nagb/assets/documents/what-we-do/preparedness-research/surveys/test-and-cut-scores-used-for-student-placement-in-postsecondary-education-fall-2011.pdf

27.

Goldrick-Rab

(2007, February). Promoting academic momentum at community colleges: Challenges and opportunities (CCRC Working Paper No. 5). New York, NY: Community College Research Center. Retrieved from http://ccrc.tc.columbia.edu/publications/academic-momentum-community-colleges.html

28.

Grubb

W. N.

Boner

Frankel

Parker

Patterson

Gabriner

. . . Wilson

(2011, June). Understanding the “crisis” in basic skills: Framing the issues in community colleges (Policy Analysis for California Education Working Paper). Retrieved from http://www.edpolicyinca.org/publications/understanding-%E2%80%9Ccrisis%E2%80%9D-basic-skills-framing-issues-community-colleges

29.

Hern

(2012). Acceleration across California: Shorter pathways in developmental English and Math. Change: The Magazine of Higher Learning, 44, 60–68.

30.

Jaggars

S. S.

Hodara

Cho

S.-W.

(2015). Three accelerated developmental education programs: Features, student outcomes, and implications. Community College Review, 43, 3–26. doi:10.1177/0091552114551752

31.

Jones

(2014, April 18). Essay on how to fix remedial education. Inside Higher Ed. Retrieved from https://www.insidehighered.com/views/2014/04/18/essay-how-fix-remedial-education

32.

Kalamkarian

H. S.

Raufman

Edgecombe

(2015, May). Statewide developmental education reform: Early implementation in Virginia and North Carolina. Community College Research Center. Retrieved from http://ccrc.tc.columbia.edu/publications/statewide-developmental-education-reform-early-implementation.html

33.

Korey

(2000, March). Dartmouth College mathematics across the curriculum evaluation summary: Mathematics and humanities courses. Retrieved from www.math.dartmouth.edu/~matc/Evaluation/humeval.pdf

34.

Logue

A. W.

(1995). Self-control: Waiting until tomorrow for what you want today. Englewood Cliffs, NJ: Prentice Hall.

35.

Martorell

McFarlin

(2011). Help or hindrance? The effects of college remediation on academic and labor market outcomes. The Review of Economics and Statistics, 93, 436–454.

36.

Melguizo

Bos

Prather

(2011). Is developmental education helping community college students persist? A critical review of the literature. American Behavioral Scientist, 55, 173–184. doi:10.1177/0002764210381873

37.

Moss

B. G.

Yeaton

W. H.

Lloyd

J. E.

(2014). Evaluating the effectiveness of developmental mathematics by embedding a randomized experiment within a regression discontinuity design. Educational Evaluation and Policy Analysis, 36, 170–185. doi:10.3102/0162373713504988

38.

Office of Academic Affairs. (2013, November). CUNY Start: Analysis of student outcomes. The City University of New York. Retrieved from http://www.cuny.edu/academics/programs/notable/CATA/cti-cunystart/how-are-we-doing.html

39.

Palmer

(2016). How to fix remediation at scale. New America. Retrieved from http://www.edcentral.org/fix-remediation-at-scale/

40.

Rodríguez

Bowden

Belfield

Scott-Clayton

(2015, May). Calculating the costs of remedial placement testing. Community College Research Center. Retrieved from http://ccrc.tc.columbia.edu/media/k2/attachments/calculating-cost-remedial-placement-analytics-2.pdf

41.

Scott-Clayton

Crosta

P. M.

Belfield

C. R.

(2014). Improving the targeting of treatment: Evidence from college remediation. Educational Evaluation and Policy Analysis, 36, 371–393. doi:10.3102/0162373713517935

42.

Scott-Clayton

Rodriguez

(2012, August). Development, discouragement, or diversion? New evidence on the effects of college remediation (NBER Working Paper No. 18328). Retrieved from http://www.nber.org/papers/w18328

43.

Smith

A. A.

(2015, May 8). States and colleges increasingly seek to alter remedial classes. Inside Higher Ed. Retrieved from https://www.insidehighered.com/news/2015/05/08/states-and-colleges-increasingly-seek-alter-remedial-classes

44.

Watanabe-Rose

Sturmey

(2008). The effects of appointment delay and reminders on appointment-keeping behavior. Behavior and Social Issues, 17, 161–168. doi:10.5210/bsi.v16i2.373

45.

Weiss

M. J.

(2010). The implications of teacher selection and the teacher effect in individually randomized group treatment trials. Journal of Research on Educational Effectiveness, 3, 381–405. doi:10.1080/19345747.2010.504289

46.

Wolfle

J. D.

Williams

M. R.

(2014). The impact of developmental mathematics courses and age, gender, and race and ethnicity on persistence and academic performance in Virginia community colleges. Community College Journal of Research and Practice, 38, 144–153. doi:10.1080/10668926.2014.851956

47.

Yamada

(2014, November). Community college Pathways’ program success: Assessing the first two years’ effectiveness of Statway®. Stanford, CA: Carnegie Foundation for the Advancement of Teaching. Retrieved from http://cdn.carnegiefoundation.org/wp-content/uploads/2014/11/CCP_Pathways_Success_Nov_2014.pdf

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.13 MB