Abstract
We examine the impact of remedial and developmental courses on college students with varying levels of academic preparedness, thus focusing on a wider range of students than previous studies. Using a regression discontinuity design, we provide causal estimates of the effects of placement in different levels of remedial courses on short-, intermediate-, and long-term outcomes at both 2- and 4-year colleges. Similar to other research, we find that remediation has negative effects for students on the margin of needing one developmental course. However, for students with lower levels of academic preparation, the effects of remediation are estimated to be positive in some subjects. These results suggest that remedial courses can help or hinder students differently depending on their incoming levels of academic preparedness. Moreover, our conclusions are largely driven by positive and negative effects observed for students at 2-year institutions, and we discuss several hypotheses that may explain these findings.
Keywords
Remediation comes at a great expense to colleges and universities, with efforts estimated to cost well more than a billion dollars a year at public colleges alone (Strong American Schools, 2008). In a 2011 study, the Alliance for Excellent Education concluded that the total cost of delivering remediation nationwide for college students enrolled during the 2007–2008 academic year was US$5.6 billion in the form of direct costs both to students (e.g., tuition) and to institutions (e.g., instructional costs), and lost lifetime wages due to the greater likelihood that remedial students will drop out of college before earning a degree. A recent report from the Center for American Progress estimates that, nationally, students pay approximately US$1.3 billion for remediation annually (Jimenez, Sargrad, Morales, & Thompson, 2016). However, the social costs of not offering remediation are of great concern amid the growing demand for skilled labor in the United States (Bailey, Jeong, & Cho, 2010; Bettinger, Boatman, & Long, 2013). Given the current proportion of college students who require remediation upon initial college entry, making postsecondary remediation efforts successful is critical to increasing the number of skilled workers nationally.
A growing body of research is emerging on both the scope and effectiveness of college remediation. Much of the existing research, however, focuses on students at the margin of passing out of remediation and compares students who score just above and below the cutoff on the remediation placement exam to establish the causal effects of the courses (e.g., see Martorell & McFarlin, 2011). These estimates are Local Average Treatment Effects (LATE) that reflect the effects observed for students needing only one remedial course, but they do not tell us anything about students who need far more remediation to be ready for college-level courses. However, to fully understand the impact of remediation and what colleges can do to help underprepared students, it is important to include the experiences of students who need more support than just a single course.
Unfortunately, there is still limited research on students who need more than just one developmental education course. In descriptive research, Bailey et al. (2010) used data from Achieving the Dream colleges to track the progression of students at multiple levels of math and reading remediation, and found that 17% of students assigned to three levels below college-level math ever completed their remedial sequence. Among all students assigned to a developmental math course three levels below college math, only 10% go on to pass their first college-level math course (Bailey et al., 2010). Only two studies have explicitly examined the causal effects that lower levels of remedial courses have on student success. A regression discontinuity (RD) study conducted by Dadger (2012) on students in Virginia’s 23 community colleges finds that being assigned to three rather than two levels of remedial math reduced the likelihood of earning a community college credential by 9- to 15 percentage points. Dadger concludes that students assigned to the lowest level of remedial math would have benefited if they had been able to skip that remedial course. There is additional work examining the outcomes of language minority students in English as a second language (ESL) and remedial courses (e.g., Hodara, 2015), but to date we know of no other work that has examined the effects of traditional remediation for students assigned to multiple levels below college-level material at both 2- and 4-year colleges.
Given the limited causal research on the effectiveness of remedial courses, our study makes several important contributions to the existing literature. Using data on both the 2- and 4-year public colleges within an entire state system, we examine the impact of multiple levels of remedial courses within three distinct subjects: math, reading, and writing. Other papers focus primarily on one subject, or combine reading and writing into a more general “English” course. This three-subject differentiation allows us to examine whether the effects of these courses differ by level of placement, as well as by subject area. As such, we are able to look across subject area and institutional type to draw conclusions about the effectiveness of these courses for students with a range of academic needs. We also examine a range of short-, intermediate-, and long-term outcomes, including 8-year completion rates, which is important when examining the effects for students who may have significant remediation to complete. 1
Using longitudinal data from the Tennessee Board of Regents (TBR) and the Tennessee Higher Education Commission (THEC), we isolate the effects of placement into developmental or remedial math, reading, and writing courses for students first attending public 2- and 4-year colleges and universities in Tennessee in the fall of 2000. We are able to examine the extent to which heterogeneous effects are detected along the student ability distribution—from students who need only one developmental course to those who need several courses—due to the state’s prior multitiered system in which students were assigned to one of four levels of math and one of three levels of reading and/or writing. We estimate the causal effects of enrolling in a certain level of remediation on the number of credits accumulated over time, persistence, and degree completion.
Our results suggest that remedial and developmental courses do differ in their impact by level of student preparation. We estimate the largest negative effects among students on the margin of needing one remedial course: In comparison with their peers placed in college-level courses, students assigned to courses one level below college level were less likely to complete a college degree in 8 years and completed fewer college credits over time. The negative effects on degree completion were largely concentrated among students attending 2-year colleges. Our findings are similar to those reported in previous research in this respect. However, students with lower levels of academic preparedness experienced much smaller negative effects from remediation, and in some cases, remedial courses are estimated to improve later student outcomes, particularly for students attending 2-year colleges. For example, we estimate that students placed in reading and writing courses two levels below college level are more likely to persist or attain a degree than similar students who were placed one level below college courses. These results suggest that remedial and developmental courses can either help or hinder students differently depending on their level of academic preparedness. Importantly, while most of the literature only examines the effectiveness of developmental courses for students at the margin of needing any remediation, our results suggest that more, rather than less remediation may be beneficial for students with weaker preparation upon initial college entry. These results suggest that states and institutions need not treat remediation as a singular policy but instead should consider it as an intervention that might vary in its impact according to student needs.
Literature Review
Two common hypotheses have surfaced as to the potential effects of college remediation on student success: First, attending remedial courses may provide students with the skills they need to be successful academically, thereby helping them to persist through to graduation. The second hypothesis, by contrast, presumes that remediation might actually slow down student progress in college as remedial courses rarely count toward graduation requirements. In addition, taking remedial courses may lead to lower self-esteem, higher frustration, and ultimately higher dropout rates (Bettinger & Long, 2007; Jacob & Lefgren, 2004). Of particular interest in this study are the effects of remedial courses not only for students on the margins of needing remediation but also for those further down the academic distribution. While students at the margin of needing any remediation could experience negative effects from having to take one course, the impact of developmental courses for students with lower levels of preparation could, in fact, be beneficial, especially for students with strong gaps in reading and writing ability, for example. The basic, foundational skills taught in these lower level courses may be more beneficial than the lessons taught in courses just below college level.
In recent years, an increasing number of studies have attempted to determine which of these competing hypotheses may explain the effects of college remediation on student outcomes. Of course, students who are less well prepared academically are more likely to be placed into remedial courses, making these students different from their nonremediated peers in important ways. Short of randomly assigning students on the margin of needing remediation to either remedial or college-level courses, it can be difficult to ascertain whether observed differences in student outcomes are caused by students’ enrollment in remedial classes, or whether they are instead explained by their lower levels of academic preparation—the very thing that required them to be remediated in the first place.
Several recent studies have tried to establish the causal effects of remediation using quasi-experimental designs with mixed results. Remediation was found to have positive effects on the probability of college persistence at a large state university in the Northeast and in 4-year colleges in Ohio (Bettinger & Long, 2009; Lesik, 2007). In most other cases, however, the results have been less positive. In a study of more than 100,000 community college students in Florida, Calcagno and Long (2008) found that assignment to developmental courses increased both persistence to the second year and the total number of credits completed, although not degree completion. A study of Texas students concluded that placement into remedial courses had little effect on the number of credits attempted, receipt of a college degree, or future labor-market earnings among students scoring around the test score cutoff (Martorell & McFarlin, 2011). And in a large urban community college system, Scott-Clayton and Rodriguez (2015) concluded that enrolling in remediation, for students at the margins of the cutoff, did not lead to positive effects on college outcomes. However, they also found that students just below the remedial cutoffs in both math and reading were no less likely to enroll in subsequent semesters, and that these students stayed enrolled for about the same number of terms as those just above the cutoff. Using data from the National Education Longitudinal Study of 1988, Attewell, Lavin, Domina, and Levey (2006) used a propensity score matching technique to create observationally similar groups of students, half of whom had taken remedial courses and half of whom had not, and concluded that it was less probable that students in remedial courses would receive a bachelor’s degree but no less probable that they would receive an associate’s degree or certificate.
The results from these studies suggest that the causal effects of remedial courses on student outcomes are primarily null to negative for students at the margin of passing out of remediation. Furthermore, much of the previous research has been limited in its focus on students just on the margin of needing remedial courses. Research that compares students who score just above and below the cutoff on the remediation placement exam does not provide any information on the effects of remedial courses on students with lower levels of academic preparation. As previously mentioned, two prior studies have examined the effects that lower levels of remedial courses have on student success. Dadger (2012) asks whether students placed into the lowest levels of math remediation would face better or worse outcomes than if they were allowed to take only two remedial courses instead of three. Using data from Virginia’s community colleges and a similar RD design to this study, she finds that allowing students to skip the lowest level of remedial math would have increased the likelihood of attaining an associate’s degree or a certificate within 4 years by 9 to 15 percentage points. From this large negative effect, she concludes that the lowest levels of remedial math courses are hindering student degree completion in the long run. Hodara (2015) examines a similar lower level discontinuity but in the area of developmental writing and ESL (the course one level below developmental writing). Using 10 years of data from an urban community college system, she employs a difference-in-differences approach to consider both the differences in assignment to developmental writing versus ESL courses and the probability of students being assigned an ESL flag across campuses. Similarly to Dadger (2012), she finds that enrolling in the lowest level ESL course leads to lower rates of persistence through college for language minority students, but the effects differ based on students’ status as a first-generation college student. Both of these studies support the notion that the lowest level remedial courses in math and English may not be helping students to be successful in subsequent terms. In this study, we bring together both subject areas to employ an RD design estimating causal effects of enrolling in differing levels of math, reading, and writing remedial courses on subsequent student outcomes.
Background and Context on Remediation in Tennessee
Tennessee has a higher education system similar to most other midsized states. In the fall of 2000, the cohort of this analysis, there were six public universities and 13 two-year institutions in the TBR system that collectively served more than 200,000 students, excluding the three University of Tennessee institutions. While the Tennessee public system does not enroll as many students as some neighboring states, such as North Carolina or Georgia, it does report similar tuition rates and student body characteristics to other regional and national averages. In the year 2000, the public colleges in Tennessee enrolled similar percentages of recent high school graduates and women, and charged similar average tuition at their 2-year and 4-year colleges as compared with the other 16 Southern Regional Education Board colleges, as well as institutions across the country. See Appendix Table A1 for further comparisons across these regions. Tennessee also offers an appropriate complement to the college samples in prior studies on remediation. Many of these studies focus on either the 2-year (Calcagno & Long, 2008; Dadger, 2012; Hodara, 2015; Scott-Clayton & Rodriguez, 2015) or 4-year sector (Bettinger & Long, 2009; Lesik, 2007) but are not able to examine how these effects might differ across sectors.
The TBR colleges are largely open-access institutions: Four of the six universities accept approximately 90% of all applicants, with one university accepting 70% and another accepting just half (U.S. News and World Report, 2015). These less-selective 4-year colleges, combined with the 13 community colleges, make Tennessee an ideal state for studying the effects of remedial courses on student outcomes, given the large numbers of students enrolled in public colleges with remedial needs. In the year 2000, more than 70% of students beginning at a community college and more than 50% of students beginning at a 4-year college enrolled in a remedial course in Tennessee. Descriptive analyses comparing students enrolled in developmental courses with all other students enrolled in college-level courses reported that students in developmental courses had higher failure rates and lower rates of persistence from 1 year to the next (Gray-Barnett, 2001). In the early 2000s, the Drop/Failure/Withdrawal rates averaged 45% in developmental mathematics courses, compared with 26% in college-level mathematics.
Unlike the literature, which tends to use the terms remedial and developmental interchangeably, in Tennessee, public 2- and 4-year colleges make a distinction between the two terms, with developmental courses offered just below college level and remedial courses offered below developmental courses. For example, in the year 2000 in Tennessee, students who lacked basic computational arithmetic skills (addition, subtraction, multiplication, division) would be placed into Remedial Arithmetic (the lowest level course three levels below college-level math), whereas students who only lacked the ability to do algebraic computations would be placed in Developmental Algebra I or II (two levels below college-level math or one level below, respectively). These courses are in contrast to college-level math courses, which are not considered remedial or developmental. Tennessee also offers remedial and developmental courses in reading and writing. These courses are intended to help prepare students for their college English, composition, and/or expository writing courses, and are heavily focused on reading comprehension and essay writing. The specific courses used to satisfy remedial and developmental reading and writing credits vary by institution, but these courses are commonly known as introductory composition or English courses. Similar to math, Tennessee previously had multiple levels of reading and writing courses. Remedial Writing was the lowest level writing course (two levels below college-level English), and Developmental Writing was just one level below college level. The same hierarchy was used for reading courses in Tennessee.
Typically, remedial and developmental courses at public institutions in Tennessee are offered for credit and count toward a student’s overall grade-point average (GPA) but rarely are they counted toward graduation requirements. In the year 2000, students could be placed into remedial or developmental courses in multiple subjects, depending on their scores on each subject’s Computer Adaptive Placement Assessment and Support System (COMPASS) placement exam. For example, students could be assigned to both Developmental Algebra II and Developmental Writing in the same semester. Research suggests that students enrolled in remedial or developmental reading are more likely to be enrolled in other remedial or developmental courses (U.S. Department of Education, National Center for Education Statistics, 2004). In Tennessee, among all students recommended for at least one remedial or developmental course in any subject, 17% were recommended for a second course in another subject, and 5% were recommended for all three.
How students were assigned to courses in Tennessee allows for estimating the causal effects of remedial and developmental classes. In Figures 1 to 3, we illustrate the placement process for math, reading, and writing courses, respectively, for the 2000 cohort of entering students. Tennessee used a combination of ACT subscores and scores on a COMPASS exam to assign students to remedial and developmental classes. In Tennessee, more than 85% of students took the ACT. In the few cases where a student had only a reported SAT score, the score was converted into an ACT score using the SAT percentile equivalent. Students with an ACT subscore less than 19 in math, less than 19 in English, and less than 19 in reading, or those 21 years of age or older were required to complete the COMPASS assessment in the respective subjects. The COMPASS test is an untimed, adaptive, computerized test that measures skills in reading, writing, and math. In math, students with the lowest ACT scores were assigned to take the COMPASS Arithmetic exam, and students with slightly higher ACT scores took the Algebra exam. In reading and writing, there was only one exam per subject. COMPASS tests were administered prior to the start of the academic year, and students received a score from 0 to 100. Colleges then used these scores to assign students to remedial or developmental courses according to the statewide policy in Tennessee. For example, students scoring 50 or above on the Algebra II test were assigned to college-level math, whereas those scoring 49 or below should have been assigned to Developmental Algebra II. For those students taking the Arithmetic test, a score of 30 or higher placed a student into Developmental Algebra, and a score of 29 or lower placed a student into Remedial Arithmetic. As shown in Figure 1, students in Tennessee in the year 2000 could technically be assigned to Developmental Algebra I through one of two avenues: (a) by scoring below a 28 on the COMPASS Algebra II test or (b) by scoring a 30 or higher on the COMPASS Arithmetic test. In practice, more than 80% of students assigned to Developmental Algebra I took the COMPASS Arithmetic test. Because the Arithmetic and Algebra II tests are different testing instruments, we do not compare the students assigned to Developmental Algebra I (the vast majority of whom took the Arithmetic test) with the students assigned to Developmental Algebra II (who took the Algebra II test) in this article.
Remedial math placement policy in Tennessee, Fall 2000.
Remedial reading placement policy in Tennessee, Fall 2000.
Remedial writing placement policy in Tennessee, Fall 2000.
As shown in Figure 2, a score of 53 to 67 on the COMPASS reading exam placed students into Developmental Reading, and that below 53 placed students into Remedial Reading. And in Figure 3, a score of 28 to 67 on the COMPASS writing exam placed students into Developmental Writing, and a score below 28 placed students into Remedial Writing. Because the policy was set at the state level, 2-year colleges administered the same COMPASS exams and adhered to the same remediation placement cutoffs as 4-year colleges.
Tables 1 and 2 provide background information on the number of students who took the COMPASS placement tests in math and reading/writing, respectively. Among students in the sample who took a COMPASS math placement test, 89% were placed into some level of Remedial or Developmental Math. This number does not include students who did not take a COMPASS math exam and who were admitted into college-level math courses for other reasons (most commonly due to an ACT score above an 18). Relatively equal proportions of students were placed into the highest and lowest level developmental and remedial math courses. Among all students in the sample who took a COMPASS reading placement test, 43% were recommended for Remedial or Developmental Reading, and among those taking a COMPASS writing exam, 64% were recommended for Remedial or Developmental Writing. In both reading and writing, most students were recommended for the developmental course only one level below the college-level course.
Course Assignment Distribution for Those Students Taking a COMPASS Placement Exam—Math
Note. RD = regression discontinuity; COMPASS = Computer Adaptive Placement Assessment and Support System.
Course Assignment Distribution for Those Students Taking a COMPASS Placement Exam—Reading and Writing
Note. The sample is limited to students who began at a Tennessee public college or university in the fall of 2000 below the age of 21 who began full-time and whose assignment to remediation was based on their scores on a COMPASS math exam. RD = regression discontinuity; COMPASS = Computer Adaptive Placement Assessment and Support System.
Beyond simply having to spend more time and money in remedial classes as a result of being assigned to one, two, or three remedial courses, we hypothesize that there may be other important differences between these treatments. In their paper examining the “development, discouragement, or diversion” role of remediation, Scott-Clayton and Rodriguez (2015) conclude that one of the primary effects of remediation is to divert students from college-level courses into remedial courses. This finding is particularly relevant for students with COMPASS scores near the cutoff for college-level courses in Tennessee. As a result of their placement into a developmental course one level below college level, these students may experience a stigma associated with this placement, may experience curricula that are not at the appropriate level, or other responses that may result in negative outcomes, particularly when compared with their college-level peers. Alternatively, students assigned to two or more levels of remedial courses undoubtedly need some level of remediation, which could lead students to benefit from these courses in different ways from those at the upper end of the distribution who may not need the course to begin with. Our analysis examines these hypotheses in a statewide context.
Data and Empirical Framework
Data
THEC and the TBR provided the student-level data for this study. These organizations both collect enrollment information and transcript data, including courses taken and grades, for each student in every term of active enrollment at a public institution in the state. Information is also available on demographic characteristics, high school background, and test scores. The dataset also includes COMPASS placement exam scores for all students and their subsequent assignment into remedial, developmental, or college-level courses based on the exam. The degree completion data are collected directly from TBR and the individual institutions and, therefore, only include students who attained a degree or certificate from a Tennessee public college. While this means we are not able to capture degree completion for students who began at a public college and later transferred out of state or to a private college within the state, the number of students transferring from public to private colleges within the state is relatively small. In 2014–2015, only 8% of the 30,354 transfer students in the state left the public system to enroll in a private college within the state (THEC, 2015). In this study, we track student outcomes over 8 years from fall 2000 through the 2008–2009 academic year.
The sample is restricted to undergraduates beginning at any public 2-year or 4-year TBR college in Tennessee in fall 2000 who took a COMPASS math, reading, or writing exam. We exclude students whose ACT subscore in math, reading, or English was high enough to place them directly into the respective college-level course. In addition, the sample only includes full-time students so that we are better able to determine their progress through college. The vast majority of students in the sample (89%) began as full-time students, making this a relatively weak restriction. We also limit the sample to students aged below 21, as more than 85% of this group have a valid ACT score upon college entry. From a methodological standpoint, this is the group for which we have the most complete data on prior academic achievement, which is crucial for determining placement into remedial courses, whereas this information is only available for a subset of older, continuing students. We recognize that the effects of placement into remedial courses may be different for students attending 2-year colleges compared with students attending 4-year colleges due to potential differences such as class sizes, instructional practice, and the availability of support service. However, placement into these courses is pervasive within both types of institutions. Still, we include both 2-year and 4-year institutions in the analysis and investigate whether there are heterogeneous effects by institution type. We discuss the implications of these sample restrictions on the generalizability of our findings in Section “Data and Empirical Framework.”
The student-level descriptive statistics for the sample are reported in Tables 3 and 4. We present sample means for a group of students within a selected bandwidth of points on either side of the COMPASS cutoff used to assign students to their remedial courses in Tennessee. More than 55% of the students in our sample are women, and more than 70% are White. As shown in Table 2, students assigned to the developmental math course one level below college math appear similar to their peers assigned to college-level math in terms of race, gender, financial aid status, and the type of college attended. Students assigned to Remedial Arithmetic (three levels below college math) do have, on average, lower ACT composite and ACT math test scores (about half an ACT point) than those students assigned to Developmental Algebra I (two levels below college math). Although this is a small but statistically significant difference, it may not be surprising given that ACT also created the COMPASS exam by which students are placed into remedial courses. Given the high positive correlation between the ACT and the COMPASS exam, we observe students with lower ACT scores scoring below the COMPASS cutoff and students with higher ACT scores scoring above.
Summary Statistics of the Sample Around the Math Placement Cutoffs
Note. Standard deviations/standard errors shown in parentheses. The sample is limited to students who began at a TBR public college or university in the fall of 2000 with complete information on gender, race, age, high school grade-point average, college financial aid, and the postsecondary institution attended. The sample is also limited to students below the age of 21 who began full-time and whose assignment to remediation was based on their scores on a COMPASS math exam (either the COMPASS Arithmetic or COMPASS Algebra II, as detailed in Figure 1). RD = regression discontinuity; TBR = Tennessee Board of Regents; COMPASS = Computer Adaptive Placement Assessment and Support System.
p < .05. **p < .01. ***p < .001.
Summary Statistics of the Sample Around the Reading and Writing Placement Cutoffs
Note. See notes in Table 3. RD = regression discontinuity.
Table 4 illustrates similar descriptive statistics for students who have taken a COMPASS reading and/or writing exam. Generally, the profile of these students looks similar to students taking a math exam with regards to race, gender, and financial aid status. More than 60% of the students are White, and more than 20% receive some kind of need-based financial aid. Students taking a reading and/or a writing exam are more likely to attend a 2-year college than students taking a math exam, particularly those assigned to the Remedial Reading and Writing, both two levels below college level. Again we see differences across groups on both the ACT composite and ACT English scores, and in the case of the lowest level of Remedial Reading, this difference amounts to over a full point on the ACT exam. In addition, students in the sample assigned to Developmental Reading and Writing courses one level below college level are more likely to be African American and less likely to be White. These differences in race, however, disappear when we restrict the sample to 2- and 4-year institutions.
Outcome Measures
This study examines the effects of remedial courses on students’ credit accumulation, success in their college-level courses, and enrollment decisions by examining outcomes in the short, intermediate, and long term. Of particular interest in studies on college remediation efforts is whether assignment to remedial courses slows students down in their early progress toward a degree so much so that they become discouraged and stop out of college (Armstrong, 1999; Jenkins, Jaggars, & Roksa, 2009). In the short term, we explore the persistence of students from the first to the second year of college (the fall of 2001 for the 2000 cohort of students). Persistence is captured by a dichotomous variable equal to 1 if the student is enrolled in the fall of 2001. We also examine students’ performance in their first college-level math and/or composition course. One of the central goals of remediation is the successful completion of college-level coursework. As such, we create a dichotomous variable equal to 1 if a student earns a grade of C or higher in her first college-level course in the corresponding subject (college-level math for math discontinuities and college-level composition for reading and writing discontinuities), and 0 otherwise. For students who never enroll in a college-level math or composition course, this outcome variable is coded 0.
In the intermediate, we explore the effects of remediation on the total number of credits completed by the third year, as well as the total number of college credits completed by the end of the third year. While the number of cumulative credits in the third year may be a good indication of student progress toward a degree, it is the number of college credits completed over time that is most indicative of progress toward degree attainment. Examining the data, we see that the number of total credits accumulated after 3 years differs little among students assigned to a developmental math, reading, or writing course one level below college level compared with students assigned to the remedial courses two or three levels below. However, a larger gap exists in the number of college credits completed over time for students assigned to the higher versus lower level courses. We include both the number of total credits and the number of college credits as outcomes to further explore the effects of remediation on credit accumulation.
Finally, we examine the effects of placement into remedial and developmental courses on student degree attainment after 8 years. In the long term, we are interested in whether there are differences in attaining a certificate, associate’s degree, or a bachelor’s degree for similar students on the margins of placement into differing levels of developmental math, reading, and writing.
Empirical Strategy
This article uses an RD design to tease out the causal effects of being placed into a remedial or developmental course in Tennessee. An RD design compares outcomes for students whose COMPASS scores fall just above and below the cutoff score(s). The analysis assumes that any differences across students on either side of the cutoff is captured by the placement test score, which is used to determine enrollment in college-level or developmental courses. Otherwise, we assume that students are equivalent, on average, on all observed and unobserved dimensions and differ only in terms of their course placement (Urquiola & Verhoogen, 2009). The statistics in Tables 3 and 4 do indicate that students may differ on a select few observable characteristics (ACT composite score, for example) right around the cutoff, but as can occur in randomized experiments, we believe that a difference between groups on one or two of these covariates does not threaten the internal validity of our study. This allows us to compare the enrollment patterns of students assigned to remedial courses two or three levels below college-level courses with those of students in courses only one level below. Similarly, we compare students in courses one level below with student placed into college-level courses in an analysis that provides an unbiased estimate of the causal impact of being placed into the lower level for students on the margin of passing the remedial exam (DesJardins & McCall, 2014; Shadish, Cook, & Campbell, 2002). In addition, as we only examine students within a narrow bandwidth around the cutoff, the results do not pertain to those students whose scores fall well below or above the cutoff score.
In Figures 4 to 6, we illustrate the number of students recommended for remediation by the distribution of student test scores on the COMPASS math, reading, and writing exams, respectively, as well as the percentage of students actually enrolling in these courses by the distribution of COMPASS scores. The graphs are centered at the cutoff for remediation placement by subject and course, and are displayed by subject area (math, reading, and writing) with the top two graphs depicting the placement policy for the two levels of remedial courses within that subject and the bottom two graphs displaying actual student enrollment in these courses. In each of the figures, it is evident that some students were assigned to the lower level course when they should have been assigned to the more advanced course and vice versa. The same is true for enrollment: While almost no students with COMPASS scores above the cutoff enrolled in the lower level course, it is clear some students with scores below the cutoff did not enroll in their assigned developmental course.
Percentage of students assigned to developmental or remedial math by ACT math score, 2000 cohort.
Percentage of students assigned to developmental or remedial reading by ACT math score, 2000 cohort.
Percentage of students assigned to developmental or remedial writing by ACT math score, 2000 cohort.
The data reveal that students closest to the cutoff are the most likely to be granted an exception to the assignment policy. In speaking with Tennessee officials, it appears that many of these exceptions originate from individual counselors either promoting or demoting students due to their exceptionally close proximity to the cutoff. While assignment exceptions are evident in all subjects, the most exceptions appear in writing courses. To address our research question in light of this “fuzzy” discontinuity, we adopt an instrumental variables strategy using two-stage least squares (2SLS) by which we treat assignment to developmental or remedial courses as the instrument for enrollment in these courses. This approach provides an estimate of the LATE for students who complied with their assignment to remediation based on the Tennessee cutoff policy. The LATE estimates pertain only to those students who actually enrolled in a developmental or remedial math, reading, or writing course. In the first stage of the analysis, we fit a model in which we regress whether a student enrolls in a developmental or remedial course on whether a student was assigned to this course based on his or her COMPASS test score as follows:
where DEVi is a dichotomous variable that indicates take-up of the assignment, or whether student i actually enrolled in a developmental or remedial course (1 = enrolled, 0 = otherwise) in the fall semester. More than 85% of students assigned to these courses enrolled in the fall semester. 2 Less than 1% of students whose COMPASS scores are higher than the remedial cutoff score (and thus, are assigned to college-level courses) enrolled in a developmental course. The variable ASSIGNi serves as the instrument and takes a value of 1 if students scored below the defined state cutoff on the forcing variable, and 0 otherwise. ASSIGNi is a sound instrument as it is strongly correlated with enrollment in developmental courses, and is exogenously determined by state policy. SCOREi is a continuous variable that measures a student’s score on the COMPASS math, reading, or writing exam. Finally, Zi is a vector that includes information on student background and college choice, including gender, race, enrollment status in the fall of 2000, and financial need, and δ i is a residual term.
In the second stage of the analysis, the fitted probability of being assigned to developmental courses obtained from the fitted model in Equation 1 is used to estimate the causal effect of remediation on outcome Yi as follows:
In this second-stage model, we use generic outcome Yi to refer to one of the six primary outcomes in our analysis. These outcomes include persistence from the first to the second year, whether a student eventually passed his or her college-level math course, the number of total credits and college credits accumulated by the third year, and completion of any degree or certificate, as well as completion of a bachelor’s degree after 8 years. β1, our coefficient of interest in Equation 2, represents the causal effect of enrollment in remediation on the outcome of interest. For continuous outcomes, we specify a second-stage ordinary-least-squares (OLS) regression model to estimate Equation 2. For the binary outcomes, we fit a linear probability model using maximum-likelihood estimation for which we assume normally distributed errors. All models in the first and second stage calculate robust standard errors to account for heteroskedasticity.
To ensure sufficient statistical power in our analyses, we pool data across all colleges in our estimation procedure rather than examining effects within individual institutions. However, we include institutional fixed effects in both stages of the estimation to account for differences between institutions and for the nonrandom clustering of students within these institutions. As such, we are estimating the weighted average of the within school effect across all Tennessee public institutions. It is quite likely, however, that there are different effects of being assigned to remedial courses for students attending 2-year colleges compared with students attending 4-year colleges (Bettinger & Long, 2007). To allow for this possible variation, we also include separate specifications where we limit the sample to include only 2-year and 4-year colleges.
The RD design assumes that students cannot manipulate their scores to make themselves fall just above or below the cutoff, and as such, the cutoff score is truly exogenous to the outcome (McCall & Bielby, 2012; Shadish et al., 2002). If students could increase their probability of passing out of remediation, perhaps by retesting, then the critical assumption of the RD design would be violated. In Tennessee, when retesting did occur, it was largely concentrated in only a few institutions. In these cases, administrators were granting exceptions to students given extenuating circumstances, or based on other information provided by the student, such as prior math or English courses taken in high school. These cases occurred at only a few institutions and are not cause for concern as the IV strategy used in the “fuzzy” RD design accounts for these issues of noncompliance by treating assignment to developmental or remedial courses as the instrument for enrollment in these courses. As a further specification check, we excluded institutions in which retesting was permitted, and the results did not change. Furthermore, it is important that we not observe endogenous sorting around the cutoff, which could be a sign that scores were somehow manipulated. This is a reasonable assumption in this case as it seems unlikely that a student would be able to manipulate his or her score to fall just above or below a cutoff score. However, to test for the possibility, we plot the raw data to check for jumps at the cut score by conducting a McCrary (2008) density test to ensure that the distribution of test scores is smooth about the cutoff. These tests (available upon request) reveal the distribution to be smooth near the cutoff score, and support the notion that students are not able to manipulate their COMPASS scores around the cutoff.
We estimate our regression equations with local linear regression utilizing the Imbens and Kalyanaraman (2012) optimal bandwidth procedure. The preferred bandwidth changes slightly as our sample of students changes, particularly for students attending 4-year colleges versus two-year colleges. Across all specifications, the bandwidth ranges from ±5 points to ±9 points, as indicated in each of our main results tables. As is common in studies utilizing a RD design, we explore a number of specifications. We reestimate the findings for alternative functional forms by including a quadratic term for the question predictor in Equation 2, and find that the estimates do not change significantly. We also test whether the results are robust when background controls were eliminated, which had no effect on the estimates. In addition, we test whether the results were sensitive to the choice of bandwidth by increasing it to as many as 14 points on either side. Overall, we found that these efforts did not change our estimates in a significant way, as evidenced in Appendix Tables 1 to 3, available in the online version of the journal.
Results
In the following section, we present the results for separate discontinuities by subject and level of academic preparedness. This yields six separate discontinuities. For each subject (math, reading, and writing), we estimate two sets of results: one for students assigned to the highest level developmental course (one level below college level) compared with their peers assigned to the college-level course, and another for students assigned to the lowest level remedial course (three levels below in math and two levels below in reading and writing) compared with students assigned to the next highest level developmental course (two levels below in math and one level below in reading and writing). We further divide the sample into students beginning at a 4-year college in the fall of 2000 and students beginning at a 2-year college in the fall of 2000 to examine differences around the cutoffs by institutional sector.
Math: College-Level Versus Developmental Algebra II (RD1)
The left panel in Table 5 presents short- and medium-term results, through the first 3 years of college, for students assigned to Developmental Algebra II (one level below college math) compared with similar peers assigned to college-level math. We observe large effects of placement into Developmental Algebra II courses over college-level math courses on students’ early persistence from the first to the second year. Students beginning college in the fall of 2000 in Developmental Algebra II are 9.3 percentage points less likely to be enrolled in the fall of 2001 compared with those enrolling in college-level math in the fall of 2000. These effects appear to be driven by the differences at the 4-year colleges. Column 2 presents the impact of assignment to Developmental Algebra II on the likelihood of passing the first college-level math course with a course grade of C or better. For the full sample, as well as the 2- and 4-year institutions, assignment to developmental math does not appear to have a statistically significant effect on eventual successful college-level math course completion.
Short- to Medium-Term Impacts on Assignment to Developmental Math Versus College-Level Math, and Developmental Math Versus Remedial Math
Note. Robust standard errors are shown in parentheses. The sample is limited to students who began at a TBR public college or university in the fall of 2000 with complete covariate information on gender, race, age, high school grade-point average, college financial aid, and the postsecondary institution attended. The sample is also limited to students below the age of 21 who began full-time and whose assignment to remediation was based on their scores on a COMPASS math exam. The bandwidth on either side of the cutoff is calculated using the procedure developed by Imbens and Kalyanaraman (2012). The statewide cutoff policy used to assign students to developmental math courses is used as an instrument for enrollment in developmental math. For students who drop out, the number of credits is listed as the number of credits the student had completed when last enrolled. Passed college-level math is equal to 1 if a student earned a C or higher in his or her first college-level math course. If a student never enrolled in college-level math, this is coded as zero. RD = regression discontinuity; M = mean at the cutoff; TBR = Tennessee Board of Regents; COMPASS = Computer Adaptive Placement Assessment and Support System.
p < .05. **p < .01. ***p < .001.
Column 3 shows that students in Developmental Algebra II have already fallen behind their peers assigned to college-level math as measured by the number of total credits they have accumulated by the end of their third year. Students enrolled in Developmental Algebra II have an average of 11.5 fewer total credits by the end of their third year compared with their peers assigned to college-level course. This is equivalent to the number of credits a student will typically earn over an entire semester. These negative effects on the total number of credits completed by the end of the third year appear to be largely driven by the differences in the 4-year colleges. There are no statistically significant differences in the number of college-level credits taken by students by the end of the third year.
Graphically, this difference can be seen in Figure 7 where each circle represents the percentage of students placed into either college-level math or Developmental Algebra II. The size of the circle represents the relative number of students reporting a COMPASS math score in the fall of their first year. The vertical line is drawn at the statewide cutoff for placement into college-level math. Students to the left of the cutoff have significantly fewer total credits after 3 years than students with COMPASS scores to the right of the cutoff. A similar relationship is observed for the number of college credits, although this is not a statistically significant effect.
Math—College-level versus developmental course in (RD1).
Math: Developmental Algebra I Versus Remedial Arithmetic (RD2)
Similar to the effects of math remediation for students close to the college-level cutoff (RD1 above), we estimate the effects of enrolling in the lowest level of remedial math (three levels below college level) compared with enrolling in Developmental Algebra I (two levels below college level). We do not see significant effects on persistence to the second year or eventually passing college-level math at this discontinuity, although it is interesting to note that the direction of the effect has changed from negative to positive in columns 5 and 6 of Table 5.
As shown in the right panel of Table 5, placement into Remedial Arithmetic results in students accumulating fewer total and college-level credits over time, but unlike in RD1, these effects are not statistically significant. At the upper end of the math distribution scores (the left panel in Table 5), we observed students completing 11 to 14 fewer credits than their peers assigned to college-level math. In contrast, at the lower end of the math distribution, we observe students completing only around one or two fewer credits, and these results are not statistically significant. As a fraction of the total number of credits completed, this is a notable difference. So, while students assigned to the lowest levels of math remediation still take fewer college-level credits after 3 years compared with their peers in the next highest course, this difference does not appear to be as dramatic as for students who just barely missed the cutoff for placement into college-level math.
Across all outcomes, the effects of enrolling in Remedial Arithmetic are much smaller in magnitude than those observed for those students needing less intensive math remediation in Developmental Algebra I, and fewer are statistically significant. Many of the estimates for the full sample are a half to two thirds the size of the effects reported in the left panel of Table 5 for RD1 in math. The effects among 4-year colleges remain large in magnitude even at the low end of needing remedial math; however, none of these effects are statistically significant.
Reading: College-Level Versus Developmental Reading (RD1)
The left panel in Table 6 presents the early and medium-term results for the effects of enrolling in the first level of Developmental Reading compared with the first College-Level English/Composition course. We see no statistically significant differences in the early college outcomes of students on the margins of needing reading remediation. Students placed into Developmental Reading (one level below college reading) are completing over seven fewer college credits by the end of their third year as compared with their peers placed into college-level English. This is the equivalent of approximately two college-level courses. These magnitudes are larger in the 4-year colleges at almost 11 fewer college-level credits after 3 years but not statistically significant among students attending 2-year colleges. The effects on credit accumulation can be seen graphically in Figure 8. Unlike the negative effects observed among students assigned to the higher level developmental math course, it appears that students in the higher level developmental reading course do not significantly differ on their early college outcomes from their peers assigned to college-level English.
Short- to Medium-Term Impacts on Assignment to Developmental Reading Versus College-Level Reading, and Developmental Reading Versus Remedial Reading
Note. See notes in Table 5. RD = regression discontinuity; M = mean at the cutoff.
p < .05. **p < .01. ***p < .001.
Reading—College-level versus developmental course (RD1).
Reading: Developmental Versus Remedial Reading (RD2)
Placement into the lowest level of Remedial Reading (two levels below college level) appears to have positive effects on student persistence and success in the first college-level English/composition course when compared with students placed into Developmental Reading (one level below college level). In the right panel of Table 6, we observe that the negative effects found for students on the margins of needing any remediation (RD1 above) either disappear or become positive for students at the lower end of the reading distribution. Whereas students placed into Developmental Reading at the upper end of the distribution are taking over six fewer college credits by the end of their third year compared with their peers in college-level English, students placed into Remedial Reading appear to be completing more total credits than their peers placed into Developmental Reading. While the effects on the number of college credits completed are not statistically significant for the overall sample, we see large effects on both total credits and college-level credits when extending our analysis to only the 2-year colleges. Students enrolling in Remedial Reading in the 2-year colleges have earned over 13 more total credits and almost 12 more college credits after 3 years than those students assigned to Developmental Reading. As stated earlier, this is roughly equivalent to an entire term. Across both 2-year and 4-year colleges, we observe large increases in student persistence to the second year, a difference of 7.8 percentage points across groups, and an even larger effect of 17.7 for students attending 2-year colleges. This is a large and statistically significant effect which we explore in more detail in the discussion below.
From a substantive perspective, the magnitudes in the 2- and 4-year only estimates are noteworthy as they are quite large and in concert with the size of effects detected in Math RD1, although in the opposite direction (column 4 in Table 5). It appears that enrolling in Remedial Reading (two levels below college level) may lead to positive outcomes for students most in need of these basic remedial skills when compared with their peers enrolled in Developmental Reading (one level below college level).
Writing: College-Level Versus Developmental Writing (RD1)
As shown in the left panel of Table 7, placement into the highest level of Developmental Writing (one level below college level) appears to have negative effects on student persistence, passing college-level composition, and credit accumulation within the first 3 years. When comparing students assigned to Developmental Writing with students assigned to College Composition, we see statistically significant differences in persistence from the first to the second year. Students on the margins of needing writing remediation (i.e., those recommended for Developmental Writing) were 10.8 percentage points less likely to enroll in their second year compared with their peers assigned to College Composition (column 1 in Table 7). Students placed into Developmental Writing are also 7.5 percentage points less likely to pass their first college composition course, and also have fewer total credits after 3 years—a pattern consistent with students at the upper developmental level of math as well. Students placed into Developmental Writing have completed 7.9 fewer total credits than their peers placed into College Composition after 3 years. They also have fewer college-level credits after their third year. These effects are the most significant for students attending 2-year colleges.
Short- to Medium-Term Impacts on Assignment to Developmental Writing Versus College-Level Writing, and Developmental Writing Versus Remedial Writing.
Note. See notes in Table 5. RD = regression discontinuity; M = mean at the cutoff.
p < .05. **p < .01. ***p < .001.
Writing: Developmental Versus Remedial Writing (RD2)
Similar to the RD1 analysis for reading, we estimate positive effects on early student persistence for students assigned to the lowest level of writing, Remedial Writing (two levels below college level). In addition, students assigned to Remedial Writing are 5.5 percentage points more likely to eventually pass their college-level composition course than their peers assigned to Developmental Writing. Columns 7 and 8 of Table 7 provide suggestive evidence that these students may have actually accumulated more credits by the end of their third year when compared with students assigned to Developmental Writing (one level below college level), although neither of these estimates are significant for the full sample. At the lowest levels of writing, however, it appears that students assigned to Remedial Writing are more likely to be enrolled (6.5 percentage points) in college in their second year than their peers assigned to Developmental Writing (column 5). These results are the largest and most significant for students beginning in 2-year colleges. In the case of both reading and writing, the difference in effects by level of academic preparedness presents the starkest evidence of heterogeneous effects according to incoming levels of academic preparation.
Long-Term Effects: Degree Completion Within 8 Years
It may very well be the case that the early differences observed above in math, reading, and writing fade, or even disappear entirely over time. A critical policy outcome concerning academic readiness is eventual degree completion. Table 8 presents the effects of assignment to remedial courses on degree completion within 8 years across all six of the discontinuities discussed above. As shown in columns 1 and 2, students in Developmental Algebra II in the full sample are 5.3 percentage points less likely to obtain a bachelor’s degree, associate’s degree, or a certificate within 8 years after initial enrollment, and 6.2 percentage points less likely to obtain a bachelor’s degree as compared with their peers enrolled in college-level math. The magnitudes of these effects on degree completion are quite large, and appear to be driven largely by students initially enrolling in 2-year colleges. At the lower end of the math distribution, we see no statistically significant differences in overall 8-year degree completion rates (columns 3 and 4) and smaller point estimates.
Long-Term Degree Completion Outcomes After 8 Years: Math, Reading, and Writing
Note. See notes in Table 5. RD = regression discontinuity; M = mean at the cutoff.
p < .05. **p < .01. ***p < .001.
Columns 5 to 8 display the results for developmental and remedial reading. Despite the gap highlighted earlier in the number of college-level credits in Year 3 for students in Developmental Reading compared with college-level English/composition, we did not observe statistically significant effects on degree completion within 8 years. At the lower end of the distribution, it appears as if enrolling in Remedial Reading as opposed to Developmental Reading results in students being 6.3 percentage points more likely to complete any degree or certificate within 8 years.
The effects of Developmental and Remedial Writing on degree completion are particularly notable. For RD1 (columns 9 and 10), we do not observe the earlier gains in credits persisting into subsequent degree completion either for the full sample or for students attending community colleges. Students in developmental writing on the margins of placement into college composition are, in fact, more than 10 percentage points less likely to complete any degree or certificate within 8 years. The estimate jumps to 21.3 percentage points for students beginning at a community college. We discuss the magnitude of these results below. Interestingly, this trend reverses for students at the lower end of the writing distribution. Students two levels before college-level writing may actually be more likely to attain any degree or certificate compared with their peers in Developmental Writing, although these effects are marginally statistically significant for only the full sample of students.
Comparisons Across Discontinuities
The results presented above provide evidence as to the effects of placement into a remedial math, reading, or writing course compared with placement into the next highest course in the sequence. A primary contribution of our article is to draw out these distinctions, and as such, we present the results separately for each discontinuity. However, it may be the case that the estimates across both of the math discontinuities, for example, may appear different in magnitude in isolation but are not actually statistically different from one another. To test whether the differences presented within subject in Tables 5 to 8 are actual, we stacked the data and treated the discontinuities within each subject as the same, including a dummy variable in the analysis to identify the original discontinuity. This analysis confirmed that the discontinuities are statistically different from one another and thus supports comparison across the results discussed above. Results from these analyses are available upon request.
Limitations
While our study has strong internal validity, we do acknowledge that our sample restrictions on age, initial enrollment status, and completion of the COMPASS math exam do influence the external validity of our results. Remedial needs are not isolated to traditionally aged college students or full-time students. In excluding older, continuing students, we acknowledge that our results do not necessarily apply to this important population of students, many of whom also have remedial needs. Future work should investigate these issues with this in mind.
One of the primary concerns with RD studies is the large sample size around the cutoff needed to detect meaningful effects. While we do have a uniform sample size across all outcome variables within a single discontinuity, a few of our discontinuities only have enough statistical power to be able to detect relatively large effect sizes. To be confident we are estimating actual zeros, we incorporate an additional cohort of students into a secondary analysis presented in Appendix Tables 6 to 8, available in the online version of the journal. We have added the 1997 freshman cohort to our analysis of the 2000 cohort, as there were few changes in remediation policy at either the institutional or state level from 1997 to 2000. Based on a comparison of student characteristics shown in Tables 3 and 4 and the online Appendix Tables 4 and 5, the 1997 and 2000 data do not differ dramatically from one another. The most notable difference between these cohorts was in the number of students taking the COMPASS exam, with more students assigned to take the COMPASS exam 1997. By 2000, the state was allowing more students to “test out” using their ACT scores. In the online Appendix Tables 6 to 8, we then combine these two cohorts of students, including a dummy variable for the students’ first year enrolled. This results in a larger sample size for each of our discontinuities. Importantly, we see that the results do not change significantly from our initial analysis, thereby confirming our main findings. For example, we observe the same pattern of negative effects for students on the margins of placement into college-level math but little to no effects for students at the lower end of the math distribution. Similar positive effects are observed for placement into lower level writing courses as well. Based on these additional data, we conclude that our zeros are actual and not the result of certain components of the analysis being underpowered.
Discussion and Implications
The effects of college remediation on credit accumulation, persistence, and graduation are of great interest to college administrators, policymakers, and taxpayers. In this study, we add to the existing literature by exploring remediation in a new context, and estimate the impact of these programs on students with differing levels of incoming academic ability. We find that remediation differentially affects students with different types and levels of prior academic preparation. Math remediation appears to have negative effects over time, but the size of those effects differs according to the student’s level of previous academic preparation. Compared with students who are ready for college-level math, we find that students taking Developmental Algebra II accumulate fewer total college-level credits than similar peers. By the end of the third year, students placed into Developmental Algebra II course have taken 11.5 fewer total credits than their peers just above the margin for placement into college-level math, which is equivalent to a semester of coursework. We also detect some observable differences in college persistence to the second year, although our estimates vary based on the type of institution a student attends (2- vs. 4-year institutions). In contrast, students who were placed three levels below college-level math in Remedial Arithmetic did not experience large negative effects when compared with their peers placed into Developmental Algebra I (two levels below college math). When comparing the outcomes of students assigned to Remedial Arithmetic with those of students assigned to Developmental Algebra I, we found the estimates to be smaller in magnitude and not statistically significant.
A similar pattern exists in reading, with students on the margin of needing a Developmental Reading course accumulating fewer college-level credits over time than their peers who initially enrolled in college-level English. However, students enrolling in the lowest level of Remedial Reading, particularly in the community colleges, appear to actually persist in college from the first to the second year at higher rates than students assigned to Developmental Reading, one level below college-level English. In other words, students who were assigned two levels below college-level English were more likely to persist than similar students who were assigned only one level below the college thresholds and by a wide margin (17.7 percentage points).
In writing courses, we also found larger negative effects for students who were on the margin of needing one Remedial Writing course, but there was virtually no difference in outcomes for students in the lowest level of Remedial Writing (a course two levels below the college threshold). It may be that the skills obtained through remedial reading and writing courses are so fundamental to success in other courses that the acquisition of these skills results in improved academic performance and persistence in the short and long term. For students in need of a high level of remedial reading and writing, enrolling in these courses, could actually lead to greater success in other college courses beyond just composition, resulting in higher rates of persistence overall. This may be especially true for students attending community colleges, as these institutions may have less additional supports for students at all levels, making the attainment of skills in the remedial classroom all the more important. Prior research has found remedial and developmental courses to be more impactful in reading or writing (Bettinger & Long, 2009). We hypothesize that such skills in reading and writing are fundamental to all college work, whereas in contrast math is critical for certain majors, but many courses and majors do not require the use of math at all.
Our analysis suggests that the effects of remediation are more nuanced than those previously observed. Past evaluations of remediation programs have largely given us mixed, mostly negative estimates of the effects of developmental courses, and the focus on these negative results has dominated both scholarly and policy discussions. However, that research is limited to students requiring only one or two developmental education courses, and hence those LATE estimates do not give a complete view of the effects of remedial and developmental education policies. Our article attempts to provide a more complete picture of the impact of remediation, and how well colleges are addressing the needs of underprepared students at all levels of academic skill.
Our results are most comparable with those of Dadger (2012) and Hodara (2015). While Hodara’s study examines ESL courses as the lowest level English course, as opposed to our study of regular remedial reading and writing courses, she finds negative effects on persistence, transfer, and degree attainment for students enrolled in these lowest level courses. However, when examining her results by subgroup, she does observe some null effects at these lowest levels, warranting further research into these differences. Dadger (2012) similarly finds that students enrolling in the lowest level math courses may have been better off skipping this course. Her study focused on math exclusively and does not necessarily dispute our findings, as we also observe null findings at the lowest levels of math preparation. Alternatively, we find that the lowest scoring students may actually be helped by taking an additional remedial reading or writing course. While students at the margin of needing any remediation may experience mostly negative effects from having to take one course, the impact of such developmental and remedial courses on students with lower levels of preparation are sometimes beneficial, especially for students with strong gaps in reading and writing ability.
We also see evidence that many of the outcomes for the lowest scoring students are largely driven by the effects of remediation for students at the 2-year institutions. Although this study does not explicitly examine whether remediation is most effective at one type of institution versus another, we have several hypotheses that may explain why the effects of taking the lowest level remedial courses might be stronger at community colleges relative to 4-year institutions. First, because more underprepared students, especially those in need of multiple courses of developmental education, attend community colleges, these institutions may devote more of their resources to helping this type of student. If this is the case, then the positive effects we observed for students at the lower end of the academic distribution could be a result of this heighted attention at community colleges, as compared with 4-year institutions. Furthermore, some hypothesize that issues of stigma are also important in influencing the outcomes of students in developmental or remedial courses (Deil-Amen & Rosenbaum, 2002; Hadden, 2000), but stigma may be less of an issue for these severely underprepared students at a community college in comparison with the 4-year institution given the larger number of peers with similar levels of preparedness attending community colleges. If so, community college students with very low levels of preparation may not face negative perceptions (or “Scarlet Letter” effects) to the same degree as similar students at 4-year colleges.
However, some students at community colleges experienced negative effects from placement on longer term outcomes such as degree completion, especially those in upper levels of developmental education. This may relate to the fact that community colleges have lower levels of funding overall (Long, 2016). With fewer resources and supports, over the longer term, some students may struggle to complete college-level courses and their degrees. The negative finding for students on the margin of needing any remediation is also consistent with the literature and findings in other states (Dadger, 2012; Scott-Clayton & Rodriguez, 2015).
Our results present an interesting puzzle about why remedial and developmental courses have different effects by student ability. Understanding the reasons driving these differences could spur insight into how to make all developmental and remedial courses effective. It may also be the case that remediation is not required of as many students as is currently deemed necessary. Our research methodology is based on the fact that placement exams are noisy measures of students’ true ability, and our results, along with those from the literature, suggest that more attention must be paid to determining how to assess which students truly need remedial instruction before pursuing college-level work. Recent research supports the notion that many students are being placed into remedial courses using a single-placement test score, as is the case of Tennessee (Hughes & Scott-Clayton, 2010). Significant numbers of these students would likely succeed in college-level courses but are instead adding additional time and money to their path through college due to their requirement to enroll in remedial coursework instead (Melguizo, Bos, Ngo, Mills, & Prather, 2016; Scott-Clayton, Crosta, & Belfield, 2014). Incorporating high school transcripts into the placement policy helps significantly reduce under- and overplacement into remedial courses, given that transcripts include data collected over several years and a placement test is typically conducted in 30 to 60 minutes (Scott-Clayton et al., 2014).
These findings are particularly relevant for today, as the majority of states are redesigning their remedial courses with the hope of improving effectiveness. One implication of our study is that states and institutions should not treat remediation as a singular policy but instead should consider it as an intervention that might vary in its impact according to student needs. While there is a movement to mainstream students just below the cutoff for college-level courses, a different tactic may be warranted for students who are multiple levels below college-level material, as our results hint at the potential benefits of intensifying remedial coursework. Policymakers should consider revising remediation policies to better target the skills gaps of incoming students, which can be much more nuanced than they may initially appear. However, there are still many unanswered questions related to whether traditional remedial and developmental courses are beneficial even for students with lower levels of preparedness. While we present evidence on the potential benefits, these estimates must be compared with the costs incurred by students and institutions, and even with estimated positive effects, the cost–benefit analysis may suggest a net cost or that other approaches to addressing the challenges faced by underprepared students would be preferable.
As an increasing number of states consider moving remediation to a corequisite model and enrolling all students in college-level courses with some additional supports, the results from this research suggest that some caution should be exercised as our findings suggest that students in reading and writing classes at these levels may benefit from the opportunity to enroll in a remedial course prior to beginning college-level courses. The skills students learn in these basic reading and writing courses are critically important to subsequent academic success, regardless of course or major. While a student may not necessarily continue to draw upon the skills nurtured in a remedial math course, the ability to read and write is fundamental to success in nearly all college courses. Corequisite courses may be assuming that students in need of these most basic skills are able to pick them up along the way, when in reality students may need a more concentrated, concerted effort at developing these skills. Our results would suggest that a remedial reading or writing course may be one way to develop those skills, although there may be other avenues as well. That said, our results support the move to corequisite remediation for students on the margins of placement into college-level courses. More focused reform efforts may be a welcome solution to targeting remedial instruction to the varying academic needs of students as they embark on their postsecondary path.
Colleges and universities should also focus their efforts on helping students assigned to remedial courses to make continued progress toward their degrees. While taking upper-level remedial courses may not have large effects on short-term persistence, it does affect the number of college-level credits a student completes by the end of the third year. Credit accumulation may be a primary reason why students in need of remediation obtain degrees at lower rates than their peers. For this reason, it is important to consider ways in which students can complete their remedial requirements yet not be deterred from taking additional credit-bearing courses. It is also important to understand why some students pass their first college-level courses after taking remedial composition courses while others do not. Answers to these questions could help better identify strategies to improve remediation programs and the ongoing support made available to students throughout their college careers.
Footnotes
Appendix
Characteristics of Tennessee Public Colleges Compared With Regional and National Averages (2000 Cohort Unless Otherwise Noted)
| US (total) | SREB | Tennessee | |
|---|---|---|---|
| Median annual tuition and required fees for in-state students, 2001–2002 (4-year public) | $3,987 | $3,407 | $3,784 |
| Median annual tuition and required fees for in-state students, 2001–2002 (2-year public) | $1,743 | $1,420 | $1,623 |
| College enrollment rates of recent high school graduates (%) | 57.6 | 55.3 | 57.6 |
| Percentage of students in public colleges and universities | 76.7 | 83.0 | 76.7 |
| Percentage of undergraduates enrolled in 2-year colleges | 45.3 | 44.0 | 39.3 |
| Percentage of 2-year college students who are women | 57.0 | 58.6 | 59.2 |
| Percentage of undergraduates who are women | 56.1 | 57.1 | 57.1 |
| Percentage of undergraduates who are Black | 12.2 | 19.5 | 19.5 |
| Percentage of undergraduates who are White | 69.8 | 66.9 | 77.1 |
| Percentage of undergraduates who are Hispanic | 10.4 | 9.4 | 1.3 |
| Percentage increase in Pell Grants awarded, 1995–1996 to 2000–2001 | 36.8 | 53.2 | 52.7 |
| Percentage of students who are enrolled part-time (2-year colleges) | 62.7 | 59.6 | 55.2 |
| Percentage of students who are enrolled part-time (4-year colleges) | 27.5 | 27.1 | 20.4 |
| Graduation rate for 2-year colleges and universities, 1998 cohort (%) | NA | 17 | 11 |
| Graduation rate for 4-year colleges and universities, 1995 cohort | NA | 48 | 43 |
| First-time student receiving financial aid at public 2-year colleges (%) | 61 | 62 | 41 |
Source. 2003 SREB Fact Book archived through ERIC at http://files.eric.ed.gov/fulltext/ED478299.pdf
Note. SREB is the Southern Regional Education Board and represents the public colleges in 16 southern states (AL, AR, DE, FL, GA, KY, LA, MD, MS, NC, OK, SC, TN, TX, VA, WV). The national and SREB averages include Tennessee. ERIC = Education Resources Information Center.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was developed under grant R305A060010 from the Institute of Education Sciences, U.S. Department of Education. The views expressed in this article are those of the authors and do not necessarily reflect the views of the funder. This research was conducted in collaboration with the National Center for Postsecondary Research (NCPR).
Notes
Authors
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
