Aiming High and Falling Short

Abstract

The United States is in the midst of an effort to intensify middle school mathematics curricula by enrolling more 8th graders in Algebra. California is at the forefront of this effort, and in 2008, the state moved to make Algebra the accountability benchmark test for 8th-grade mathematics. This article takes advantage of this unevenly implemented policy to understand the effects of curricular intensification in middle school mathematics. Using district-level panel data from all California K–12 public school districts, we estimate the effects of increasing 8th-grade Algebra enrollment rates on a 10th-grade mathematics achievement measure. We find that enrolling more students in advanced courses has negative average effects on students’ achievement, driven by negative effects in large districts.

Keywords

curricular policy contextual effects mathematics education

U.S. students persistently trail their international peers in mathematics achievement, performing particularly poorly on the application of mathematical concepts (Gonzalez et al., 2008; Organization for Economic Co-Operation and Development, 2009; Schmidt, 2012). This skills gap impedes U.S. economic growth and competitiveness (Goldin & Katz, 2008). Over the last several decades, U.S. schools have dramatically intensified high school mathematics curricula in an attempt to improve U.S. mathematics achievement (Domina & Saldana, 2012). Central to this movement is the effort to enroll a greater proportion of students in Algebra while they are in middle school. Algebra serves a crucial gatekeeping function in U.S. schools. For students who fail to master Algebra in eighth or ninth grade, the path to advanced training in mathematics and, subsequently, many well-paid and high-status careers is blocked (Adelman, 1999; Attewell & Domina, 2008; Long, Conger, & Iatorola, 2012). Furthermore, it is a uniquely challenging course, drawing heavily upon the concrete procedural skills that students develop in elementary mathematics and requiring students to develop a new set of abstract reasoning skills (Carraher & Schliemann, 2007; Howe, 2005; Vogel, 2008).

In this article, we take advantage of an ambitious but unevenly implemented effort to enroll all California 8th graders in Algebra to examine the consequences of curricular intensification for student achievement. Using district-level panel data from all California public K–12 school districts, we estimate the effects of increasing 8th-grade Algebra enrollment rates on 10th graders performance on the mathematics portion of the California High School Exit Exam (CAHSEE). Our analyses make two major contributions to the fledging literature on curricular intensification in middle and high schools: First, we provide a system-level view of the consequences of changing course placement practices, rather than the student-level effects of advanced courses course placement. Second, we explore the effects of changing mathematics course-enrollment patterns for achievement in several mathematical content areas, including basic arithmetic, pre-algebraic functions, and algebra. Our findings are counter-intuitive, suggesting that enrolling more students in advanced math courses has negative consequences for mathematics achievement.

Context: Eighth-Grade Algebra-for-All in California

Over the past three decades, California has been at the forefront of the national effort to universalize eighth-grade Algebra (Loveless, 2008). In 1987, California’s State Superintendent of Public Instruction argued that detracking middle schools was a central step toward raising academic standards in high schools. In 1992, the state department of education called for “heterogeneous grouping and detracking as a goal” and the 1997 revision of the state’s content standards called on middle schools to enroll all eighth graders in Algebra I. In 1999, the California State Senate passed the Public School Accountability Act (PSAA). By penalizing schools for enrolling eighth graders in pre-algebra or other general math courses, the law created powerful incentives for schools to place more eighth graders in Algebra (Domina, Penner, Penner, & Conley, 2014). The adoption of these standards spurred rapid intensification in middle school mathematics. Between 1999 and 2008, the proportion of California eighth graders enrolled in Algebra more than tripled, from 16% to 51% (Rosin, Barondess, & Leichty, 2009).

In 2008, the state’s Board of Education voted to make the Algebra California Standards Test (CST) the “sole test of record” for the state’s eighth graders. This vote required eighth graders to demonstrate proficiency on the state’s end-of-course Algebra standards exam to satisfy accountability expectations under the No Child Left Behind Act and California’s Public Schools Accountability Act (Rosin et al., 2009). However, California’s eighth-grade Algebra mandate was never fully implemented. Responding to a challenge from California school administrators and school boards, the courts postponed the policy’s implementation in the spring of 2010. Later that year, under pressure from the Obama administration and teachers unions, the California Academic Content Standards Commission adopted the Common Core State Standards, which recommend pre-Algebra content for eighth graders. Although California’s revised middle school math standards continue to encourage schools to enroll eighth graders in Algebra, state policy no longer mandates accelerated Algebra (Wurman & Evers, 2011).

California’s decades-long effort to universalize eighth-grade Algebra created a fluid policy environment. Over the last several years, districts across the state have moved to enroll more students in early Algebra. Many districts anticipated the eighth-grade Algebra mandate and began to track greater proportions of eighth graders into Algebra courses long before the state attempted to mandate the course. Many district officials supported the mandate, viewing it as a tool to mitigate inequalities in the opportunity to learn. Other district officials, however, remained committed to the idea of sorting students into middle school mathematics courses based on their prior achievement, and were reluctant to increase eighth-grade Algebra enrollments even after the state created incentives to do so. Furthermore, after the state stepped away from its eighth-grade Algebra mandate, eighth-grade Algebra enrollment rates began to decline in some—but not all—California districts. As a result, patterns of eighth-grade Algebra enrollment rates vary considerably across districts. This variation makes it possible to estimate the effects of increasing eighth-grade Algebra on student mathematics achievement.

Curricular Intensification and Its Consequences

The American curricular intensification movement is predicated on the notion that students learn more in academically challenging educational environments; a theory that is often referred to as “opportunity to learn” (Porter, 2002). Consistent with this theory, several studies indicate that students who are exposed to rigorous curriculum and instruction experience greater achievement gains on average than those who are not (e.g., Argys, Rees, & Brewer, 1996; Attewell & Domina, 2008; Domina, 2014; Gamoran & Hannigan, 2000; Gamoran, Porter, Smithson, & White, 1997; Long et al., 2012; Schmidt et al., 2001). These studies employ a wide range of methods to isolate the effect of rigorous course enrollment for students’ later outcomes. Taken together, their results suggest that efforts to enroll more students in accelerated Algebra courses should boost student achievement by influencing student exposure to rigorous academic content, effective instructional strategies, and high-achieving peers.

However, this literature is limited in two important regards: First, efforts to estimate the effects of curricular intensification using observational data are subject to considerable selection bias, because students who enroll in advanced courses differ from students who do not on a wide range of characteristics. Relatively few studies have attempted to estimate the effects of advanced course-taking in experimental or rigorous quasi-experimental settings, and those that do have returned sharply mixed results. Heppen et al. (2012) report the results of an experiment in which high-achieving eighth graders in 68 randomly selected small, rural middle schools were offered access to an online Algebra course. In this case, access to online Algebra had a moderate positive effect on these high-achieving students’ Algebra achievement as measured at the end of eighth grade (effect size = 0.39), as well as their subsequent high school math course-taking. However, instrumental variable analyses taking advantage of rapid curricular intensification in 10 North Carolina school districts indicate that accelerated Algebra has a negative effect on student achievement (Clotfelter, Ladd, & Vigdor, 2012a, 2012b).

Second, policy efforts like California’s eighth-grade Algebra push do more than change a handful of students’ course-enrollment patterns. Rather, they aim to make broad systematic changes in school curricula and organization. These systematic changes may have spillover effects for students who enroll in eighth-grade Algebra as well as their peers who enroll in less advanced mathematics courses, particularly when schools transition from highly differentiated systems of mathematics instruction to relatively untracked Algebra-for-all systems. As a result, studies that identify the effects of eighth-grade Algebra enrollment for a given student may provide limited insight into the effects of an effort to accelerate Algebra enrollment for a large proportion of students. By reducing low-level courses and integrating students who once would have taken these courses into more advanced classrooms, these policies likely have a wide range of intended and unintended consequences on the content, pedagogy, and social organization of secondary schools (Gamoran & Hannigan, 2000). Policies that increase the number of students in advanced courses increase the demand for teachers in these courses, often leading schools to assign new teachers or teachers who had previously specialized in teaching lower level courses to advanced courses (Clotfelter et al., 2012a). Furthermore, large-scale curricular intensification likely changes the distribution of student skills within advanced courses and less advanced courses alike. Increasing the proportion of students enrolled in eighth-grade Algebra courses may lead to increases in the degree of skill heterogeneity and lower mean prior achievement scores in Algebra courses by adding more low-achieving students to these courses. At the same time, this policy shift likely decreases skill heterogeneity and lowers mean prior achievement scores in pre-Algebra courses, as the highest skill students from pre-Algebra are promoted to Algebra, leaving behind only the students deemed most underprepared for Algebra in eighth grade. These changes in classroom composition may have independent consequences on student learning (Nomi, 2012; Zimmer & Toma, 2000), as well as effects on teacher instructional content and methods (McPartland & Schneider, 1996). The net effects of curricular change thus include direct course-enrollment effects for students who enroll in different courses under a given placement regime than they would have otherwise, as well as spillover effects for students whose course enrollments may not be affected by the shift but whose learning environments are.

Perhaps due to these spillover effects, evaluations of broad-based curricular intensification efforts return fewer positive results than analyses of the student-level effects of advanced course enrollment (Stein, Kaufman, Sherman, & Hillen, 2011). Allensworth, Nomi, Montgomery, and Lee (2009) find no evidence to suggest that a Chicago Public Schools effort to enroll all ninth graders in Algebra I and college prep English improved student achievement, graduation rates, or college-going. Although difference-in-difference analyses suggest that the “double-dose” Algebra curriculum that Chicago implemented as a part of this effort was effective for low-achieving students (Nomi & Allensworth, 2009), Nomi (2012) finds that curricular intensification in Chicago had unintended negative effects for high-achieving students.

Furthermore, preliminary evidence from California is similarly discouraging. Liang, Heckman, and Abedi (2012) provide a descriptive analysis of statewide student-level data indicating that approximately 60% of students who take Algebra in the eighth grade fail to score proficient on the end-of-course Algebra CST. Furthermore, they demonstrate that students who fail eighth-grade Algebra and thus take the Algebra CST again at the end of their ninth-grade year score lower on average than students who take the Algebra CST for the first time at the end of ninth grade. While this analysis does not control for differences between these two groups of students and the schools they attend, it indicates that accelerated Algebra may not boost achievement among California’s students. Similarly, case study data point to declines in student mathematics achievement in one California district that dramatically increased eighth-grade Algebra enrollment rates (Domina et al., 2014).

Evaluating California’s Eighth-Grade Algebra-for-All Effort

This article provides a uniquely rigorous evaluation of the net effects of changing middle school mathematics placement policies. We use panel data from California public school districts between 2003–2004 and 2009–2010 to estimate the effects of increasing the percent of students enrolled in eighth-grade Algebra on students’ mathematics achievement, as measured by the CAHSEE. Like students in middle and junior high schools throughout the United States, eighth graders in most California middle schools have the option of enrolling in one of several tiered mathematics courses, including remedial mathematics, general mathematics or pre-Algebra, Algebra, and, for a handful of particularly advanced students, Geometry or higher level mathematics. As secondary math courses are nearly universally sequenced in American secondary schools, eighth-grade course enrollments largely determine students’ chances of enrolling in more advanced courses throughout high school. In particular, students must take Algebra in eighth grade to take Calculus before they graduate from high school.

Our analyses draw upon district-level data collected from the California Basic Educational Data System (CBEDS) and the CAHSEE, describing middle school math course-enrollment patterns and subsequent mathematics achievement for students enrolled in all California public school districts that serve students from middle school to high school.¹ Our analyses use the district as the unit of analysis for two reasons: First, as California does not have a statewide student-level data system, student- or even school-level data that include middle school mathematics course enrollments and 10th-grade test scores are unavailable. Second, districts play a crucial role in determining math course placement practices in California, accounting for more than half of the variation in 8th-grade Algebra enrollment rates among traditional middle schools. Table 1 reports descriptive data for 8th graders enrolled in California unified school districts in each cohort between 2003–2004 and 2009–2010. In each of these years, the 222 school districts that are at the focus of this study enrolled approximately 300,000 8th graders. In 2003–2004, approximately 40% of these students enrolled in Algebra or a more advanced math course during their 8th-grade year. By 2009–2010, that percentage was more than 60%. This growth in 8th-grade Algebra enrollment rates is equivalent to approximately a standard deviation in the pooled unweighted distribution of district-level 8th-grade Algebra enrollment. By comparison, National Assessment of Educational Progress (NAEP) data indicate that 8th-grade Algebra enrollment rates among public school students nationwide increased from 24% to 35% between 2000 and 2010.

Table 1

Descriptive Statistics, Eighth Graders Enrolled in California Unified Public School Districts, 2003–2004 to 2009–2010

	2003–2004	2004–2005	2005–2006	2006–2007	2007–2008	2008–2009	2009–2010
Balanced panel
% in Algebra or higher	0.38	0.43	0.48	0.53	0.54	0.59	0.63
% in Algebra or higher (std)	0.01	0.22	0.43	0.64	0.70	0.93	1.07
% Black/Hispanic	0.50	0.51	0.51	0.52	0.53	0.54	0.54
% ELL	0.27	0.26	0.20	0.21	0.21	0.20	0.20
District API	702.0	708.8	725.5	735.8	741.1	752.7	764.9
n (districts)	222	222	222	222	222	222	222
Weighted n	301,892	300,552	292,629	292,185	291,754	286,736	279,175
All districts
% in Algebra or higher	0.38	0.42	0.46	0.51	0.54	0.58	0.60
% in Algebra or higher (std)	0.00	0.17	0.37	0.56	0.70	0.90	0.97
% Black/Hispanic	0.49	0.50	0.52	0.53	0.53	0.53	0.54
% ELL	0.27	0.25	0.21	0.21	0.22	0.20	0.20
District API	700.1	706.9	718.9	732.2	737.3	748.1	760.8
n (districts)	282	285	293	288	296	309	300
Weighted n	336,084	336,010	341,332	336,131	333,089	337,277	317,902

Note. Std = standardized; ELL = English-language learner; API = Academic Performance Index.

The analyses that follow utilize a balanced panel that excludes approximately 100 districts that do not report data in at least one study year. As these excluded districts are relatively small, our balanced panel accounts for more than 85% of California eighth graders in unified school districts in any given year. Furthermore, as the descriptive data for all California districts reported in Table 1 indicate, the differences between districts that provide balanced data and those that do not are not pronounced.

Predictors of Eighth-Grade Algebra Enrollment Rates in California K–12 School Districts

Prior to estimating the achievement effects of increasing eighth-grade Algebra enrollment rates, we provide a descriptive look at district-level middle school math course placement trends. We first do so by reporting trends in eighth-grade Algebra enrollment rates across the state and within the state’s 12 largest school districts.

We then explore district variation in eighth-grade Algebra enrollment trends by providing an enrollment-weighted unconditional correlation matrix. The focal variable in this correlation matrix represents the proportion of eighth graders in a given district (d) who completed the end-of-course CST in Algebra I or higher (e.g., Geometry or Algebra II) in each year (t) between 2004 and 2010. We use CST course completion as a proxy for students’ eighth-grade mathematics course completion. California students who take basic skills or pre-Algebra courses sit for the General Math CST in the spring of their eighth-grade year, while students who take Algebra courses sit for the Algebra CST and students who take Geometry sit for the Geometry CST. Nearly all students—including special education students and English-language learners (ELL)—take these tests under California and federal accountability policy.² The correlation matrix reports the extent to which these district-level eighth-grade enrollment rates vary over time and with district demographics, including variables representing the proportion of eighth graders who are Black or Hispanic³; the proportion of eighth graders who are classified as ELL; and the natural log of district eighth-grade enrollments. In addition, we correlate eighth-grade Algebra enrollment rates against a lagged measure of the districts’ Academic Performance Index (API) scores and a lagged measure of district eighth-grade Algebra enrollment rates. The API is a composite measure of student achievement that is central to California’s school accountability system. The composite reports the weighted average of students’ Math, English, Science, and History CST scores within a district (or school), with Math and English accounting for 85% of the score. Under California accountability policy, eighth-grade algebra enrollment rates are an input in the algorithm that generates districts’ API scores. Districts receive API scores for test performance from time t in time t + 1. The correlation between a school’s API in time t − 1, and eighth-grade algebra enrollment rates in time t partially captures the extent to which districts respond to accountability pressures by increasing eighth-grade Algebra rates. Finally, the bivariate correlation between contemporary and lagged eighth-grade Algebra enrollment rates provides a sense of the extent to which middle school mathematics placement practices are path dependent at the district level.

Panel Analysis: California K–12 School Districts, 2004–2010 Eighth Graders

Following an examination of the patterns of Algebra enrollment across the timespan of our panel, we next estimate district fixed effects models to investigate the effects of changing eighth-grade Algebra enrollment rates in California public school districts on student achievement. The most basic of these analyses takes the following form:

Y_{d, t + 2} = β_{0} + β_{1} {(% Alg)}_{d, t} + β_{2} {(X)}_{d, t} + β_{3} {(API)}_{d, t - 1} + α_{d} + α_{t} + ε_{d, t} .

In this model, the dependent variable, $Y_{d, t + 2}$ , is a district-level measure of student achievement on the mathematics portion of the spring 10th-grade CAHSEE. This exam, which is designed under contract by Educational Testing Service (ETS), addresses content covered California’s standards for 6th- and 7th-grade mathematics and Algebra courses. It consists of 80 scored multiple-choice questions (and an additional 12 unscored questions being tested for future use) covering Algebra, measurement and geometry, statistics, data analysis and probability, number sense, and mathematical reasoning.

The CAHSEE mathematics exam is particularly useful for this analysis as it is the only mathematics exam that nearly all California public school students take at the same time during the middle and high school years. State law requires all California public school students take this exam for the first time in the spring of their 10th-grade year. Students must pass this exam (as well as a parallel exam in English-language arts) to earn a high school diploma.⁴ Our analyses use CAHSEE data from 2 years after students in the panel completed 8th grade (March 2006 to March 2012 CAHSEE administration). By contrast, neither mathematics CSTs nor college admissions tests provide appropriately representative samples for our purposes. Fewer than half of California high school students take the SAT in a given year, and a smaller proportion takes the ACT. Even though nearly every student in California takes the Algebra CST at some point, the timing of when a student takes the exam varies from 8th to 11th grade depending on when he or she enrolled in Algebra. The same is true for higher level mathematics, although the share of students who complete higher level math courses also decreases as the course gets more difficult.

That said, the CAHSEE mathematics exam is not without its limitations. As the exam is designed to ensure students have a minimal level of mathematics competency, it gives considerable weight to pre-Algebra mathematics topics and may not accurately capture achievement for students at the top of the skills distribution. Approximately 8% of California 10th graders score the highest possible score on this exam. Although this proportion has not changed appreciably over the study period, these ceiling effects may negatively bias our findings. Furthermore, if changing enrollments in 8th-grade Algebra courses primarily influences student achievement in Algebra and more advanced mathematics topics, one might expect overall CAHSEE mathematics exam scores to understate the effects of changing 8th-grade Algebra placements.

We address these measurement concerns in two ways: First, we have compiled evidence regarding the CAHSEE’s validity from a variety of sources in a report, available by request. Our own analyses as well as previously published reports point to a positive .70 correlation between student CAHSEE scores and mathematics CST scores that does not vary considerably over time. Notably, this correlation is particularly robust for 10th graders enrolled in relatively advanced mathematics courses. For example, the correlation between CAHSEE scores and Summative Mathematics CSTs is .74 in the 2011–2012. Furthermore, CAHSEE scores correlate closely with students’ post-secondary enrollment outcomes (Human Resource Research Organization, 2012). Second, we conduct supplementary analyses of five CAHSEE subscales: probability and statistics, number sense, Algebra and functions, measurement and geometry, and Algebra I. Each of these subscales uses data from 12 to 17 test items that align closely with state content standards (Becker, Watters, & Sacramento, 2008; Becker, Wise, Mardoin, & Watters, 2012; Becker, Wise, & Watters, 2010; Wise et al., 2006; Wise et al., 2004). These analyses make it possible to investigate heterogeneity in the effects of curricular intensification across different mathematical domains. The results of these analyses provide important insights into the CAHSEE’s validity as a measure of mathematics achievement in this setting. If curricular intensification has different effects on high-level skills than low-level skills, the relatively weak coverage of high-level mathematics in the CAHSEE may lead to important biases. However, evidence of consistent effects of curricular intensification across CAHSEE subscales may serve to mitigate these measurement concerns.

The model includes district fixed effects, $α_{d}$ , to account for unobserved time-invariant differences between districts, and cohort fixed effects, $α_{t}$ , to account for common trends across years. The fixed effects remove potential time-invariant changes between-district differences as well as time-variant changes common to all districts that are related to both districts’ Algebra enrollment rates and CAHSEE scores. We also include a vector of time-variant district characteristics to control for potential observable differences among districts, $X_{d, t}$ These include the proportion of eighth graders who are Black or Hispanic, the share of ELL, the natural log of district enrollment, and lagged API scores.⁵

The key predictor variable in this analysis is the percent of district eighth graders who enroll in eighth-grade Algebra or higher. To ease interpretation, we standardize this measure on the 2004–2005 distribution, so that zero is equal to the 2004–2005 mean and −1 and 1 are equivalent to 1 standard deviation above and below that mean. Assuming that district CAHSEE test score averages change over time at a common rate after controlling for observed time-varying district characteristics, then $β_{1}$ in Equation 1 can be interpreted as an unbiased estimate of the change in district-level eighth-grade Algebra enrollment rates on CAHSEE test scores. If, however, test score averages follow district-specific trends—and particularly if these district-specific trends correlate with eighth-grade Algebra course placement trends—Equation 1 may provide biased estimates of the effects of increasing eighth-grade Algebra enrollments. To address this concern, we additionally estimate a random growth model (Papke, 1994; Wooldridge, 2002; Zimmer & Buddin, 2006).

This model estimates district-specific intercepts ( $α_{d}$ ) and district-specific time trends ( $α_{d} \times τ$ ), and can be represented as,

Y_{d, t + 2} = β_{1} {(% Alg)}_{d, t} + β_{2} {(X)}_{d, t} + α_{d} + α_{t} + α_{d} \times τ + ε_{d, t} .

This random growth model controls for both time-invariant between-district differences, as well as differences in districts’ average growth rate.⁶ It should also be noted that the relationship between the independent variables and the dependent variable in a random growth model is only identified off non-linear changes over time. The time fixed effects in this model account for year-to-year secular changes in achievement that deviate from the common linear in achievement. Equation 2 yields unbiased estimates of the effect of eighth-grade Algebra enrollment on student achievement if changes in algebra enrollment are exogenous to the independent variables as well as unmeasured district characteristics that are time-invariant or linearly time-varying.

These analyses estimate a highly policy-relevant parameter that has largely been neglected elsewhere in the literature on accelerated Algebra and other forms of curricular intensification: The mean achievement effects of enrolling more students in advanced math courses. This parameter is the net effect of curricular intensification. In contrast to studies that estimate the effects of advanced course enrollment only for students at risk of moving into advanced courses, our analyses capture the important ways in which enrolling more students in advanced courses not only alters mathematics course experiences for students who change courses but also alters experiences for their peers who are left in low-level courses, and their peers who would have taken high-level courses even prior to the change in placement practices.

However, neither Equation 1 nor Equation 2 addresses the potentially confounding consequences of short-term changes in district organization or management. If such changes systematically precede changes in middle school mathematics placement practices, estimates of the effects of eighth-grade Algebra placement may be biased. One particularly troubling potential confounder is administrative turnover: The arrival of new administrative leadership might cause both eighth-grade Algebra enrollment rates and later student achievement to shift. As we do not have access to statewide panel data on district leadership, we are unable to evaluate this possibility. However, discussions with district administrators suggest that administrative turnover often occurs after districts intensify middle school mathematics curricula, rather than before.

All multivariate analyses are weighted by the mean of each district’s eighth-grade enrollment across the panel. In addition, we estimate a series of supplementary models in which we investigate the extent to which the effects of curricular intensification vary by district size. Prior to the eighth-grade Algebra-for-all effort, relatively large districts may have provided more highly differentiated mathematics instruction than low-enrollment districts (where there were few students to split between tracked mathematics course sequences). These supplementary models thus consider the extent to which the effects of increasing eighth-grade Algebra result from changes to the content, instruction, and peer composition of mathematics courses in complex and differentiated educational systems. We use district-level cluster-robust standard errors estimation throughout to address potential heteroskedasticity and serial correlation among observations.

Changes in Eighth-Grade Algebra Enrollment Rates

California districts dramatically intensified middle school mathematics curricula over the last decade. This increase in eighth-grade Algebra enrollment rates clearly predates the State Board of Education’s attempt to make eighth-grade Algebra the mathematics “course of record” for accountability purposes. However, eighth-grade Algebra enrollment rates jumped in the year immediately after the state announced this policy shift, increasing from 54% in 2007–2008 to 60% in 2008–2009. Statewide eighth-grade Algebra enrollment rates continued to rise after this announcement, despite legal efforts to overturn the eighth-grade Algebra mandate.

California’s eighth-grade student body has remained relatively demographically stable during this period of rapid curricular change. Free and reduced lunch enrollment rates vary between 51% and 56% during this time period, fluctuating gradually with broader shifts in economic conditions. The racial and ethnic composition of California eighth graders also changed little during this time period, with approximately 54% of eighth graders identifying as Black, Hispanic, Native American, or Pacific Islander.⁷ The most striking demographic shift apparent in this table concerns the proportion of ELL in California schools. In 2003–2004, 27% of California eighth graders were classified as ELL; by 2009–2010, that number had dropped to 20%. More than three fourths of these students are native Spanish speakers. This decline in ELL enrollment seems to be largely a function of changing practices for reclassifying non-native speakers as English-language proficient.

While Table 1 suggests that accountability pressures from the State Board of Education led districts and schools across the state to enroll more students in eighth-grade Algebra, Figure 1 indicates that districts across the state took very different paths toward intensifying middle school mathematics curricula. This figure presents line graphs representing eighth-grade Algebra enrollment rate trends between 2003–2004 and 2009–2010 for the 12 largest California unified school districts. Just one of these districts, Anaheim Union, seems to have responded directly to the state’s Algebra accountability mandate, nearly universalizing eighth-grade Algebra by doubling the proportion of eighth graders placed into Algebra or more advanced math courses in 2009–2010. Other districts, including Corona-Norco, Garden Grove, and Los Angeles, seem to have responded to early signals from the state, increasing eighth-grade Algebra enrollment rates by more than 20 percentage points in the years leading up to the passage of the eighth-grade Algebra-for-all mandate. Several large California public school districts acted much more gradually to increase eighth-grade Algebra enrollment rates. Capistrano is typical of this approach, gradually increasing eighth-grade Algebra enrollment rates from approximately 20% to 40% over the study period. Middle school math enrollment trends follow a somewhat idiosyncratic pattern in the state’s largest public school district, Los Angeles Unified, where eighth-grade Algebra enrollment rates spiked at 67% in 2005 before decreasing to 49% in 2007 and increasing again to 60% by 2010. The degree of between-district heterogeneity is even more pronounced among smaller districts.

Figure 1.

Percent of eighth graders completed Algebra, Geometry, or Algebra II CST in the 12 largest California unified public school districts, 2004–2010

Table 2 provides a more systematic look at eighth-grade Algebra enrollment rates in California school districts. These population-weighted unconditional correlations provide another way of understanding the secular increase in eighth-grade Algebra enrollments, indicating that there is a positive .23 linear correlation between year and eighth-grade Algebra enrollment rates. (Supplementary multivariate models, which are available by request, clearly indicate that this secular trend is significant even after controlling for demographic changes.) However, neither the size nor the ethnic or language composition of California public school districts is associated with eighth-grade Algebra enrollment. Similarly, although relatively high-performing districts tend to have higher rates of eighth-grade Algebra enrollment than lower performing districts, this correlation is weak. Indeed, the single district-level factor that is closely correlated with eighth-grade Algebra enrollment rates is the lagged eighth-grade Algebra enrollment rate; a finding that indicates that patterns in eighth-grade Algebra enrollment are highly path dependent.

Table 2

Unconditional Correlation Matrix, Predictors of Eighth-Grade Algebra Enrollment Rate, California Public School Districts 2003–2004 to 2009–2010

	Eighth-grade Algebra (%)	Year	% Black/Hispanic	% ELL	Eighth-grade enrollment (ln)	District API (lag)	Eighth-grade Algebra (lag)
Eighth-grade Algebra (%)	1.00
Year	.23	1.00
% Black/Hispanic	.06	.03	1.00
% ELL	.03	−.22	.74	1.00
Eighth-grade enrollment (ln)	.03	−.02	.45	.38	1.00
District API (lag)	.09	.29	−.84	−.73	−.38	1.00
Eighth-grade Algebra (lag)	.80	.25	.06	−.01	.05	.11	1.00

Note. API = Academic Performance Index; ELL = English-language learner.

Effects of Increasing Eighth-Grade Algebra Enrollment on Student Achievement

The analyses reported in Table 3 take advantage of these uneven patterns in district 8th-grade Algebra enrollment rates to estimate the effects of changes in the proportion of students enrolled in 8th-grade Algebra or higher on mathematics achievement. The first model in Table 3 does so using a district fixed effects approach. This model considers the relationship between district 8th-grade Algebra enrollment rates in a given year and mean CAHSEE math scores for 10th graders in the district 2 years later. The negative and statistically significant “% eighth graders ≥ Algebra” coefficient in this model indicates that efforts to enroll more middle school students in advanced mathematics courses have unintended negative consequences for student mathematics achievement. This model indicates that a 1 standard deviation increase from mean 2004 to 2005 8th-grade Algebra enrollment rates (such as might occur if a district were to increase 8th-grade Algebra enrollment from approximately 38% to 60%) decreases mean student CAHSEE math scores by approximately 0.07 standard deviations. By way of comparison, this estimated negative effect is approximately the same size as the average positive achievement effects associated with the federal No Child Left Behind Act (Dee & Jacob, 2011) or about 15% of the Black–White achievement gap in mathematics (Reardon, 2008).

Table 3

Fixed Effects and Random Growth Model Coefficients, Predictors of District Mean CAHSEE Math Test Scores, California Public School Districts 2003–2004 to 2009–2010 (Balanced Panel)

	Fixed effects	Random growth
% eighth graders ≥ Algebra (std)	−0.07** (0.02)	−0.05* (0.02)
2004 (eighth)	−0.28** (0.10)	—
2005 (eighth)	−0.14^† (0.08)	0.06 (0.04)
2006 (eighth)	−0.10^† (0.06)	0.08 (0.05)
2007 (eighth)	−0.13** (0.04)	−0.01 (0.04)
2008 (eighth)	−0.02 (0.03)	0.01 (0.02)
2009 (eighth)	—	—
2010 (eighth)	0.00 (0.03)	−0.05^† (0.03)
% Black/Hispanic	−1.42 (1.04)	−0.54 (1.00)
% ELL	0.18 (0.36)	0.2 (0.51)
Ln(eighth-grade enrollment)	0.05 (0.28)	−0.13 (0.22)
Lagged district API	0.47*** (0.11)	0.03 (0.10)
Constant	0.82 (2.28)	0.12*** (0.02)

Adjusted R²	.40	.04
No. of districts	220	213
ρ	.84	.33

Note. CAHSEE = California High School Exit Exam; std = standardized; ELL = English-language learner; API = Academic Performance Index.

†

p < .10. *p < .05. **p < .01. ***p < .001.

The internal validity of the district fixed effects identification strategy hinges on the assumption that the only time-varying within-district characteristics that systematically covary with both eighth-grade Algebra placement patterns and CAHSEE math scores are captured by our controls for observable demographic characteristics and time fixed effects accounting for statewide year-to-year changes. We relax this assumption by estimating a random growth model that, in addition to the district fixed effects, allows for different district-specific linear time trends. This model accounts for district fixed effects, district-specific time trends, and time fixed effects, accounting for between-district variation in linear CAHSEE score trends. This random growth model returns coefficients similar in magnitude and direction as the main district fixed effects model. However, as these models only use the non-linear variation in the independent variables to identify causal effects, they are somewhat less precise.

The analyses reported in Table 4 test the appropriateness of these fixed effects and random growth models through a series of placebo tests designed to examine the possibility that unobserved changes coinciding with increases in Algebra enrollment confound our findings. We regress demographic variables, standardized English-language arts (ELA) CAHSEE scores, and leads and lags of standardized math and ELA CAHSEE scores on the percent of students enrolled algebra or higher—the main coefficient of interest—and the other control variables. If the coefficient for algebra enrollment is statistically significant in these models, this suggests that our results in Table 3 may be confounded by omitted variables not captured by our analytic strategy.

Table 4

Misattribution Checks

Dependent variables	Fixed effects model	Random growth model
% Black/Hispanic	0.00 (0.00)	0.00 (0.00)
% ELL	0.02^† (0.01)	0.01* (0.00)
% FRPL	0.00 (0.01)	0.01 (0.01)
Grade 8 enrollment (ln)	−0.01^† (0.01)	0.000 (0.00)
Lagged API	0.03* (0.01)	0.01* (0.01)
ELA CAHSEE	−0.05* (0.02)	−0.01 (0.02)
Twice lag CAHSEE math	−0.05^† (0.029)	0.020 (0.034)
Twice lag CAHSEE ELA	−0.07* (0.030)	0.020 (0.030)
One lag CAHSEE math	−0.05* (0.021)	0.020 (0.026)
One lag CAHSEE ELA	−0.06** (0.022)	0.000 (0.026)
One lead CAHSEE math	−0.030 (0.021)	0.020 (0.015)
One lead CAHSEE ELA	−0.04^† (0.019)	0.000 (0.015)
Two leads CAHSEE math	−0.020 (0.021)	−0.010 (0.021)
Two leads CAHSEE ELA	−0.020 (0.020)	0.000 (0.022)

Note. Fixed effects and random growth model coefficients for the percent of students enrolled in at least Algebra predicting eighth grade percent minority, logged enrollment, and contemporaneous mean CAHSEE scores, California Public School Districts 2003–2004 to 2009–2010 (balanced panel). All models include the cohort controls as well as demographic, enrollment, and API controls used in Table 3, with the exception of the independent variable when it is used as the dependent variable (e.g., % minority). ELL = English-language learner; FRPL = free or reduced-price lunch; API = Academic Performance Index; ELA = English-language arts; CAHSEE = California High School Exit Exam.

†

p < .10. *p < .05. **p < .01. ***p < .001.

The first column of Table 4 reports the results of analyses using the district fixed effects estimation strategy and the second column reports the results of analyses using the random growth modeling strategy. The first five rows assesses whether the percent of students in at least algebra is significantly related with districts’ demographics and API scores. In most cases, the fixed effect and random growth model return marginally statistically insignificant results, with the exception of the percent of ELL, Grade 8 enrollment, and lagged API scores. However, these point estimates are quite small, especially in models that use the random growth specification. We next evaluate whether districts’ ELA CAHSEE scores, and leads and lags of ELA and math CAHSEE scores, are associated with districts’ algebra enrollment rates. It does appear that in the fixed effects specification, the percent of students enrolled in at least algebra is associated with ELA CAHSEE scores and some leads and lags of math and ELA CAHSEE scores. This result calls into question the internal validity of Table 3’s fixed effects specification. However, the random growth model specification does not return any significant relationships between the percent of students enrolled in at least algebra and ELA CAHSEE scores and the leads and lags of math and ELA CAHSEE scores. This indicates that there is a potentially linear time-varying confounder that is not controlled for in the fixed effects specification, but that is accounted for in the random growth model specification. This analysis highlights the random growth model’s importance for generating unbiased estimates of the achievement effects of eighth-grade Algebra enrollment rates.

Although administered to students in the spring of their 10th-grade year, the CAHSEE is a criterion-referenced test designed to measure students’ mastery of basic Algebra and pre-Algebra mathematics content. As this test is not designed to capture student proficiency with more advanced mathematics concepts, it is possible that the analyses reported in the first model of Table 3 may provide negatively biased estimates of the relationship between 8th-grade Algebra enrollment rates and mathematics student learning. For example, redirecting students from grade-level 8th-grade mathematics courses to 8th-grade Algebra may have negative effects on their mastery of basic arithmetic and pre-algebraic mathematic content even as it boosts their algebraic understanding. In such a scenario, the summary CAHSEE test score might exaggerate the negative effects associated with increasing 8th-grade Algebra enrollment.

The models reported in Table 5 use detailed CAHSEE subscale score results to consider the extent of this potential bias. These subscales measure the percent of CAHSEE test score items students answered correctly in several distinct mathematics domains ranging from the relatively simple (Number Sense) to the more advanced (Measurement and Geometry; Algebra I). These analyses indicate that the effects of increasing eighth-grade Algebra enrollment rates are consistently negative across the five domains. For example, a 1 standard deviation increase in district eighth-grade Algebra enrollment rates is associated with a 0.05 standard deviation decline in the percentage of Number Sense questions students answer correctly. This result is not significantly different from the significant 0.04 standard deviation decline in Algebra I achievement. Similarly, a 1 standard deviation increase in eighth-grade Algebra enrollment rates is associated with a 0.04 standard deviation decline in Measurement and Geometry achievement. These results are quite similar across the fixed effects and random growth model specifications. The results reported in Table 5 thus not only provide some reassurance that the CAHSEE test adequately captures student mathematics achievement at least through basic Algebra and geometry but also suggest that the declines in CAHSEE scores resulting from an increase in eighth-grade Algebra enrollment are present and similar in magnitude in all major content areas assessed by the CAHSEE. These findings suggest that enrolling more students in eighth-grade Algebra does not just undermine student achievement in pre-algebraic mathematic content, but also undermines achievement on a relatively wide range of mathematics areas. Although it remains possible that increases in eighth-grade Algebra enrollment rates could lead to downstream improvements in higher level mathematics, these findings do not provide support for this argument.

Table 5

Fixed Effects and Random Growth Model Coefficients, Predictors of District Mean Percent Correct CAHSEE Math Test Subscales, California Public School Districts 2003–2004 to 2009–2010 (Balanced Panel)

	Panel A: Fixed effects models					Panel B: Random growth models
	(1) Probability and statistics % correct (std)	(2) Number sense % correct (std)	(3) Algebra and functions % correct (std)	(4) Geometry % correct (std)	(5) Algebra I % correct (std)	(1) Probability and statistics % correct (std)	(2) Number sense % correct (std)	(3) Algebra and functions % correct (std)	(4) Geometry % correct (std)	(5) Algebra I % correct (std)
% eighth graders ≥ Algebra (std)	−0.05*** (0.01)	−0.05*** (0.01)	−0.04** (0.01)	−0.04** (0.02)	−0.04* (0.02)	−0.05** (0.02)	−0.05** (0.02)	−0.04* (0.02)	−0.05** (0.02)	−0.04* (0.02)
2004 (eighth)	0.00 (0.07)	−0.28*** (0.07)	−0.1 (0.07)	−0.11 (0.07)	−0.20** (0.08)	—	—	—	—	—
2005 (eighth)	0.00 (0.06)	−0.18** (0.06)	−0.08 (0.06)	0.02 (0.07)	−0.05 (0.07)	0.00 (0.03)	0.03 (0.02)	0.00 (0.03)	0.10*** (0.03)	0.10*** (0.03)
2006 (eighth)	−0.10** (0.04)	−0.02 (0.04)	−0.07^† (0.04)	0.06 (0.04)	−0.12** (0.04)	−0.08** (0.02)	0.17*** (0.03)	0.00 (0.02)	0.15*** (0.03)	0 (0.03)
2007 (eighth)	−0.09** (0.03)	−0.04 (0.03)	−0.01 (0.03)	0.04 (0.03)	−0.10** (0.03)	−0.07** (0.02)	0.09*** (0.02)	0.04^† (0.02)	0.10*** (0.02)	−0.02 (0.02)
2008 (eighth)	0.02 (0.02)	−0.15*** (0.02)	0.04^† (0.02)	−0.01 (0.02)	−0.12*** (0.02)	0.01 (0.02)	−0.10*** (0.02)	0.05* (0.02)	0.00 (0.02)	−0.10*** (0.02)
2009 (eighth)	—	—	—	—	—	—	—	—	—	—
2010 (eighth)	−0.05* (0.03)	−0.16*** (0.03)	0.03 (0.03)	0.10*** (0.03)	0.07** (0.03)	−0.06* (0.02)	−0.22*** (0.02)	0.01 (0.03)	0.07** (0.03)	0.04 (0.03)
% Black/Hispanic	0.40 (1.19)	0.39 (1.16)	0.47 (1.17)	0.28 (1.14)	−0.16 (1.11)	−0.29 (1.09)	−0.03 (1.12)	−0.09 (1.15)	−0.55 (1.12)	−0.48 (1.00)
% ELL	0.02 (0.23)	0.03 (0.27)	0.19 (0.23)	0.05 (0.29)	0.52* (0.21)	0.39 (0.36)	0.39 (0.33)	0.14 (0.33)	0.38 (0.37)	0.37 (0.38)
Ln(eighth-grade enrollment)	0.13 (0.19)	0.08 (0.19)	0.05 (0.19)	0.00 (0.19)	0.14 (0.20)	−0.25 (0.20)	−0.31 (0.20)	−0.27 (0.21)	−0.36^† (0.21)	−0.23 (0.21)
Lagged district API	0.16* (0.07)	0.17* (0.07)	0.19* (0.07)	0.20* (0.08)	0.22** (0.08)	−0.11 (0.11)	−0.13 (0.11)	−0.17 (0.12)	−0.14 (0.12)	−0.14 (0.12)
Constant	−0.89 (1.91)	−0.41 (1.91)	−0.31 (1.89)	0.19 (1.86)	−0.66 (1.87)	0.05* (0.02)	0.11*** (0.02)	0.08** (0.02)	0.08*** (0.02)	0.09*** (0.03)

Adjusted R²	.06	.19	.12	.14	.17	.04	.15	.02	.04	.07
No. of districts	222	222	222	222	222	222	222	222	222	222
ρ	.85	.85	.85	.85	.85	.18	.17	.17	.17	.17

Note. CAHSEE = California High School Exit Exam; std = standardized; API = Academic Performance Index; ELL = English-language learner.

†

p < .10. *p < .05. **p < .01. ***p < .001.

The district fixed effects analyses reported in Tables 3 through 5 identify the effects of curricular intensification off of within-district changes in the percent of eighth graders enrolled in Algebra or higher over time. In Table 6, we build on these results by examining (a) whether the effects that we observe are driven by large year-to-year shifts in Algebra enrollment (such as the sharp increase that Figure 1 shows between 2008 and 2009 in Anaheim), (b) whether enrollment changes that occurred before and after the 2008 policy had similar effects, and (c) whether the effects that we see are short-term implementation costs associated with curricular intensification. We examine each of these points in turn, first using the district fixed effects approach (Panel A) and second using the random growth modeling approach (Panel B).

Table 6

Fixed Effects and Random Growth Model Coefficients, Predictors of District Mean CAHSEE Math Test Scores, California Public School Districts 2003–2004 to 2009–2010 (Balanced Panel)

	Panel A: Fixed effects models				Panel B: Random growth models
	(1)	(2)	(3)	(4)	(1)	(2)	(3)	(4)
% eighth graders ≥ Algebra (std)	−0.07* (0.03)	−0.07** (0.02)	−0.05** (0.02)	−0.05* (0.02)	−0.03 (0.03)	−0.04* (0.02)	−0.04 (0.03)	−0.04 (0.03)
>2 SD incr.	0.06 (0.05)	—	—	—	−0.00 (0.04)	—	—	—
>1 SD incr.	−0.01 (0.04)	—	—	—	−0.01 (0.03)	—	—	—
>2 SD dcr.	−0.02 (0.05)	—	—	—	0.02 (0.04)	—	—	—
>1 SD dcr.	0.05 (0.08)	—	—	—	0.10 (0.09)	—	—	—
Post-policy period	—	−0.02 (0.03)	—	—	—	−0.03 (0.03)	—	—
% Algebra × Post-policy	—	0.03 (0.03)	—	—	—	0.04^† (0.02)	—	—
% eighth graders ≥ Algebra (lag, std)	—	—	−0.01 (0.02)	—	—	—	0.01 (0.01)	—
% eighth graders ≥ Algebra (2nd lag, std)	—	—	—	−0.02 (0.02)	—	—	—	0.00 (0.02)
2005 (eighth)	−0.15 (0.09)	—	−0.16^† (0.09)	—	—	—	0.04 (0.05)	—
2006 (eighth)	−0.14* (0.06)	—	−0.14* (0.06)	−0.15* (0.07)	0.04 (0.05)	—	—	—
2007 (eighth)	−0.15** (0.05)	—	−0.15*** (0.04)	−0.16** (0.05)	−0.04 (0.04)	—	−0.04 (0.04)	−0.06** (0.02)
2008 (eighth)	−0.04 (0.03)	—	−0.04 (0.03)	−0.04 (0.03)	−0.01 (0.02)	—	0.00 (0.02)	−0.01 (0.02)
2009 (eighth)	—	—	—	—	—	—	—	—
2010 (eighth)	0.01 (0.03)	—	0.01 (0.03)	0.01 (0.03)	−0.03 (0.03)	—	−0.04 (0.03)	−0.03 (0.03)
Controls	^†	^†	^†	^†	^†	^†	^†	^†
Constant	−0.54 (2.55)	0.51 (2.29)	0.19 (2.43)	0.47 (2.44)	0.10*** (0.02)	0.12*** (0.02)	0.10*** (0.02)	0.08*** (0.02)

Adjusted R²	.34	.38	.34	.29	.04	.01	.03	.03
No. of districts	219	220	219	216	211	213	211	209
ρ	.89	.81	.89	.9	.37	.32	.36	.38

Note. CAHSEE = California High School Exit Exam; incr. = increasing; dcr. = decreasing; std = standardized.

†

p < .10. *p < .05. **p < .01. ***p < .001.

The first set of models we estimate in Table 6 (column 1) investigate whether large year-to-year changes in eighth-grade Algebra enrollment rates have disproportionately large achievement consequences. These models add indicator variables for districts in the years in which eighth-grade Algebra enrollment rates increased or decreased by more than a standard deviation (>1 SD increasing and >1 SD decreasing) and more than 2 standard deviations (>2 SD increasing and >2 SD decreasing) as defined in the 2004 district eighth-grade Algebra enrollment rate distribution. Using both the fixed effects and the random growth approach, these models return coefficients that are consistent with our previous findings. The linear eighth-grade Algebra enrollment rate term is negative in both models, although this coefficient is not statistically significant in the less efficient random growth model. Importantly, none of coefficients associated with districts that experienced relatively large year-to-year changes in eighth-grade Algebra enrollment rates are statistically significant, indicating that the effects of changes in eighth-grade Algebra enrollment rates are fairly linear. Although districts that make large increases in eighth-grade Algebra enrollment rates tend to see larger achievement declines than districts that make change eighth-grade Algebra enrollment rates more incrementally, large changes do not appear to have disproportionately large effects and do not drive negative eighth-grade Algebra rate effects statewide.

The second set of models in Table 6 (column 2) test whether changes in eighth-grade Algebra enrollment rates that occur prior to the California Board of Education’s attempt to mandate Algebra for all eighth graders have different achievement effects than changes that occur after the mandate. (The “post-policy” indicator in this model takes a value of 1 for observations in the 2009 and 2010 school year and a value of 0 for all other years.) If the best-organized districts increased their eighth-grade Algebra enrollment before the state’s eighth-grade Algebra policy went into effect, one might expect the Post-Policy × Eighth-Grade Algebra interaction in this analysis to be negative. However, one might expect a positive interaction if the development of effective instructional practices accompanied the statewide move toward increased Algebra enrollment. The main effect of eighth-grade Algebra enrollment rate is significant and negative, indicating that increasing eighth-grade Algebra enrollments by a standard deviation in the pre-policy period decreased CAHSEE scores. These effects are consistent across the two modeling strategies, with the fixed effects model returning a −0.07 effect and the random growth model returning a −0.04 effect. In both models, the Eighth-Grade Algebra × Post-Policy interaction returns a positive coefficient. Although not statistically significant, its sign indicates that post-policy increases in eighth-grade Algebra enrollment rates may have had less pronounced negative effects on math achievement than pre-policy increases.

Models 3 and 4 add lagged eighth-grade Algebra enrollment rate control variables to the basic fixed effects model to investigate the extent to which year-to-year changes in eighth-grade Algebra enrollment rates have lasting negative consequences for CAHSEE math scores. If these lagged scores were significant and positive, they would suggest that observed negative effects of increases in eighth-grade Algebra enrollment are short-lived and that average test scores tend to bounce back as districts design strategies to effectively educate a larger proportion of students in advanced middle school math courses. However, the coefficients for lagged eighth-grade Algebra enrollment rates (in Model 3) and twice-lagged eighth-grade Algebra enrollment rates (in Model 4) are both small and not statistically significant, suggesting that eighth-grade Algebra enrollment rate changes have lasting consequences for district CAHSEE math test score trajectories. Although controlling for the lagged score does not reduce the magnitude of the main effect for eighth-grade Algebra enrollment rate, it does reduce the precision of this estimate in the random growth model. As a result, the random growth model estimate of the effect of eighth-grade Algebra enrollment is not significant net of lags.

The findings reported thus suggest that efforts to intensify middle school mathematics curricula have unintended negative consequences for student mathematics achievement across a broad range of domains. However, the analyses reported in Table 7 provide a glimmer of hope, indicating that curricular intensification efforts do not always have negative effects on student achievement. To estimate these models, we split California unified public school districts into tertiles based on their 2004 eighth-grade enrollments and then estimate a district fixed effects analysis of the effects of curricular intensification on CAHSEE math scores for each of these subgroups. The resulting analysis suggests that the negative effects of increasing eighth-grade Algebra occur exclusively in relatively large school districts (in this analysis, districts enrolling 850 or more eighth graders annually, a category that includes California’s largest urban school districts as well as mid-sized suburban and rural districts).⁸ These findings are nearly identical whether estimated using a fixed effects or random growth modeling strategy and are robust to alternate district size categorizations. In small- and middle-sized districts, middle school mathematics curricular intensification has no effect on student achievement. In large districts, however, a 1 standard deviation increase in eighth-grade Algebra enrollment decreases mean CAHSEE scores by 0.05 to 0.07 standard deviations, net of time-invariant district characteristics and controls. This result does not seem to be driven by any one large district. For example, the model returns nearly identical results if Los Angeles Unified or any other large district is excluded from the analysis. Although we are unable to investigate the reasons underlying these differential effects, we suspect that curricular change entails particularly pronounced logistical challenges in large schools and districts. As these have the capacity to offer highly differentiated middle school mathematics instruction, the decision to enroll more students in mathematics is likely require the reassignment of large numbers of teachers as well as profound changes to the organization and composition of both Algebra and less advanced mathematics courses. These bureaucratic challenges are important because more than 80% of the eighth graders enrolled in the analysis districts attended high-enrollment districts.

Table 7

Fixed Effects and Random Growth Model Coefficients, Predictors of District Mean CAHSEE Math Test Scores by 2003–2004 District Enrollment Tertile, California Public School Districts 2003–2004 to 2009–2010 (Balanced Panel)

	Panel A: Fixed effects models			Panel B: Random growth models
	Low enrollment	Mid-enrollment	High enrollment	Low enrollment	Mid-enrollment	High enrollment
% eighth graders ≥ Algebra (std)	0.01 (0.02)	0.01 (0.03)	−0.07* (0.03)	0.03 (0.02)	0.00 (0.02)	−0.05* (0.02)
2004 (eighth)	−0.05 (0.13)	−0.17 (0.11)	−0.38* (0.15)	0.00 (0.00)	0.00 (0.00)	0.00 (0.00)
2005 (eighth)	−0.03 (0.11)	−0.02 (0.09)	−0.19 (0.13)	0.03 (0.06)	0.07^† (0.04)	0.10* (0.04)
2006 (eighth)	0.09 (0.10)	−0.05 (0.08)	−0.12 (0.09)	0.14* (0.06)	0.08 (0.05)	0.14** (0.05)
2007 (eighth)	−0.02 (0.08)	−0.10^† (0.06)	−0.16* (0.07)	0.04 (0.05)	−0.02 (0.04)	0.01 (0.05)
2008 (eighth)	−0.05 (0.06)	−0.05 (0.05)	−0.01 (0.04)	0.00 (0.05)	−0.04 (0.04)	0.06* (0.03)
2009 (eighth)	—	—	—	—	—	—
2010 (eighth)	−0.01 (0.06)	0.02 (0.06)	−0.05 (0.04)	−0.01 (0.07)	0.00 (0.06)	−0.13** (0.04)
% Black/Hispanic	1.58 (1.05)	−0.64 (0.94)	−2.6 (1.62)	1.93 (1.76)	−0.24 (1.65)	−2.50* (1.08)
% ELL	0.67 (0.56)	−0.4 (0.71)	0.96^† (0.53)	0.9 (0.64)	0.52 (0.76)	1.27 (0.79)
Ln(eighth-grade enrollment)	0.2 (0.18)	0.1 (0.22)	0.07 (0.54)	0.04 (0.19)	−0.08 (0.27)	−0.32 (0.51)
Lagged district API	0.16 (0.10)	0.40** (0.13)	0.53* (0.20)	0.18 (0.13)	−0.12 (0.14)	0.13 (0.15)
Constant	−1.54^† (0.90)	−0.02 (1.34)	1.11 (4.17)	−0.02 (0.03)	0.11*** (0.03)	0.14*** (0.03)

Adjusted R²	.07	.25	.35	.02	.00	.05
No. of districts	61	79	80	56	77	80

Note. CAHSEE = California High School Exit Exam; std = standardized; ELL = English-language learner; API = Academic Performance Index.

†

p < .10. *p < .05. **p < .01. ***p < .001.

In sum, the results of our district fixed effects analyses paint a very discouraging picture of the effects of intensifying middle school mathematics curricula by enrolling more students in eighth-grade Algebra. Contrary to the common-sense predictions of “opportunity to learn” theory and the findings of previous observational studies, these analyses suggest that broad-based efforts to enroll more students in eighth-grade Algebra have negative effects on student achievement in large school districts and no benefits in small or medium districts.

Discussion

The push to increase Algebra taking rates among eighth graders is a prime example of a broader effort to improve U.S. educational and economic competitiveness by increasing the academic rigor of K–12 schools. Although this push has increased Algebra enrollments for traditionally underserved students in California and nationally, it is less clear if it has improved student achievement. Our analyses of a panel of district-level data from California’s school districts highlight a potentially serious unintended consequence of these efforts. We find that district-level increases in eighth-grade Algebra enrollment rates correspond with declines in average CAHSEE scores, including scores on CAHSEE test items covering basic Algebraic skills.

Although disappointing, these results may help to make sense of a puzzling set of findings that have emerged in recent research regarding the consequences of eighth-grade Algebra. Using data from a randomly control trial involving rural New England middle schools that had previously not offered eighth-grade Algebra, Heppen and colleagues (2012) find that enrolling relatively high-achieving students in eighth-grade Algebra courses has positive consequences for student achievement. By contrast, in a series of instrumental variable analyses that take advantage of policy-driven changes in eighth-grade Algebra placements in North Carolina middle schools, Clotfelter and colleagues (2012a) find that eighth-grade Algebra enrollment has negative effects for student mathematics achievement.

The contradiction between these two studies is difficult to resolve if one assumes that the treatment—eighth-grade Algebra—is comparable across these two settings. However, that assumption may not be valid. In this article, we suggest that changes in middle school mathematics placement regimes may have spillover effects for instruction and learning in middle school mathematics courses. Thus, when a district or school moves to enroll more students in eighth-grade Algebra, it changes not just whether a given individual receives access to Algebra instruction, but also affects the teachers and peers that all individuals are likely to encounter, both in Algebra as well as in other classes. Put differently, we suspect that Algebra (and pre-Algebra) means something different in schools that enroll 80% of eighth graders in Algebra than in schools that enroll 40% of eighth graders in Algebra. Likewise, we suspect that intensifying the curriculum to put more students into eighth-grade Algebra is a more challenging task in districts where course placement changes affect a large number of students and teachers.

As such, from a policy perspective, we believe it is important to understand not just the effects of placing any given individual into Algebra ceteris paribus, but also the effects of implementing a broad-based Algebra-for-All policy. By allowing for the possibility that curricular intensification policies may change the broader dynamics of peer and teacher interactions, our panel data models test the net effects of increasing eighth-grade Algebra enrollments on student achievement. Although we lack the data to provide a detailed account of the mechanisms through which these effects occur, our analyses clearly indicate that these net effects are negative. In particular, our results suggest that future work should carefully attend to the challenges associated with implementing curricular intensification policies in large districts, as these districts appear particularly vulnerable to iatrogenic effects.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

1.

Unlike much of the United States, several California localities maintain separate elementary and secondary school districts. As these districts report data separately, we are unable to link 8th-grade math course enrollments in these districts with measures of student 10th-grade mathematics achievement. Therefore, our analyses include only “unified” public school districts in California, or those that administer both elementary and secondary schools. In addition, the analyses exclude data reported separately by state boards of education (which often administer relatively small vocational schools and other special programs for “at-risk” youth”) and charter schools. Approximately two thirds of California 8th graders enroll in the districts that are included in our analyses.

2.

End-of-course tests provide important advantages over course title as a measure of course completion. As California school accountability policy requires all districts report data on end-of-course test-taking for all students using a common form, comparable data are available on this measure across districts and over time. Although course-enrollment data are not publicly available for California public school students, analyses of data from one large California public school district indicates that end-of-course tests provide a relatively reliable proxy for course content. In this district, approximately 99% of eighth graders who enroll in pre-Algebra courses take the eighth-grade General Mathematics California Standards Test (CST). Similarly, 99% of students in Algebra I courses enroll in the eighth-grade Algebra CST. In addition, several schools in this district offer a 2-year Algebra course sequence. Approximately 95% of the eighth graders who take the first year of this 2-year Algebra course sequence take the eighth-grade General Mathematics CST. Analyses of data from another large California public school district point to a similarly high level of correspondence between course enrollment and end-of-course CST completion (Taylor, 2011).

3.

We do not include a control for percent of students who are eligible for free or reduced lunch because this variable is highly correlated with the percent of minority students and the percent of English-language learner (ELL) students (correlations between percent minority and both of these variables is roughly .7). It is also worth noting that socio-economic composition of California school districts changes very little during the study period, so that these district fixed effects largely account for these demographic characteristics.

4.

Although students may retake the California High School Exit Exam (CAHSEE) multiple times after failing initially, we use only the first CAHSEE attempt in our analyses.

5.

During the timespan of our district-level panel, California included incentives for districts to enroll students in Algebra in its Academic Performance Index (API) calculation. It is therefore likely that the percent of students enrolled in at least Algebra in time t, is a function of a districts’ API score from t − 1. For this reason, we control for the districts’ prior API score.

6.

To estimate this, we start with the model: $Y_{d t + 2} = β_{1} {(% Alg)}_{d t} + β_{2} X_{d t} + α_{t} + α_{t} \times τ + ε_{d t}$ . We take the first-difference of this equation to remove district-specific intercepts and are left with a constant trend for each district. We then estimate the first-difference of Equation 2 using district and time fixed effects to remove the district-specific constant trends (see Papke, 1994; Wooldridge, 2002; Zimmer & Buddin, 2006, for more details about the random growth model).

7.

During this period, the Latino share of the California eighth-grade population increased by approximately two percentage points, while the African American share declined.

8.

Districts in the lowest enrollment tertile enroll fewer than 300 eighth graders annually; districts in the middle tertile enroll between 300 and 850 eighth graders annually; eighth-grade enrollments in the top tertile range from 850 to more than 50,000. The districts in the lower two tertiles are nearly all located in rural areas, and enroll somewhat fewer students of color and ELL than the large districts. However, these demographic differences are not as pronounced as one might expect. Forty-three percent of students low-enrollment districts are Black or Hispanic, compared with 47% of students in the middle enrollment districts and 59% of students in high-enrollment districts.

Authors

THURSTON DOMINA is an associate professor of education and sociology at the University of California, Irvine. His work focuses on policies and practices related placement and instruction in American secondary and post-secondary education.

ANDREW MCEACHIN, is an assistant professor of Education Policy and Program Evaluation at North Carolina State University, College of Education. His research focuses on the design and impact of accountability systems, math education policy, and the economics of education.

ANDREW PENNER is an associate professor of sociology and chancellor’s fellow at the University of California, Irvine. His research focuses on gender and race inequality in the labor market and educational system.

EMILY PENNER is a postdoctoral fellow at the Center for Educational Policy Analysis at Stanford University. Her research focuses on educational inequality and policy, and considers the ways that districts, schools, and teachers, peers, and parents can contribute to or ameliorate educational inequality.

References

Adelman

(1999). Answers in the toolbox: Academic intensity, attendance patterns and bachelor’s degree attainment. Washington, DC: U.S. Department of Education.

Allensworth

Nomi

Montgomery

Lee

V. E.

(2009). College preparatory curriculum for all: Academic consequences of requiring algebra and English I for ninth graders in Chicago. Educational Evaluation and Policy Analysis, 31, 367–391.

Argys

L. M.

Rees

D. I.

Brewer

D. J.

(1996). Detracking America’s schools: Equity at zero cost? Journal of Policy Analysis and Management, 15, 623–645.

Attewell

Domina

(2008). Raising the bar: Curricular intensity and academic performance. Educational Evaluation and Policy Analysis, 30, 51–71.

Becker

D. E.

Watters

Sacramento

C. A.

(2008). Independent evaluation of the California High School Exit Examination (CAHSEE) fourth biennial report (FR-08-12). Alexandria, VA: Human Resources Research Organization. Retrieved from http://www.cde.ca.gov/ta/tg/hs/documents/fourthbiennialrpt.pdf

Becker

D. E.

Wise

L. L.

Mardoin

M. M.

Watters

(2012). Independent evaluation of the California High School Exit Examination: 2012 biennial report (FR-11-82). Alexandria, VA: Human Resources Research Organization.

Becker

D. E.

Wise

L. L.

Watters

(2010). Independent evaluation of the California High School Exit Exam (CAHSEE) 2010 biennial report (FR-10-01). Alexandria, VA: Human Resources Research Organization.

Carraher

D. W.

Schliemann

A. D.

(2007). Early algebra. In Lester

F. K.

(Ed.), Second handbook of research on mathematics teaching and learning (pp. 669–706). Reston, VA: National Council of Teachers of Mathematics.

Clotfelter

C. T.

Ladd

H. F.

Vigdor

J. L.

(2012a). The aftermath of accelerating algebra: Evidence from a district policy initiative (National Center for Analysis of Longitudinal Data in Education Research, Working Paper 69). Available from www.caldercenter.org

10.

Clotfelter

C. T.

Ladd

H. F.

Vigdor

J. L.

(2012b). Algebra for 8th graders: Evidence on its effects from 10 North Carolina districts (No. w18649). Cambridge, MA: National Bureau of Economic Research.

11.

Dee

T. S.

Jacob

(2011). The impact of No Child Left Behind on student achievement. Journal of Policy Analysis and Management, 30, 418–446.

12.

Domina

(2014). The link between middle school mathematics course placement and achievement. Child Development. Advance online publication. doi:10.1111/cdev.12255

13.

Domina

Penner

Conley

A. M.

(2014). Algebra for all: California’s 8th grade algebra initiative as constrained curricula. Teachers College Record, 116(8). Retrieved from http://www.tcrecord.org/Content.asp?ContentId=17511

14.

Domina

Saldana

(2012). Does raising the bar level the playing field? Mathematics curricular intensification and inequality in American high schools, 1982–2004. American Educational Research Journal, 49, 685–708.

15.

Gamoran

Hannigan

E. C.

(2000). Algebra for everyone? Benefits of college-preparatory mathematics for students with diverse abilities in early secondary school. Educational Evaluation and Policy Analysis, 22, 241–254.

16.

Gamoran

Porter

A. C.

Smithson

White

P. A.

(1997). Upgrading high school mathematics instruction: Improving learning opportunities for low-achieving, low-income youth. Educational Evaluation and Policy Analysis, 19, 325–338.

17.

Goldin

C. D.

Katz

L. F.

(2008). The race between education and technology. Cambridge, MA: Belknap Press.

18.

Gonzalez

Williams

Jocelyn

Roey

Kastberg

Brenwald

(2008). Highlights from TIMSS 2007: Mathematics and science achievement of U.S. fourth and eighth-grade students in an international context. Washington, DC: National Center for Education Statistics.

19.

Heppen

J. B.

Walters

Clements

Faria

Tobey

Sorensen

Culp

(2012). Access to Algebra I: The effects of online mathematics for grade 8 students (NCEE 2012–4021). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education.

20.

Howe

(2005). Comments on NAEP algebra problems. Retrieved from http://www.brookings.edu/~/media/events/2005/9/14%20algebraic%20reasoning/howe_presentation.pdf

21.

Human Resource Research Organization. (2012). Independent evaluation of the California High School Exit Examination: 2012 evaluation. Alexandria, VA: Human Resources Research Organization. Retrieved from http://www.cde.ca.gov/ta/tg/hs/documents/cahsee2012evalrpt.pdf

22.

Liang

Heckman

P. E.

Abedi

(2012). What do the California Standards Test results reveal about the movement toward eighth-grade algebra for all? Educational Evaluation and Policy Analysis, 34, 328–343.

23.

Long

M. C.

Conger

Iatarola

(2012, April). Effects of high school course-taking on secondary and postsecondary success. American Educational Research Journal, 49, 285–322.

24.

Loveless

(2008). The misplaced math student lost in eighth-grade algebra (The 2008 Brown Center Report on American Education). Washington, DC: Brookings Institute.

25.

McPartland

J. M.

Schneider

(1996). Opportunities to learn and student diversity: Prospects and pitfalls of a common core curriculum. Sociology of Education, 69, 66–81.

26.

Nomi

(2012). The unintended consequences of an algebra-for-all policy on high-skill students: Effects on instructional organization and students’ academic outcomes. Educational Evaluation and Policy Analysis, 34, 489–505.

27.

Nomi

Allensworth

(2009). “Double-dose” algebra as an alternative strategy to remediation: Effects on students’ academic outcomes. Journal of Research on Educational Effectiveness, 2, 111–148.

28.

Organization for Economic Co-Operation and Development. (2009). Learning mathematics for life: A perspective from PISA. Paris, France: Author.

29.

Papke

L. E.

(1994). Tax policy and urban development: Evidence from the Indiana enterprise zone program. Journal of Public Economics, 54, 37–49.

30.

Porter

A. C.

(2002). Measuring the content of instruction: Uses in research and practice. Educational Researcher, 31(7), 3–14.

31.

Reardon

(2008). Thirteen ways of looking at the black-white test score gap (Stanford Institute for Research on Education Policy & Practice, Working Paper, 8). Retrieved from http://www.stanford.edu/group/irepp/cgi-bin/joomla/docman/thirteen-ways-of-looking-at-the-black-white-test-score-gap/download.html

32.

Rosin

M. S.

Barondess

Leichty

(2009). Algebra policy in California: Great expectations and serious challenges. Mountain View, CA: EdSource.

33.

Schmidt

W. H.

(2012). At the precipice: The story of mathematics education in the United States. Peabody Journal of Education, 87, 133–156.

34.

Schmidt

W. H.

McKnight

C. C.

Houang

R. T.

Wang

H. C.

Wiley

D. E.

Cogan

L. S.

Wolfe

R. G.

(2001). Why schools matter: A cross-national comparison of curriculum and learning. San Francisco, CA: Jossey-Bass.

35.

Stein

M. K.

Kaufman

J. H.

Sherman

Hillen

A. F.

(2011). Algebra: A challenge at the crossroads of policy and practice. Review of Educational Research, 81, 453–492.

36.

Taylor

D. J.

(2011). Outcomes of placing low performing eighth grade students in Algebra content courses (Doctoral dissertation). Available from ProQuest Dissertations and Theses database. (UMI No. 3474480)

37.

Vogel

(2008). Algebra: Changing the equation. District Administration, 44, 34–40.

38.

Wise

L. L.

Becker

D. S.

Harris

C. D.

Taylor

L. R.

Johnstone

C. J.

Miller

N. A.

. . . Moody

(2006). Independent evaluation of the California High School Exit Examination (CAHSEE): Third biennial report. Sacramento: California Department of Education.

39.

Wise

L. L.

Harris

C. D.

Koger

L. E.

Bacci

E. D.

Ford

J. P.

Brown

D. G.

. . . Coumbe

K. L.

(2004). Independent evasluation of the California High School Exit Examination (CAHSEE): Second biennial report (FR-04-01). Alexandria, VA: Human Resources Research Organization.

40.

Wooldridge

J. M.

(2002). Econometric analysis of cross section and panel data. Cambridge, MA: MIT Press.

41.

Wurman

Evers

W. M.

(2011, February 09). New education dashboard: Less rigorous, less meaningful. Education Week, 30(20). Retrieved from http://www.edweek.org/ew/articles/2011/01/28/20evers.h30.html

42.

Zimmer

R. W.

Buddin

(2006). Charter school performance in two large urban districts. Journal of Urban Economics, 60, 307–326.

43.

Zimmer

R. W.

Toma

E. F.

(2000). Peer effects in private and public schools across countries. Journal of Policy Analysis and Management, 19, 75–92.