Competing With Charter Schools: Selection,Retention,and Achievement in Los Angeles Pilot Schools

Abstract

Charter schools place competitive pressure on school districts to retain students and public funding. Many districts also have moved to decentralize control of budgets and teacher hiring down to school principals, independent of competitive pressures. But almost no evaluation evidence gauges the effectiveness of charter-like schools, relative to traditional public schools. We find that autonomous pilot schools in Los Angeles enroll more low-income and Spanish-speaking students, compared with traditional schools. Pilot pupils are significantly less likely to exit the school district. But pilot pupils displayed lower test scores in mathematics and fell slightly below traditional students in English-language arts, taking into account prior performance and their propensity to enter pilot schools. We tracked 6,732 students entering pilot high schools between 2008 and 2012, statistically matched in multiple ways with traditional peers from identical sending middle schools. We discuss the advantages of our evaluation strategy and the implications of our findings for education leaders and policy makers.

Keywords

education content area quasi-experimental design methodology (if appropriate)economic evaluation design and evaluation of programs and policies

Many school districts have created small or site-managed high schools, responding to the competitive threat from charter schools or pressed by neighborhood activists who seek more responsive campuses.¹ New York City, for instance, disassembled scores of large schools, then created human-scale organizations, often highlighting a distinct curricular mission (Abdulkadiroğlu et al., 2013; Bloom & Unterman, 2014; O’Day et al., 2011). Chicago, Miami, and San Francisco have expanded magnet schools or experimented with “site-based budgeting,” granting principals and teachers discretion over budgets and hiring decisions, detaching campuses from bureaucratic oversight and most labor rules.

This article examines the intriguing case of pilot schools, a close cousin of charters in which principals shape their campus budget and hire and fire their own teachers. But pilot teachers—growing in number across Boston and Los Angeles (LA)—remain in the district personnel system, retaining fringe benefits and union membership (Abdulkadiroğlu et al., 2011). This article examines the relative efficacy of pilot schools in Los Angeles to attract students, retain them in the district, and lift their achievement early in high school.

In contrast to the extensive literature on charter impacts, evidence on these alternative schools remains limited. Like charters, within-district alternatives to traditional public schools (TPS) vary substantially in their organizational structure and the kinds of students and families they serve. Nor is competitive pressure from charters the only force that spurs district leaders to create a diversifying set of campuses, including semiautonomous pilot schools. When evaluating family selection, student retention, or achievement effects, we must take into account differences in local contexts. (By retention, we refer to a student’s likelihood of remaining within, not exiting, the school district.)

We focus on the rise and effects of alternative high schools in the nation’s second largest school district, the Los Angeles Unified School District (LAUSD). In the mid-2000s, community demands for increased choice led to the establishment of pilot high schools, small high schools (SHS) that enjoy substantially greater autonomy than traditional district schools. We discuss how pilot advocates rejected large, conventional high schools while preferring to maintain distance from the charter movement. Using detailed administrative data, we estimate causal effects of pilot high schools over their initial period of expansion from 2008 to 2013, as charter schools were acquiring nearly one fifth of district enrollment. We examine whether pilots attract a representative set of students and lift learning more effectively than traditional high schools. We also compiled data on possible mechanisms through which pilots may differentially affect learning, including courses taken by pupils and teacher attributes.

Students entering pilot schools were more likely to be eligible for free or reduced price lunch (FRPL) and had lower test scores, on average, than TPS students, rooted in the poor, often immigrant parts of LA in which they were first founded. To identify causal effects of pilots, we statistically match 6,732 pilot pupils to similar TPS students residing in the same neighborhoods. We account for the geographic location of pilot schools relative to TPS by matching students on sending middle school, in addition to demographic attributes and eighth-grade test scores.

Our results show that pilots succeeded in reducing exit rates from the district during the transition from 9th to 10th grade. Student retention is a key issue in districts losing enrollment over time, including LAUSD. While charters tend to display higher rates of student retention and graduation compared with nearby TPS, there is little prior evidence on whether charter-like schools, still overseen by school boards, prove more effective in retaining students (Angrist et al., 2016; Epple et al., 2015; Lauen, 2009; Nichols-Barrer et al., 2016).

Despite positive retention effects, pilots failed to elevate ninth-grade math and English-language arts (ELA) scores on the California Standards Tests (CST). Students in pilot high schools took more advanced math courses but tested below peers who remained in TPS, with higher gaps in ninth-grade math achievement for students who performed highly in eighth grade.² This dampening of achievement was centered within particular pilots and failed to improve as the count of pilots grew to 29 campuses by 2013, no longer situated solely in the poorest parts of LA. We identified few mediating factors to explain these differences in achievement, the exception being differences in math course taking. These achievement results differ substantially from two other large-scale studies of site-managed high schools in Boston and New York, which we detail below. Understanding these differing effects is an important policy question as districts introduce new site-managed schools to compete with charters and respond to neighborhood activists seeking locally responsive institutions.

This article describes the institutional origins of pilots and reviews the literature on causal effects of decentralized, site-run public schools. Next, we describe our data and estimation strategy. Findings are then presented for main results and mediating mechanisms. We conclude by placing our evidence in the context of prior literature, along with implications for policy.

Institutional Origins of Pilot Schools

Support for decentralized school management stems from several sources. Early research on “school effects” pointed to the importance of strong principal leadership and organized cohesion among teachers in fostering a motivating workplace and effective pedagogies (Purkey & Smith, 1985; Reynolds & Teddlie, 2000). Pro-market scholars also supported decentralizing authority out to principals, along with flexible labor contracts, as strategies to retain strong teachers and raise academic expectations (Chubb & Moe, 1990; Scott, 2013). Finally, progressive educators and activists recurrently push to detach schools from state rules or regulation by downtown bureaucracy, apparent in a variety of “neighborhood control” movements (Bryk et al., 2010; Fliegel & MacGuire, 1993; Ravitch, 1974).

Rising support for decentered management and liberalized parental choice, combined with competitive pressure to raise quality, have spurred the creation of diverse forms of schools (reviews, Epple et al., 2015; Fuller, 2015). Charters serve 6% of all K–12 students nationwide, and the sector continues to expand. Facing losses in enrollment and funding to charters, urban districts increasingly devise competing forms of site-managed schools. For example, magnet programs have expanded in recent years, enrolling 2.3 million students, situated on over 3,000 campuses nationwide (Rich, 2014; Snyder & Dillow, 2012). In total, nearly one third of all students nationwide leave neighborhood attendance zones to attend charters, magnets, pilots, or other TPS outside their residential area (Goldring et al., 2013; Jacob & Wolf, 2012).

We focus on pilot schools, a recent response to the charter school threat in Boston and LA, along with efforts by pro-equity advocates to create locally responsive schools. Pilots enjoy variable freedom from district bureaucracy and operate under “thin” labor contracts. Principals gain authority over school budgets and teacher hiring, while teachers retain fringe benefits, stay within the district personnel system, and remain union members. Pilots in LA typically display a distinct curricular mission, such as the arts, mathematics and science, health professions, or social justice. Teachers sign a “right to work” agreement each year and forgo most due process rights (Abdulkadiroğlu et al., 2011; Martinez & Hunter Quartz, 2012).³

Pilot schools in LA emerged from a two-decade-long struggle that aimed to decentralize the management and shrink the size of high schools. By the late 1990s, two major civic efforts—aiming to devolve control of school budgets and teacher hiring out to principals—had been eroded by union opposition and discomfort among district leaders (Kerchner et al., 2008). Meanwhile, the pilot model emerged in Boston in 1995, as charter schools took root and many progressive educators sought a third ground, rejecting large conventional high schools, while remaining skeptical of the charter model. Several years later, LAUSD Chief Ray Cortines sent a team to Boston, returning with this appealing model for Latino leaders and advocates who hoped to shrink overcrowded high schools and gain neighborhood influence over teacher hiring and curricular relevance.⁴

The 10 original pilot high schools in LA were established in 2006, negotiated by LAUSD officials, Latino community activists, and the United Teachers of Los Angeles (UTLA). The novel governance arrangement created the Belmont Zone of Choice, situated in low-income Latino neighborhoods located north and east of downtown, including the heavily immigrant areas of Pico Union and East LA. Parents were now allowed to select from among all schools located within the zone, no longer bound by their particular attendance area. LAUSD was midway through a massive school construction program, eventually building 130 new facilities to ease severe overcrowding in poor areas (Nesoff, 2007). This presented the opportunity of siting new pilot high schools in entirely new buildings.

Parents living in attendance zones within these communities were awarded first priority to enroll their teenage children. In addition, under the Belmont Zone of Choice, families could search for pilot schools or TPS not located in their immediate attendance zone (LAUSD, 2016). This institutional and geographic history likely shaped which families selected into the new pilot schools and how they may differ from demographic and economic attributes of families district-wide. These largely Latino and first- or second-generation immigrant parents were less likely to share middle-class features of many LAUSD families selecting into charter schools.

The UTLA agreed in 2011 to lift the cap on pilots, given the widening enthusiasm for this model expressed by equity-minded teachers, many parents, and Latino leaders (Kohl & Farris-Berg, 2014). LAUSD also consolidated the family application process, pitching pilots and other site-run schools within a wider school-choice framework. Table 1 specifies the organizational differences between pilot and charter schools.

Table 1.

Summary Differences Between Pilot and Charter School Models.

Element	Charter Schools	Pilot Schools
School site managers	Staff of independent nonprofit	School district employee
Teachers	Private employees	District employees
Benefits	Young teachers, scarce benefits	Mix of ages, regular benefits
Labor rules	No labor contract	“Thin” labor MOU with union
Attendance boundaries	No attendance boundaries	Most pilots draw students from within boundaries
Budget	Total flexibility	Annual budget negotiated with district

Note. MOU = memo of understanding.

The geographic distribution of charter and pilot schools shifted remarkably across LAUSD between 2002 and 2015 (Figure 1). We see that pilot growth has been strongest on the north edge of downtown (the Pico Union area), to the east (Boyle Heights and East LA, corresponding to the Belmont Zone), and south to the Watts District (small pentagons). In contrast, early charter growth occurred in middle-class areas to the west (close to Santa Monica) and north in the San Fernando Valley (triangles). This included both TPS converting to charter status and start-up charter schools. By 2015, charter operators had moved into poorer neighborhoods, including where pilots had located and South LA, home to a blend of Black and Latino families (dots).

Figure 1.

Spread of charter and pilot schools in Los Angeles Unified School District, 2002–2015.

Expanding the count of pilots then became a key plank in LAUSD’s 2009 Public School Choice Initiative, which aimed to off-load up to 150 schools to nonprofit or pilot groups, including charter firms (Strunk et al., 2016). By 2019, the district operated 44 pilot (elementary and high) schools, along with 46 additional schools with intermediate levels of autonomy from the downtown administration, lying between TPS and pilots.⁵ High schools make up three fourths of all LAUSD pilots. Our analysis includes the 29 pilot high schools established prior to 2013.⁶

Earlier Findings—Selection, Retention, and Achievement in Decentralized Schools

Earlier work most relevant to our study comes from the 2011 evaluation of Boston’s pilot schools conducted by Abdulkadiroğlu and colleagues (2011). Since LAUSD imported the Boston model of pilot schools, the two networks share many of the same institutional characteristics.⁷ Exploiting excess numbers of students applying to pilots in Boston, these researchers compared student attributes and outcomes relative to applicants who lost in lottery selection and remained in traditional schools.

They found that compared with TPS peers, students entering pilot high schools displayed higher eighth-grade test scores (prior to winning lottery admission into a pilot), moderately lower rates of FRPL eligibility, and substantially lower rates of limited English proficiency. Latino students were underrepresented in Boston pilot schools.

Abdulkadiroğlu et al. (2011) then estimated causal effects from a sample of 1,973 students who applied to a pilot high school and were accepted or rejected based on a lottery. For the sample of students who attended 10th grade between 2003 and 2006, they find that attending a pilot high school had no effect on ELA and math test scores, controlling for student demographics and eighth-grade test scores. In contrast, charter school students realized test score gains of about .20 SD in math and .30 SD in ELA.

Another prominent example of site-managed schools comes from New York City, where the district created more than 150 SHS between 2002 and 2008. Like pilots, these schools were given greater autonomy than TPS with respect to curriculum and teacher hiring. Unlike pilots, New York’s SHS were not granted exemptions from collective bargaining agreements (Abdulkadiroğlu et al., 2013). SHS enrolled greater shares of Black and Latino students than the district as a whole. In contrast to Boston pilot schools, SHS students were negatively selected with respect to eighth-grade math and ELA test scores, but no more likely than TPS students to be English learners or qualify for subsidized lunch.

Using a lottery design in New York City—tracking applicants to newly devised SHS, split by randomly selected admittees and pupils returning to TPS—Abdulkadiroğlu et al. (2013) found positive effects of SHS on test scores, graduation rates, and college enrollment. One of the strongest effects pertained to the holding power of SHS: These students remained in the same school for nearly half-year longer than controls. Achievement on Regents Exams in 9th or 10th grade was higher for SHS students in the order of .11–.16 SD in each year. The high school graduation rate of SHS pupils was 9% higher than for controls, mostly due to a higher rate of passing the Regents Exam; the college entry rate was 7% higher for SHS graduates.

These studies from Boston and New York do not focus on the retention of students within the district as a primary outcome. However, student retention is a key motivating factor in the establishment of charter-like schools and may also speak to student and parent satisfaction. In Boston, Abdulkadiroğlu et al. (2011) found that being accepted into a pilot school reduced the likelihood of switching schools. Site-managed schools may produce environmental benefits—such as greater cohesion between staff and administrators, better engagement with parents, or more effective catering to individual student needs—that offset achievement effects and limited academic or extracurricular offerings.

A larger literature reports impacts of charter schools relative to TPS. While charter schools have generated positive effects on student attainment in a variety of settings, impacts on test scores are mixed (Berends, 2015; Betts & Tang, 2016). Much has been learned about the value-added effects of charter schools in LA. Using data on charter and TPS students for 2007–2011, Shin et al. (2016) report that charter high schools enrolled lower percentages of Latino students than TPS, along with substantially fewer limited English proficient students and those eligible for free or reduced price meals. Charter students were positively selected in terms of eighth-grade test scores, with a difference of about .50 SD between charters and TPS. Students who switched to a charter high school from a TPS displayed significant ELA test score gains of between .10 and .20 SD, relative to observably similar students remaining in a TPS.

These findings are consistent with an earlier study of LA charters, which found an additional .07 SD of ELA test score growth and .08 SD math growth for charter high school students, relative to matched TPS students (Raymond, 2014). In the absence of administrative data on charter students in LA, we cannot estimate how a pilot student would have counterfactually performed at a charter school. However, the charter effects found in earlier studies provide useful benchmarks for the pilot school effects that we estimate.

Evaluation Questions and Method

Given this prior work on site-managed alternative schools, we aimed to evaluate which students select into pilot high schools in LA, relative to district profiles, are they more likely to remain in the school district (retention) and display higher levels of achievement, compared with nearly identical (matched) students who remain in traditional high schools (TPS)? Recall that the pilot movement began within the heavily Latino and immigrant area of East LA, fueled by progressive educators and district reformers intent on creating smaller, more culturally responsive high schools. Other constituencies, especially the teacher union, concurred that LAUSD must devise more innovative schools to compete with charter schools. In addition, the push by Latino activists centered on human-scale campuses that would be more relevant to the cultural and linguistic attributes of their neighborhoods.

Data

Our analysis draws on longitudinal data provided by LAUSD, including student, teacher, school, and course records from fall 2007 to spring 2013. We focus on students in eighth and ninth grades who are transitioning from middle to high school. Our measures of student achievement are scores on the math and ELA sections of the CST, which students took in the eighth and ninth grades over the time series. We normalize test scores by grade and year. Since students have the option of taking several different math tests, we also take into account the math test type, available in all years prior to the 2012–2013 school year.

We observe the gender, ethnicity, parent’s education, and home language of each student, as well as their English proficiency level and eligibility for FRPL in each school year. We also have student attendance information for each semester. From the course records, we identified the courses a student took in each semester, as well as the (anonymized) identity of the course instructor. These identifiers are linked to several teacher characteristics including gender, ethnicity, education level, and years of teaching experience.

We restrict our sample to students who start eighth grade between 2007–2008 and 2011–2012 and appear in both eighth and ninth grades in our data. We exclude students who attend continuation high schools for pupils who have exhibited significant behavior problems or have been expelled from TPS, along with students with missing ELA or math test scores, or who did not take either the General Math or Algebra I math test in eighth grade. Based on these inclusion criteria, of the more than 200,000 students who started ninth grade between 2007–2008 and 2012–2013, we utilize just over 137,000 for our analysis (the analysis sample). Further details on sample construction are in the Data Appendix (appearing online).

Figure 2 displays the count of schools included in the total and matched sample over time. We see the growth in the count of pilot high schools over the time series. In addition, note that many TPS are dropped from the matched sample, given that their pupils cannot be closely matched to their pilot counterparts. This stems from our local matching procedure: students with almost identical individual attributes and attending the same middle schools. And we will see how pilot school students come from poorer, more often Spanish-speaking families than peers attending the average LAUSD high school.

Figure 2.

Number of high schools in pilot and traditional public school samples with students in Grade 9. Source: Los Angeles Unified School District data.

Estimation Strategy

Depending on their residential location, parents hold the option of sending their child to a pilot high school or TPS. Since assignment to pilots is nonrandom, our identification strategy relies on matching treated students (who attend a pilot in ninth grade) to initially similar untreated students (attending a TPS). Since the initial set of LA pilot high schools was not oversubscribed, we could not replicate the lottery-admission method employed in earlier work.

While randomized control trials are the gold standard for estimating causal effects, true experimental effects are in practice very similar to those estimated from high-quality quasi-experimental designs. Previous studies that estimated the effect of school organizational type (charter, pilot, and magnet) on student test scores have found that well-designed matching estimators produce nearly the same results as randomized evaluations (Bifulco, 2012; Fortson et al., 2012; Furgeson et al., 2012; Tuttle et al., 2013).

Sound matching estimators must be both local, in that treatment and control subjects are taken from the same setting, and focal, in that the characteristics used in matching predict both the treatment and the outcome of interest (Clair et al., 2014; Steiner et al., 2011). For our purposes, this requires local matching of students within feeder middle schools, typical student demographics, and baseline (eighth grade) test scores in the matching model. These matching estimators have been used in prior studies of charter and pilot schools (Abdulkadiroğlu et al., 2011; Lauen et al., 2015; Raymond, 2014; Shin et al., 2016).

We match treated to untreated students in two steps. First, we exactly match students on the middle school they attended, gender, ethnicity, and year. In the second step, we perform many-to-one matching based on their propensity score. The equation below describes our formal specification. Two students belong to the same group j if they are middle school classmates (i.e., attending the same eighth-grade school in the same year) and belong to the same gender and ethnicity (Latino or non-Latino). Any group j with no treated students or fewer than three untreated students is dropped from the sample. The logit propensity score model for student i in group j with eighth-grade characteristics X is:

P ({Treated}_{i j} = 1 | X_{i j}) = \frac{1}{1 + e^{- β_{j}^{'} X_{i j}}},

where

\begin{matrix} β' X = β_{0} + β_{1} EnglishScore + β_{2} {EnglishScore}^{2} + β_{3} 1 \{Algebra I test\} \\ + β_{4} GeneralMathScore \times 1 \{General math test\} + β_{5} Algebra I Score \times 1 \{Algebra I test\} + β_{6} {GeneralMathScore}^{2} \times 1 \{General math test\} \\ + β_{7} Algebra I Score \times 1 \{Algebra I test\} \\ + β_{8} 1 \{Parent Ed < High school\} + β_{9} 1 \{Parent Ed missing\} \\ + β_{10} 1 \{Free / RP lunch\} + β_{11} Attendance + β_{12} {Attendance}^{2} \\ + β_{13} 1 \{Attendance missing\} . \end{matrix}

The prediction variables X include quadratic terms in math and ELA scores, interacted with test type (for math), dummies for parent’s education less than high school and parent’s education missing, a dummy for FRPL eligibility, a quadratic in attendance (with missing attendance coded as 0), and a dummy for missing attendance. Interacting math test score with test type allows us to account for differences in scaling across math test types. We set GeneralMathScore to zero if a student took the Algebra I test and vice versa. The set of prediction variables X includes almost all student family and demographic information made available by LAUSD. Our results are robust to using different sets of prediction variables to estimate the propensity score.

We match each pilot student to four TPS students using nearest-neighbors matching, with replacement and without caliper. Since untreated students are matched with replacement, the ratio of TPS to pilot students is less than four to one. We are able to find a match for 6,732 of the 7,390 pilot students in our sample (9% unmatched, Table 2). The final matched sample contains 6,732 pilot and 10,055 TPS students. We report our main results for both the analysis sample and the matched sample to indicate the sensitivity of our findings to the matching procedure. All outcome regressions include the full set of baseline controls available in our data, including but not limited to the characteristics used in matching.⁸ Drawing causal inferences from our study rests on the assumption that both observed and unobserved confounders are similar in the treatment and matched control sample.

Table 2.

Student Characteristics at the Eighth Grade.

Element	Analysis Sample		Matched Sample
Element	TPS	Pilot	TPS	Pilot
Number of students	137,077	7,390	10,055	6,732
Year of fall semester	2,008.7	2,009.8	2,009.6	2,009.8
Female	48.6%	51.2%	50.9%	51.6%
Ethnicity
American Indian	0.3%	0.2%	0.2%	0.1%
Asian	3.1%	3.0%	3.9%	2.1%
Black	8.0%	3.2%	2.0%	1.9%
Filipino	2.1%	2.1%	1.0%	1.7%
Latino	80.6%	89.1%	93.0%	92.1%
Pacific Islander	0.4%	0.3%	0.2%	0.2%
White	5.7%	1.2%	1.4%	0.9%
Parent’s education
Decline to answer	18.5%	27.7%	27.0%	27.6%
Less than high school	39.2%	36.3%	37.8%	37.4%
High school grade	20.4%	18.3%	19.0%	18.4%
Some college	10.9%	7.6%	7.8%	7.3%
College degree	6.4%	4.1%	4.0%	3.8%
Grade school/postgraduate	2.3%	1.2%	1.3%	1.1%
Missing	2.3%	4.8%	3.2%	4.3%
Home language
English	23.8%	15.7%	12.7%	13.9%
Spanish	70.1%	78.1%	83.1%	81.1%
Other	5.6%	4.9%	3.3%	3.8%
Free lunch	82.2%	91.6%	93.0%	92.3%
Limited English proficiency (EP)
English only	24.4%	16.4%	13.2%	14.5%
Limited EP	22.7%	24.2%	25.6%	24.5%
Initially fluent EP	10.3%	9.5%	10.3%	9.6%
Reclassified fluent EP	42.6%	50.0%	50.9%	51.5%
Algebra I course in eighth grade	64.3%	70.5%	67.9%	70.2%
Algebra I CST in eighth grade	61.3%	68.8%	66.2%	68.5%
CST z-scores (normalized by grade year within analysis sample)
ELA	0.00	−0.09	−0.12	−0.10
Math—General	0.01	−0.14	−0.12	−0.15
Math—Algebra I	0.00	−0.06	−0.04	−0.06
Science	0.00	−0.06	−0.07	−0.06
Social sciences	0.01	−0.14	−0.13	−0.15

Note. The analytic sample includes all students who attended eighth grade in Los Angeles Unified School District between 2007 and 2011 and are eligible for matching—they must be present in our data in both eighth and ninth grade (for more details, see Table A1). The matched sample is constructed using within-group propensity score matching, as described in the text (“propensity matching and estimation strategy”). In the matched sample, all balance statistics (standardized mean differences) except for year of eighth-grade fall semester are less than 10 in magnitude. For year of eighth-grade fall semester, the balance statistic is 11.0 and the difference in means is 0.14. CST = California Standards Tests; ELA = English-language arts; TPS = traditional public schools.

Findings

Comparing Pilot and TPS Students

We first report differences between how students attending pilot schools and their neighborhoods may differ from TPS peers. Table 2 displays this comparison for the eighth-grade attributes of students who enter a pilot high school for ninth grade and those who attend a TPS. We report these differences for two samples: the analysis sample of students eligible for matching and the smaller propensity-score matched sample.

In the analysis sample, we see that pilot students are more likely of Latino heritage, compared with TPS peers (89% and 81%, respectively), and more frequently speak Spanish in the home (78% and 70%). Just over four in five TPS eighth graders are eligible for FRPLs, compared with 92% of pilot pupils. Scores on the CST for pilot pupils back in the eighth grade fall significantly below the average TPS student. These selection patterns contrast starkly with Boston pilot high schools, where entering test scores were higher than for peers attending TPS. After matching pilot and TPS students in LA, using the method described above, including an exact match on sending middle school, we predictably see few differences between the two subsectors.⁹

The neighborhoods in which pilots are situated differ as well, compared with areas surrounding TPS (Table 3). We see that census tracts in which pilot schools are located host greater shares of Latino households (78% vs. 66%), compared with the average TPS tract in 2012, along with larger shares of families living in poverty (28% and 20%, respectively). The percentage of households in which Spanish is the dominant language is higher in pilot neighborhoods (72%), compared with the average TPS location (60%). These differences likely stem from the institutional history of pilots, first created in heavily immigrant and second-generation Latino parts of LAUSD, as described above.

Table 3.

Census Tract Characteristics of LAUSD High Schools-ACS 5-Year Estimates, 2012.

Element	TPS	Pilot
Percent Hispanic or Latino	66.1 (27.0)	78.4 (21.8)
Percentage of households with one or more children	44.4 (16.7)	48.0 (18.8)
Average household size	3.4 (0.8)	3.5 (0.8)
Percentage of adults 25+ with less than ninth grade	22.8 (12.6)	31.2 (8.3)
Percentage of adults 25+ with ninth to 12th grade and no diploma	13.3 (6.5)	15.7 (3.5)
Percent born outside United States	41.4 (11.0)	47.2 (11.1)
Percentage of population 5+ speaking a language other than English at home	70.8 (20.3)	81.2 (6.4)
Percentage of population 5+ speaking Spanish at home	60.3 (26.8)	71.5 (19.2)
Median income, all households	47,596.2 (21,392.1)	39,502.9 (15,063.6)
Median income, Hispanic or Latino households	48,211.8 (24,254.6)	38,573.2 (14,094.8)
Median income, households with children	49,408.1 (31,456.2)	35,595.6 (19,012.8)
Percent below poverty level	20.3 (12.6)	27.6 (11.9)
Percentage of women aged 20–64 in labor force	69.1 (9.5)	67.1 (7.6)
Unemployment rate for women aged 20–64	13.5 (6.0)	12.5 (4.7)
Observations	80	29

Note. ACS = American Community Survey; LAUSD = Los Angeles Unified School District; TPS = traditional public schools.

Factors Predicting Pilot School Attendance

Next, we identify how student attributes may help to predict entry into a pilot school in the ninth grade, following completion of middle school. This holds substantive importance, moving beyond descriptive differences between pupils in the two subsectors. As described above, our matching strategy has two steps: exact matching followed by propensity score matching. Since we perform propensity score matching within more than 600 distinct groups defined by a pupil’s middle school, year, ethnicity, and gender, we cannot report all the coefficients derived from our matching procedure. For illustrative purposes, we include results from the logistic regressions of attending a pilot on pupil characteristics for the entire analytic sample (online appendix Table A3). The dependent variable is equal to one if a student attends a pilot school in ninth grade, zero if a student attends a TPS.

Most predictors are highly significant, including eighth-grade math and ELA test scores, Latino ethnicity, parent education level, and the child’s FRPL eligibility (Column 1). Adding controls for middle school attended and year reduces the magnitude and significance of several effects, particularly those for ethnicity, parent education, and FRPL eligibility (Column 4).¹⁰ The mediating effect of the sending middle school likely reflects the ethnic and class segregation of schools in LAUSD. Including controls for middle school by year, better reflecting the actual matching procedure, we see that female pupils and students with higher ELA scores enter pilots with greater likelihoods than TPS peers (Column 5). But students who sat for an Algebra I exam in eighth grade (rather than in high school) were less likely to attend a pilot.

Going forward, we include a full set of baseline controls in all regressions: quadratic terms for math and ELA scores; an indicator for taking Algebra I in eighth grade; a quadratic in attendance rates; and controls for middle school attended, year, sex, ethnicity, home language, English-language proficiency, parental education, and FRPL eligibility. Our main results are robust to including middle-school-by-year fixed effects. We also obtain very similar results from a difference-in-differences model with student fixed effects.¹¹

Retention of Pilot Students in LAUSD

After matching students with almost identical likelihoods of entering a pilot in ninth grade, we turn to differences in student retention within the LAUSD district. Minimizing the exit of families from the district is a key aim of pilots, along with similar alternative schools, in large urban systems with declining enrollments due to middle-class flight or falling fertility rates (LAUSD suffers from both). The share of students who remain in any LAUSD school in the 10th grade after attending a pilot in ninth grade is 8 percentage points higher, compared with ninth graders who attended a TPS (Table 4, Column 1). After matching students on their propensity to enter a pilot, this advantage declines to 4 percentage points (Column 2), although still sizable in practical terms and significant at the 10% level.

Table 4.

Student Retention in Los Angeles Unified School District (LAUSD).

Attended LAUSD School in 10th Grade
Element	(1)	(2)	(3)	(4)	(5)
Element	Analysis Sample	Matched Sample	Matched Sample	Matched Sample	Matched Sample
Pilot high school	.080*** (.025)	.044* (.025)	.067** (.027)	.061** (.024)	.023 (.023)
Pilot × Low Eighth-Grade ELA Score			−.033* (.019)
Pilot × Low Eighth Grade Math Score				−.025 (.020)
Pilot × Algebra I Test in Eighth Grade					.040** (.019)
N	122,206	11,355	11,355	11,355	11,355
N, TPS	117,244	6,868	6,868	6,868	6,868
N, pilots	4,962	4,487	4,487	4,487	4,487
Mean of dependent variable	.755	.780	.780	.780	.780

Note. Logit marginal effects. Columns 3–5 include main effects (not shown). Standard errors clustered at the high school level are reported in parentheses. We include the following categorical variables (measured as of eighth grade) as controls: middle school, year, gender, parent education, home language, English-language proficiency, an indicator for taking Algebra I in eighth grade, free or reduced price lunch eligibility, and missing attendance. We also include the following continuous variables (measured as of eighth grade): linear and quadratic terms in math test scores, linear and quadratic terms in ELA test scores, and linear and quadratic terms in student attendance. ELA = English-language arts; TPS = traditional public schools.

*p < .10. **p < .05. ***p < .01.

Columns 3–5 add interactions with measures of student achievement in eighth grade: indicators for being in the bottom 50% of the ELA or math score distributions and for taking Algebra I in eighth grade. The retention-within-district effect is larger (more positive) for initially high-achieving students: those with higher eighth-grade ELA performance or who took Algebra I in eighth grade (Columns 3 and 5).

More research is required to understand why the holding power of pilot schools is stronger relative to TPS, which spur greater exit from LAUSD. It is encouraging that higher achieving students in eighth grade persist at higher rates after entering a pilot. These nontraditional schools may deliver on their promise of personalized attention and a richer social climate. Data made available for the present study are insufficient to inform such possible mechanisms. Survey data from students, reporting more positive feelings toward pilot teachers, do back this working hypothesis (Estrada, 2017).

Achievement Effects From Pilot Schools

Next, we turn to whether pilot students outperform TPS peers when it comes to math and ELA test scores. Table 5 displays the results for math achievement in the ninth grade taking into account eighth-grade performance and, in Columns 2–5, after matching pupils on the likelihood of entering a pilot. After matching, pilot ninth graders do considerably worse in math, .15 SD below their matched TPS peers (Column 2). Results are similar for the analysis sample (Column 1). The last row of Table 5 shows that the mean of the dependent variable is −.15, so the average ninth-grade math score for the entire matched sample is .15 SD below the average LAUSD student. Pilot students fall another one sixth of an SD below this average.

Table 5.

Estimating Math for Pilot and TPS Students.

Element	Ninth-Grade Math Score
	(1)	(2)	(3)	(4)	(5)
	Analysis Sample	Matched Sample	Matched Sample	Matched Sample	Matched Sample
Pilot high school	−.137*** (.045)	−.151*** (.051)	−.200*** (.061)	−.210*** (.060)	−.103** (.051)
Pilot × Low Eighth Grade ELA Score			.092*** (.034)
Pilot × Low Eighth Grade Math Score				.117*** (.031)
Pilot × Algebra I Test in Eighth Grade					−.072* (.039)
N	141,610	16,449	16,449	16,449	16,449
N, TPS	134,351	9,836	9,836	9,836	9,836
N, pilots	7,259	6,613	6,613	6,613	6,613
Mean of dependent variable	−.000	−.147	−.147	−.147	−.147

Note. Columns 3–5 include main effects (not shown). Standard errors clustered at the high school level are reported in parentheses. See the text or the note for Table 4 for the included control variables. TPS = traditional public schools; OLS = ordinary least squares.

*p < .10. **p < .05. ***p < .01.

Examining heterogeneity of pilot effects for differing students, we find that initially high-performing students perform worse in ninth-grade math, relative to high-performing matched TPS peers. Seen in Columns 4-6, students who fall in the upper half of the eighth-grade reading or math score distribution or took Algebra in eighth grade experience a significantly greater negative effect from attending a pilot on subsequent math performance than their peers who had lower test scores or did not take Algebra I in eight grade.

Results for ELA achievement among ninth graders attending a pilot are statistically indistinguishable from zero. As seen in the last row of Table 6, the average ELA score for the matched sample is .10 SD below the overall district mean. In the matched sample, pilot students perform only .01 SD below this mean relative to TPS peers, and the estimate is statistically insignificant (Column 2). We observe no heterogeneity in treatment effects for earlier eighth-grade achievement. In both math and ELA, we could not discern that pilot students performed at higher levels, compared with similar peers attending TPS. These achievement differences cannot be explained by the retention effects shown in Table 4, since we compare the performance of pilot and TPS students in ninth grade.

Table 6.

Estimating English-language Arts Scores for Pilot and TPS Students.

Element	Ninth-Grade ELA Score
	(1)	(2)	(3)	(4)	(5)
	Analysis Sample	Matched Sample	Matched Sample	Matched Sample	Matched Sample
Pilot high school	−.009 (.022)	−.013 (.023)	−.004 (.023)	−.015 (.023)	−.002 (.030)
Pilot × Low Eighth Grade ELA Score			−.017 (.027)
Pilot × Low Eighth Grade Math Score				.003 (.020)
Pilot × Algebra I Test in Eighth Grade					−.017 (.023)
N	142,563	16,575	16,575	16,575	16,575
N, TPS	135,263	9,923	9,923	9,923	9,923
N, pilots	7,300	6,652	6,652	6,652	6,652
Mean of dependent variable	.000	−.098	−.098	−.098	−.098

Note. Columns 3–5 include main effects (not shown). Standard errors clustered at the high school level are reported in parentheses. See the text or the note for Table 4 for the included control variables. ELA = English-language arts; TPS = traditional public schools; OLS = ordinary least squares.

*p < .10. **p < .05. ***p < .01.

Organizational Mechanisms

The lower math performance of pilot students may stem from narrower course offerings or less encouragement to pursue math in ninth grade. Remember that many pilots focus on a curricular mission related to performing arts, music, or social justice—perhaps discouraging study of mathematics. Results for ninth-grade math course taking appear in Table 7. We allow effects on course taking to differ by whether students took pre-Algebra (“Algebra readiness”) or Algebra I in eighth grade. Almost all students take at least one of these classes in eighth grade.

Table 7.

Ninth-Grade Math Courses (Matched Sample).

Element	Algebra I Class in Ninth Grade				Algebra II or Geometry Class in Ninth Grade				Algebra I Class and Algebra II or Geometry Class in Ninth Grade
Element	(1)	(2)	(3)	(4)	(5)	(6)	(7)	(8)	(9)	(10)	(11)	(12)
Pilot high school	−.021 (.035)	−.038 (.058)	−.017 (.032)	−.050* (.030)	.058* (.035)	.207*** (.066)	.040 (.030)	.055 (.036)	.013 (.008)	.029** (.012)	.006 (.007)	.008 (.008)
Pilot × Algebra I Class in Eighth Grade		.024 (.046)				−.170*** (.050)				−.023*** (.009)
Pilot × Algebra Readiness Class in Eighth Grade			−.011 (.045)				.119** (.051)				.022*** (.008)
Pilot × Low Eighth Grade Math Score				.051** (.023)				.006 (.028)				.010* (.005)
N	16,733	16,733	16,733	16,733	16,706	16,706	16,706	16,706	14,727	14,727	14,727	14,727
N, TPS	10,020	10,020	10,020	10,020	10,003	10,003	10,003	10,003	8,688	8,688	8,688	8,688
N, pilots	6,713	6,713	6,713	6,713	6,703	6,703	6,703	6,703	6,039	6,039	6,039	6,039
Mean of dependent variable	.498	.498	.498	.498	.375	.375	.375	.375	.013	.013	.013	.013

Note. Logit marginal effects. Columns 2–4, 6–8, and 10–12 include main effects (not shown). Standard errors clustered at the high school level are reported in parentheses. See the text or the note for Table 4 for the included control variables. TPS = traditional public schools.

*p < .10. **p < .05. ***p < .01.

Pilot students were less likely to take Algebra I in ninth grade; this result is restricted to students who were in the upper half of eighth-grade math CST score (Column 4). However, pilot students are significantly more likely to take advanced math courses (Algebra II or Geometry) in ninth grade, compared with matched TPS pupils (Columns 5–8).¹² This effect stems from students who took Algebra readiness in eighth grade instead of Algebra I (Column 7), suggesting that pilot students either skip Algebra I or take Algebra I simultaneously with higher level courses. Column 11 shows that pilot students who took Algebra readiness in eighth grade are more likely than TPS students to take Algebra I concurrently with Algebra II or Geometry. Just 1% of students take both Algebra I and a higher math course in ninth grade (matched sample).

These results suggest that pilots push lower achieving students to take higher levels of math than they would in a TPS. Predictably, pilot students then perform less well relative to TPS peers. Higher achieving students fare worse (above in Table 5), even though they are only slightly more likely to take higher level math than similarly prepared TPS students.

This may reveal an organizational constraint inside many pilot schools, which may be less able to offer a full range of math courses or tailor the level to a student’s prior preparation compared to TPS. One result could be mixing of low- and high-achieving pupils in the same classroom. One interpretation of our results is that this heterogeneity tends to suppress the achievement of stronger students in math. We looked at other possible mediators of this effect, finding that pilot math teachers hold less experience and less graduate training than TPS math teachers on average, though these differences are not statistically significant (Table 8, Columns 1 and 5). We do find a significantly lower probability of postgraduate training among teachers of pilot students who took Algebra I in eighth grade (Column 6), relative to teachers of similar TPS students.

Table 8.

Characteristics of Ninth Grade Math Teachers (Matched Sample).

Element	Years Teaching				Post BA
Element	(1)	(2)	(3)	(4)	(5)	(6)	(7)	(8)
Pilot high school	−.533 (1.462)	−.093 (1.388)	−.799 (1.574)	−.595 (1.630)	−.022 (0.066)	−.029 (0.075)	−.045 (0.063)	−.020 (0.067)
Pilot × Algebra I Class in Eighth Grade		−.672 (0.720)				−.077** (0.035)
Pilot × Algebra Readiness Class in Eighth Grade			.745 (0.735)				.065* (0.036)
Pilot × Low Eighth Grade Math Score				.119 (0.613)				−.003 (0.026)
N	10,357	10,357	10,357	10,357	14,500	14,500	14,500	14,500
N, TPS	6,699	6,699	6,699	6,699	8,535	8,535	8,535	8,535
N, pilots	3,658	3,658	3,658	3,658	5,965	5,965	5,965	5,965
Mean of dependent variable	10.207	10.207	10.207	10.207	0.218	0.218	0.218	0.218

Note. Logit marginal effects in Columns 1–4 and ordinary least squares coefficients in Columns 5–8. Columns 2–4 and 6–8 include main effects (not shown). Standard errors clustered at the high school level are reported in parentheses. See the text or the note for Table 4 for the included control variables.

*p < .10. **p < .05. ***p < .01.

Varying Effects Over Time and Among Schools

Next, we examine whether achievement levels differ over time or for particular student subgroups. Figure 3 presents mean test scores over time in the analysis sample and matched sample, without including controls for pupil characteristics. Pilot students, for example, outperformed TPS peers in ELA for a fleeting moment early in the movement (Figure 3, panel 3). In 2009, before the pilot movement widened, their ninth graders scored almost .10 SD higher in ELA than TPS peers in the matched sample, an advantage that diminished as more pilots came online. The picture for math is similar, yet as pilots have spread, their students have slipped further behind TPS students with similar backgrounds (Figure 3, panel 7).

Figure 3.

Pilot and traditional public schools student test scores by subsample. These graphs show averages for Grade 9 (in black, left) and the corresponding number of observations (in gray, right) in each year. Test scores are normalized within grade-year. Source: Los Angeles Unified School District data.

Finally, we found differences in the effects of individual pilot schools, both cross-sectionally and over time. To estimate these effects, we regress student test scores in ninth grade on pilot fixed effects, year fixed effects, and pilot school by year fixed effects, along with a full set of baseline student-level controls.¹³

Figure 4 displays the range of school-by-year effect estimates in each year for the matched sample. Each circle shows achievement at pilot school s in year t relative to the average performance of all matched TPS in the same year. The size of circles reflects enrollment size. The top panel, for instance, shows that in 2009 math performance at one pilot ranged over half an SD above the district average for TPS. In recent years, several pilots host students who achieve in math at more than one third an SD below the LAUSD average. Between-school variation is less for ELA achievement among pilots. Still, we find wide variation in the achievement of pilot pupils, heterogeneity similarly observed among charter schools in prior research. We also see that the decline over time in pilot performance relative to TPS (Figure 3) is due to two factors: a downward trend in the performance of early pilot schools (shown in black) and a compositional shift as lower performing pilots opened after 2009 (shown in red).

Figure 4.

School-year fixed effects analysis.

Discussion—Decentralizing School Control in Diversifying Organizational Fields

As urban educators struggle to meet the diverse values and preferences of particular communities, school organizations are experimenting with varying curricular offerings, governed in decentralized ways. The spread of charter schools places stiff pressure on district bureaucracies to innovate as well, even to mimic the smaller form and decentralized governance of charters. The accumulating literature on charter schools suggests that pupil selection into these new types of nontraditional schools will be far from random (relative to peers entering TPS) and depend on local conditions (Berends, 2015; Dauter & Fuller, 2016; Epple et al., 2015). What is not well understood is how inventive schools—especially those spawned and monitored by local districts—retain families and lift the achievement of children.

Pilot schools offer an intriguing hybrid, mimicking key features of charters while retaining teachers within unions and district personnel systems. Complementing charters, pilots fit well into the “portfolio” strategy of management, where local school boards certify and monitor site-run schools. But Boston’s experiment with pilots has shown mixed results: Despite small gains in elementary schools, pilot high schools have yielded no significant gains in math or ELA relative to TPS (Abdulkadiroğlu et al., 2011).

Against this backdrop, our study examined a larger count of pilot high schools, 29 campuses serving more high school students than found in Boston. In sharp contrast to Boston, students selecting LAUSD pilots were lower achieving and more likely to be Latino and from low-income Spanish-speaking families than TPS peers. This systematic selection pattern stemmed from how advocates of pilot schools crafted an institutional niche—separating from the downtown bureaucracy and charter proponents—and creating small campuses that might better respond to the immigrant communities in which they were originally situated. The spatial locations and institutional positioning of pilots thus influenced who entered and perhaps the achievement effects of these alternative schools.

After carefully matching “treated” and untreated TPS students, we found that pilot pupils were over 4% less likely to exit LAUSD, and this holding power was stronger for the higher achieving subset of youth from mostly poor families. Yet, pilot students gained little from attending a pilot in terms of achievement, relative to TPS students, and did worse in math by the end of ninth grade (about .15 SD lower). This average effect on math achievement varied by subgroup as well. Students who did better in the eighth grade or had completed Algebra I before entering high school displayed even lower math achievement after entering a pilot, compared with TPS counterparts.

These heterogeneous effects may stem from differing patterns of course taking. Pilots appear to assign lower achieving students to higher level math courses than does the average TPS. This may stem from the organizational constraint of smaller size and a narrow range of math teachers within pilots. Another possibility is that many pilots stress curricular themes like social justice or theatre arts, giving less attention to mathematics. After showing discernible promise in the early years of the pilot movement, the achievement advantages of a few pilot schools have dwindled, then flipped in favor of TPS peers. The between-school variation in the value-added of pilots that we discovered is reminiscent of the heterogeneity of charter-school effects and their sensitivity to local conditions revealed in the earlier literature.

These null or negative effects for pilots in LA differ markedly from the positive results reported for New York’s SHS of choice (Abdulkadiroğlu et al., 2013; Bloom et al., 2010) and for early-college high schools (Berger et al., 2013; Edmunds et al., 2017; Lauen et al., 2017). These latter models display proof-of-concept that SHS, often run independently of central bureaucracy, can outperform TPS when ensuring an orderly learning environment, rigorous personalized instruction, ongoing coaching, and technical assistance for teachers.

Our results for pilots also stand in contrast to the modest positive results earlier estimated for charter schools in LA. Whereas the charter and pilot sectors in Boston select similar groups of students—with higher initial test scores and lower probabilities of being Latino or qualifying for FRPL than TPS—pilot students in LA differ markedly from their charter counterparts in terms of prior achievement and family background. Despite these differences, charters have been found to be more effective than pilots at raising achievement in both Boston and LA. On the other hand, charter and pilot schools more effectively hold onto students and families, relative to higher exit rates for peers attending TPS. We can thus far only speculate on what organizational mechanisms advantage charters relative to pilot schools.

The present study does hold several limitations. First, we employed a quasi-experimental rather than a lottery design, necessary when waiting lists do not exist or cannot be compiled in a uniform fashion. This means that we must rely on the assumption of no hidden confounding to draw causal inferences from our estimates. This is a strong assumption. But our design (local propensity score matching with pretests of outcome) has performed well when tested through within-study comparisons on test score outcomes (Bifulco, 2012; Cook et al., 2008). The external validity of our study is limited by drawing data from a single location (LAUSD), albeit the nation’s second largest district and hosting many nontraditional schools.

Finally, the outcome measures—standardized test scores—do not capture all the elements of college and career readiness one would ideally include. Since we first observe pilot school outcomes in the 2008–2009 academic year and our data end with the 2012–2013 academic year, we do not have sufficient sample size for estimating effects on high school graduation or further educational attainment. This is especially limiting when considering the effects of specialized schools, from pilots focusing on the arts, to those centered on preparing students for health occupations. LAUSD and other urban districts are widening the range of outcome measures and their capacity to track students into postsecondary institutions.

Our findings also prompt questions on which future research might focus. Several of the earliest pilot schools, created by progressive teachers and local activists, performed well in their early years, then faded in terms of value-added benefits for students. Why the vanguard campuses would lose their spirit or technical potency remains a pivotal question. As the popularity and count of pilot schools grows, a wider variety of students are entering these campuses. Waning selectivity and roots in immigrant communities likely affects the social organization of pilots and their effects on student learning.

In-depth qualitative work, more generally, could illuminate why some pilots outperform TPS while many others do not. And do pilots—many focused on the arts, digital technology, or social justice themes—assign less instructional time to core subjects, namely math and ELA, on which state testing is centered? Pilot school principals and LAUSD designers might delve into the curricular balance that these innovative campuses offer, or fail to provide.

Overall, these findings tell a cautionary tale. District-led competitors to charters may prove effective in attracting and holding on to students from particular ethnicities and class backgrounds. But after their initial success in boosting achievement, relative to TPS peers, the proliferation of pilots in LA may have reduced average quality or the selectivity of teachers moving to pilots. Lower levels of experience and graduate training observed among pilot teachers may have undercut effectiveness, along with the narrower range of math courses. As urban districts devise more diverse forms of schooling, the magnetic attractiveness resulting from smaller size or greater personalization does not necessarily ensure stronger student learning.

Supplemental Material

Supplemental Material, Supplemental_File - Competing With Charter Schools: Selection, Retention, and Achievement in Los Angeles Pilot Schools

Supplemental Material, Supplemental_File for Competing With Charter Schools: Selection, Retention, and Achievement in Los Angeles Pilot Schools by Caitlin Kearns, Douglas Lee Lauen and Bruce Fuller in Evaluation Review

Footnotes

Authors’ Note

Ari Bennett, Rachel Bonkovsky, Delia Estrada, Rosie Martinez, Donna Muncy, and Jose Navarro taught us much about pilot schools over the years. Cristian Ugarte contributed to the data analysis. The California Endowment supported the time and efforts of Mr. Ugarte and Ms. Kearns. Our research on pilot schools was aided by Anisah Waite as well, funded by the Spencer Foundation.

Acknowledgments

Many thanks to Cynthia Lim and Kathy Hayes at the Los Angeles Unified School District for sharing data and commenting on our earlier work. Appreciation goes to Chris Walters for early advice and ideas.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research and/or authorship of this article: This work was supported by Spencer Foundation and Stuart Foundation.

ORCID iD

Bruce Fuller

Supplemental Material

Supplemental material for this article is available online.

Notes

References

Abdulkadiroğlu

Angrist

J. D.

Dynarski

S. M.

Kane

T. J.

Pathak

P. A.

(2011). Accountability and flexibility in public schools: Evidence from Boston’s charters and pilots. The Quarterly Journal of Economics, 126(2), 699–748. https://doi.org/10.1093/qje/qjr017

Abdulkadiroğlu

Pathak

P. A.

(2013). Small high schools and student achievement: Lottery-based evidence from New York City (NBER Working Paper No. 19576). National Bureau of Economic Research.

Angrist

Cohodes

Dynarski

Pathak

Walters

(2016). Stand and deliver: Effects of Boston’s charter high schools on college preparation, entry, and choice. Journal of Labor Economics, 34, 275–318.

Berends

(2015). Sociology and school choice: What we know after two decades of charter schools. Annual Review of Sociology, 41, 159–180.

Berger

Turk-Bicakci

Garet

Song

Knudson

Haxton

Zeiser

Hoshen

Ford

Stephan

Keating

Cassidy

(2013). Early college, early success: Early college high school initiative impact study. American Institutes for Research.

Betts

J. R.

Tang

Y. E.

(2016). A meta-analysis of the literature on the effect of charter schools on student achievement. Society for Research on Educational Effectiveness. https://sandera.ucsd.edu/_research-and-publications_old2/DISC%20PAPER%20Betts%20Tang%20Charter%20Lit%20Review%202018%2001.pdf

Bifulco

(2012). Can nonexperimental estimates replicate estimates based on random assignment in evaluations of school choice? A within-study comparison. Journal of Policy Analysis and Management, 31(3), 729–751.

Bloom

H. S.

Thompson

Unterman

(2010). Transforming the high school experience: How New York City’s new small schools are boosting student achievement and graduation rates. MDRC. https://eric.ed.gov/?id=ED511106

Bloom

H. S.

Unterman

(2014). Can small high schools of choice improve educational prospects for disadvantaged students? Journal of Policy Analysis and Management, 33, 290–319. https://doi.org/10.1002/pam.21748

10.

Bryk

A. S.

Sebring

P. B.

Allensworth

Luppescu

Easton

J. Q.

(2010). Organizing schools for improvement: Lessons from Chicago. University of Chicago Press.

11.

Chambers

J. G.

Levin

J. D.

Shambaugh

(2010). Exploring weighted student formulas as a policy for improving equity for distributing resources to schools: A case study of two California school districts. Economics of Education Review, 29(2), 283–300.

12.

Chubb

J. E.

Moe

T. M.

(1990). America’s public schools: Choice is a panacea. The Brookings Review, 8(3), 4–12.

13.

Clair

T. S.

Cook

T. D.

Hallberg

(2014). Examining the internal validity and statistical precision of the comparative interrupted time series design by comparison with a randomized experiment. American Journal of Evaluation. https://doi.org/10.1177/1098214014527337

14.

Cook

T. D.

Shadish

W. R.

Wong

V. C.

(2008). Three conditions under which experiments and observational studies produce comparable causal estimates: New findings from within-study comparisons. Journal of Policy Analysis and Management, 27(4), 724–750. https://doi.org/10.1002/pam.20375

15.

Dauter

Fuller

(2016). Student movement in social context: The influence of time, peers, and place. American Educational Research Journal, 53(1), 33–70.

16.

Edmunds

J. A.

Unlu

Glennie

Bernstein

Fesler

Furey

Arshavsky

(2017). Smoothing the transition to postsecondary education: The impact of the early college model. Journal of Research on Educational Effectiveness, 10(2), 297–325.

17.

Epple

Romano

Zimmer

(2015). Charter schools: A survey of research on their characteristics and effectiveness (Working Paper 21256). National Bureau of Economic Research.

18.

Estrada

(2017). Utilizing the lens of cultural proficiency to judge the impact of autonomous schools from the student perspective [Doctoral dissertation]. Claremont Graduate University.

19.

Fliegel

MacGuire

(1993). Miracle in East Harlem. Random House.

20.

Fortson

Verbitsky-Savitz

Kopa

Gleason

(2012). Using an experimental evaluation of charter schools to test whether nonexperimental comparison group methods can replicate experimental impact estimates (NCEE 2012–4019). National Center for Education Evaluation and Regional Assistance. http://eric.ed.gov/?id=ED531481

21.

Fuller

(2015). Organizing locally: How the new decentralists improve education, health care, and trade. University of Chicago Press.

22.

Furgeson

Gill

Haimson

Killewald

McCullough

Nichols-Barrer

Teh

Verbitsky-Savitz

Bowen

Demeritt

Hill

Lake

(2012). Charter-school management organizations: Diverse strategies and diverse student impacts. Mathematica Policy Research. http://eric.ed.gov/?id=ED528536

23.

Goldring

Gray

Bitterman

(2013). Characteristics of public and private elementary and secondary school teachers in the U.S.: Results from the 2011–12 schools and staffing survey. National Center for Education Statistics.

24.

Jacob

A. M.

Wolf

P. J.

(2012). Milwaukee longitudinal school choice evaluation: Annual school testing summary report 2010–11. https://scholarworks.uark.edu/cgi/viewcontent.cgi?article=1073&context=scdp

25.

Kerchner

C. T.

Menefee-Libey

D. J.

Mulfinger

L. S.

Clayton

S. E.

(2008). Learning from L.A.: Institutional change in American public education. Harvard Education Press.

26.

Kohl

Farris-Berg

(2014, November 13). Teacher-powered pilot schools face obstacles [Blog]. http://blogs.edweek.org/edweek/on_california/2014/11/teacher-powered_pilot_schools_face_obstacles.html

27.

Lauen

D. L.

(2009). To choose or not to choose: High school choice and graduation in Chicago. Educational Evaluation and Policy Analysis, 31(3), 179–199.

28.

Lauen

D. L.

Barrett

Fuller

S. F.

Janda

(2017). Early colleges at scale: Impacts on secondary and postsecondary outcomes. American Journal of Education, 123(4), 523–551.

29.

Lauen

D. L.

Fuller

Dauter

(2015). Positioning charter schools in Los Angeles: Diversity of form and homogeneity of effects. American Journal of Education, 121(2), 213–239.

30.

Los Angeles Unified School District. (2016). Los Angeles pilot schools manual. Office of School Choice. https://achieve.lausd.net/cms/lib/CA01000043/Centricity/Domain/262/Pilot%20Manual%2016-17_v3_02.03.17.pdf

31.

Malen

Rice

J. K.

Matlach

L. K.

Bowsher

Hoyer

K. M.

Hyde

L. H.

(2015). Developing organizational capacity for implementing complex education reform initiatives: Insights from a multiyear study of a teacher incentive fund program. Educational Administration Quarterly, 51(1), 133–176.

32.

Martinez

R. A.

Quartz

K. H.

(2012). Zoned for change: A historical case study of the Belmont zone of choice. Teachers College Record, 114(10), n10.

33.

Nesoff

(2007). The Belmont zone of choice: Community-driven action for school change. Horace, 23(4), n4.

34.

Nichols-Barrer

Gleason

Gill

Tuttle

(2016). Student selection, attrition, and replacement in KIPP middle schools. Educational Evaluation & Policy Analysis, 38, 5–20.

35.

O’Day

J. A.

Bitter

C. S.

Gomez

L. M.

(2011). Education reform in New York City: Ambitious change in the nation’s most complex school system. Harvard Education Press.

36.

Purkey

S. C.

Smith

M. S.

(1985). School reform: The district policy implications of the effective Schools literature. The Elementary School Journal, 85(3), 353–389.

37.

Ravitch

(1974). The great school wars, New York City, 1805–1973: A history of the public schools as a battlefield of social change. Basic Books.

38.

Raymond

(2014). Charter school performance in Los Angeles. Stanford University. http://credo.stanford.edu

39.

Reynolds

Teddlie

(2000). The Processes of school effectiveness. In David

Teddlie

Reynolds

(Eds.), The International handbook of school effectiveness research (pp. 134–159). Psychology Press.

40.

Rich

(2014, January 27). Bill to offer an option to give vouchers.New York Times. http://www.nytimes.com/2014/01/28/education/senator-to-propose-schoolvouchers-program.html?_r=0

41.

Scott

(2013). A Rosa Parks moment? School choice and the marketization of civil rights. Critical Studies in Education, 54(1), 5–18.

42.

Shin

H. J.

Fuller

Dauter

(2016). Heterogeneous effects of charter schools: Unpacking family selection and achievement growth in Los Angeles. Journal of School Choice, 11(1), 60–94.

43.

Snyder

T. D.

Dillow

S. A.

(2012). Digest of education statistics, 2011 (NCES 2012–001). National Center for Education Statistics.

44.

Steiner

P. M.

Cook

T. D.

Shadish

W. R.

(2011). On the importance of reliable covariate measurement in selection bias adjustments using propensity scores. Journal of Educational and Behavioral Statistics, 36(2), 213–236. https://doi.org/10.3102/1076998610375835

45.

Strunk

Marsh

Hashim

Bush-Mecenas

Weinstein

(2016). The impact of turnaround reform on student outcomes: Evidence and insights from the Los Angeles Unified School District. Education Finance and Policy, 11, 251–282.

46.

Tuttle

C. C.

Gill

Gleason

Knechtel

Nichols-Barrer

Resch

(2013). KIPP middle schools: Impacts on achievement and other outcomes. Final Report. Mathematica Policy Research. http://eric.ed.gov/?id=ED540912

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.03 MB