The Efficacy and Implementation of Michigan’s Partnership Model of School and District Turnaround: Mixed-Methods Evidence From the First 2 Years of Reform Implementation

Effects of Prior Reforms

The shift in school turnaround and accountability under ESSA comes on the heels of a string of reforms that produced positive average effects, but with substantial heterogeneity across contexts (Redding & Nguyen, 2020; Schueler et al., 2021). Here, we discuss prior reforms beginning with NCLB and culminating with ESEA waivers. There is some evidence that NCLB interventions improved student achievement and closed achievement gaps (Dee & Jacob, 2011), although effects were uneven across grade levels, subject areas, and state contexts (Dee & Jacob, 2011; Ladd, 2017; Lee & Reeves, 2012; Strunk et al., 2014; Strunk & McEachin, 2014). However, NCLB’s emphasis on proficiency rates also created perverse incentives for schools to focus on the so-called “bubble” students to the detriment of students throughout the achievement distribution or to push out students expected to score below proficient. In particular, improvements in student achievement and decreased achievement gaps were concentrated among students near the cusp of the test proficiency threshold—suggesting districts may have triaged supports to the students most likely to increase proficiency rates (Balfanz et al., 2007; Booher-Jennings, 2005; Cullen & Reback, 2006; Darling-Hammond, 2006; Ho, 2008; Jacob, 2005; Jennings & Bearak, 2014; Krieg, 2008; Reback, 2008; Strunk et al., 2014).

NCLB interventions gave way to RTTT and SIG whole-school reform interventions, which introduced school and district turnaround to the federal policy parlance around low-performing schools. Unlike prior iterations of whole-school reform such as those taking place under NCLB (and previously the Comprehensive School Reform program), turnaround aimed to impel rapid and dramatic changes in low-performing schools (Aladjem et al., 2010; Herman et al., 2008; Peurach & Neumerski, 2015). In addition, federal turnaround policy prescribed specific strategies for states to follow—providing considerably less discretion to state and local education leaders than prior federal policy. Turnaround schools in Massachusetts, California, Ohio, and Tennessee made student achievement gains under RTTT and SIG (Carlson & Lavertu, 2018; LiCalsi et al., 2015; LiCalsi & Píriz, 2016; Papay & Hannon, 2018; Sun et al., 2017; Zimmer et al., 2017). Some reforms also produced positive effects on proximate outcomes such as increased teacher retention and quality, and reduced student absenteeism (Papay & Hannon, 2018; Pham et al., 2020; Sun et al., 2017). Across multiple states, the positive effects of SIG were largely sustained after the reforms concluded (Sun et al., 2020).

However, positive average treatment effects mask heterogeneity both across and within interventions. Turnaround in North Carolina, Tennessee, and Texas produced mixed effects (Dickey-Griffith, 2013; Heissel & Ladd, 2018; Henry & Guthrie, 2019; Zimmer et al., 2017), and some of the interventions yielding positive overall effects also produced null or negative effects in particular contexts or cohorts (Carlson & Lavertu, 2018; Strunk, Marsh, Hashim, et al., 2016; Zimmer et al., 2017). The heterogeneity of effects highlights variation in both the features and implementation of state turnaround interventions. Even under SIG, which prescribed specific turnaround models and practices, there was variation in the practices states chose to implement (Dragoset et al., 2015).

As RTTT and SIG grants concluded, states replaced their tightly mandated turnaround interventions with more flexible policies under ESEA waivers, which required states to identify and support their low-performing schools but—like ESSA—did not mandate how states should do so. These waiver-based reforms yielded few positive effects, although the reform models themselves and their implementation again varied widely across states. Of the states with evaluations of waiver reforms, one—Kentucky—produced positive effects on student achievement, which the authors attributed in part to clearly articulated state guidance for the comprehensive school improvement planning process (Bonilla & Dee, 2020). Waiver-based reforms in New York, Michigan, Rhode Island, and Louisiana produced either null or negative effects on student achievement (Atchison, 2020; Dee & Dizon-Ross, 2019; Dougherty & Weiner, 2017; Hemelt & Jacob, 2017, 2018). A similar state-initiated reform in North Carolina that focused on tailoring turnaround services to school needs also led to negative effects on student achievement, early grade outcomes, and teacher retention (Henry & Harbatkin, 2020; Henry et al., 2022). By contrast, an Ohio reform that involved supporting district and school leaders in developing comprehensive plans produced positive effects on school-level student achievement (Player & Katz, 2016). In Washington state, strategies delineated in school improvement plans under SIG and ESEA waivers predicted improvements in student achievement and absenteeism (Sun et al., 2019). These findings highlight the importance of the comprehensive planning process in more loosely mandated turnaround reforms like ESEA waivers and the new wave of reforms under ESSA.

As in other states, Michigan’s reforms under these earlier federal policies produced mixed effects. Students in Priority schools, which comprised the bottom 5% of schools under the state’s ESEA waiver reform, fared worse than their counterparts in other low-performing schools in math and reading in both the first and second years of the intervention (Hemelt & Jacob, 2017). In Focus schools (schools with the widest achievement gaps between high- and low-achieving students), a short-run decrease in the math achievement gap was driven by stagnant performance among high-achieving students rather than growth among lower achievers. There were no effects on the reading achievement gap or math or reading performance (Hemelt & Jacob, 2018).

School Turnaround Under ESSA

It is unclear how evidence stemming from prior reforms will generalize to ESSA turnaround interventions. Although there were some positive effects of prior turnaround efforts, several studies suggest schools that were subject to more disruptive reform models under RTTT and SIG fared better than schools under less disruptive models (Carlson & Lavertu, 2018; Dee, 2012; Strunk et al., 2014; Sun et al., 2017). It is thus far unclear whether and to what extent a flexible and tailored turnaround model in combination with a more nuanced view of school performance will translate into improved student outcomes. As no peer-reviewed studies have yet examined school turnaround policies under ESSA, little is known about whether these new features are conducive to improved student outcomes.

However, prior literature points to four mechanisms through which less disruptive ESSA reforms like the Partnership Model could impact student achievement: the comprehensive planning process, external supports, the threat of accountability, and improvements to human capital. First, the quality of comprehensive plans can vary, and their content plays an important role in reform implementation (Anfara et al., 2006; Mintrop & MacLellan, 2002; Redding & Searby, 2020; Strunk, Marsh, Bush-Mecenas, & Duque, 2016). This is important because a comprehensive planning process that generates clearly articulated plans has the potential to induce meaningful improvements in low-performing schools and districts—while a pro forma approach may not (Bonilla & Dee, 2020; Huber & Conway, 2015; Player & Katz, 2016; Strunk, Marsh, Hashim, et al., 2016; Sun et al., 2019).

Second, the scope and quality of external supports and resources made available to turnaround schools and districts matter. Schools may be low-performing because they lack the resources needed to build the systems that undergird improvement (Karcher & Knight, 2021). Although early attempts at school reform focused largely on accountability, RTTT and SIG paired accountability with an influx of federal funding for states to support low-performing schools. Positive effects under RTTT and SIG were unique in both their prevalence and magnitude, suggesting that these new resources may have played an important role (see Carlson & Lavertu, 2018; LiCalsi et al., 2015; LiCalsi & Píriz, 2016; Papay & Hannon, 2018). Third, there is evidence from the NCLB era that the possibility of high-stakes consequences for failing to make reasonable progress can induce improvement (Carnoy & Loeb, 2002; Chiang, 2009; Dee & Jacob, 2011; Hanushek & Raymond, 2005; Jacob, 2005). Under NCLB in particular, the threat of sanctions appeared to increase student achievement in some contexts (Reback et al., 2014; Springer, 2008).

Finally, an abundance of research highlights the importance of human capital to school effectiveness and improvement. A stable and highly effective teacher and principal workforce is critical to successful turnaround (Harbatkin, 2022; Henry et al., 2020; Pham et al., 2020; Pham, 2022). When teachers and principals do turn over, schools need a robust local educator labor market from which to draw replacements (Malen & Rice, 2016; Strunk, Marsh, Hashim, et al., 2016). Principals in particular are responsible for building the framework for school improvement, for example, by setting school climate and culture, coalescing staff around a shared mission, and acting as instructional leaders (Burkhauser, 2017; Duke, 2004; Duke & Salmonowicz, 2010; Hitt et al., 2018; Leithwood & Jantzi, 1990; Weiner, 2016; Woulfin & Weiner, 2019).

Although prior research points to these mechanisms as important for successful school improvement under earlier reforms, there is thus far no evidence on how they may contribute to turnaround under ESSA. This article adds to the turnaround literature in three ways. First, we examine implementation in addition to impact to unpack not just whether Partnership turnaround affected school outcomes but also how it affected them. There are, to date, few studies that combine both impact and implementation analyses (see Henry & Harbatkin, 2020; Papay et al., 2021; Strunk, Marsh, Hashim, et al., 2016, for notable exceptions). Second, we provide early evidence on the implementation and effects of a generalizable school turnaround reform under ESSA. Although its implementation began a year before states were required to begin intervening in low-performing schools under ESSA, the model is the state’s intervention under ESSA, meets ESSA requirements for identifying and supporting low-performing schools, and is similar to ESSA turnaround models throughout the country in its focus on customizing supports to school and district-specific needs. Thus, results are likely generalizable to ESSA turnaround reforms across the country. Finally, we examine the effects of ESSA turnaround on low-achieving students, who tended not to improve under prior accountability interventions that classified schools using proficiency rates. Although previous research has estimated the effects of accountability interventions on the lowest achieving students, this study builds on prior work by examining the effects of a turnaround reform that defines progress using a multidimensional measure of school effectiveness rather than a singular focus on proficiency rates.

Administrative Data

Sample

The event study draws from statewide administrative data on students and teachers from 2013–2014 through 2018–2019 provided by MDE and the Center for Educational Performance and Information (CEPI). This includes approximately 9 million student-year observations representing 2,244,145 students. To examine the impact of Partnership on student outcomes, we limit the student sample to just those who were enrolled in a Partnership school during these years and identify students in Partnership schools as the treated group and students enrolled in near-selected schools for each cohort as the comparison group. This analytic sample contains 971,656 student-year observations representing 306,307 unique students. Our main models further restrict this sample by identifying students as being either in the treatment or in the comparison group based only on the school they attended in the year of Partnership identification (see Table 1 for counts of students and schools by treatment condition in the identification year). By classifying students as treatment or comparison based on their school assignment at the time of Partnership identification, we are estimating an “intent-to-treat” (ITT) effect of Partnership based on school assignment prior to the intervention. This time-invariant treatment assignment provides the most conservative effect estimates by avoiding selection bias from families choosing to transfer students in or out of Partnership schools in response to the intervention. ³

OPEN IN VIEWER

Table 1

Number of Unique Schools and Students by Cohort in Their Identification Year

	Treatment		Comparison
	Schools	ITT students	Schools	ITT students
Cohort 1 (Round 1)	37	16,424	80	35,156
Cohort 2 (Rounds 2 and 3)	82	33,327	156	57,786

Note. ITT = intent-to-treat.

Thus far, there have been three rounds of Partnership identification, and the analytic sample therefore contains three rounds of treated schools. We provide detail on how each round of Partnership schools was selected in Supplemental Appendix B (in the online version of the journal). Because the state identified Rounds 2 and 3 in the same school year and treated them on the same implementation and evaluation schedules, we combine them into a single treatment group we call Cohort 2. We therefore separately examine each cohort of schools—Cohort 1 (comprising 37 Round 1 schools) and Cohort 2 (comprising 39 Round 2 and 43 Round 3 schools for a total of 82 schools). Because Round 2 and 3 schools were identified using different metrics and 6 months apart, we also conduct supplemental analyses to unpack differences in effects between these two rounds.

For each cohort, we construct a comparison group of near-selected low-performing schools that were never identified for Partnership. The Cohort 1 comparison group comprises the 80 Priority schools from the 2015–2016 school year that were not selected as Partnership schools in Round 1, 2, or 3. The Cohort 2 comparison group includes the 156 schools in the bottom 10% on Michigan’s ESSA index system in 2017–2018 that were not identified for treatment in any round of Partnership. ⁴

Treated schools in both Cohort 1 and Cohort 2 are, on average, lower performing than the schools in their respective comparison groups, which is not surprising given that schools were selected for Partnership on the basis of low performance. Supplemental Appendix Table A-2 (in the online version of the journal) shows that treated and comparison schools have similar shares of students who are economically disadvantaged and that treated schools have larger shares of students who are Black, slightly larger shares of students who receive special education services, and lower shares of students who are White, Hispanic or Latinx, and English Learners. The final two columns of Supplemental Table A-2 (in the online version of the journal) make clear that comparison schools are more observably similar to treated schools than other schools throughout the state. As we describe later, the causal interpretation of the estimates in our event study does not rely on baseline equivalence but rather on parallel pretreatment trajectories on the outcome and evidence that there was no anticipatory response to treatment assignment. Collectively, data used for analyses of Cohort 1 include 414,456 student-year observations (149,135 unique students). Cohort 2 data include 689,407 student-year observations (230,240 unique students). ⁵

Although the decision to assign treatment status based on students’ location in the identification year most cleanly addresses selection issues that are endogenous to Partnership identification, the estimates from these models may be attenuated for two reasons. First, students in a Partnership school during the identification year could, during implementation, move to either a comparison school in our sample or a non-comparison school that we do not include in our sample. Similarly, students in comparison schools during the identification year may transfer to Partnership schools (or to non-comparison, nontreated schools) after identification. To that end, not all students classified as treated would have received the full dose—or in some cases, any dose—of treatment, and some students classified as comparison would have received some dose of treatment. Second, some schools in the comparison group may receive some element of the treatment, being that Partnership is implemented through districts with a particular focus on targeted schools within them. To the extent that Partnership contributes to improved district-level systems and functioning, comparison schools in Partnership districts may improve alongside treated schools.

We estimate models on three additional populations to examine the extent to which each of these potential sources of attenuation bias may contribute to our preferred estimates. First, we employ Sun et al.’s (2017) “all starters” approach by assigning students to treatment or comparison based on the school they attended during the first implementation year rather than the identification year. For Cohort 1, this approach creates a sample based on school assignment in the 2017–2018 school year and follows the same cohort of students into the next year. For Cohort 2, where we have only 1 year of outcome data, this approach produces a treatment-on-the-treated (TOT) estimate because it estimates the effect of treatment based on a single year of implementation. ⁶ To the extent that our preferred estimates are attenuated by student mobility, the all starters models will find larger effects.

Second, we return to the original student assignment approach (i.e., treatment assignment based on school in the identification year) but drop non-Partnership schools in Partnership districts from our comparison group. This approach drops 16,515 and 22,943 students in 29 and 42 schools from the Cohort 1 and 2 comparison groups, respectively. To the extent that the Partnership Model improved district-level systems that led to increases in student achievement for students in low-performing, non-Partnership schools in Partnership districts, these secondary effects would cause estimated effects of the intervention to be larger than in our preferred models. Thus, both specification tests should confirm that our preferred approach returns the most conservative estimate of Partnership Model impact.

Finally, we estimate models in which we move students in non-Partnership schools in Partnership districts (i.e., the students we dropped out in the prior specification) from the comparison group to the treated group to more directly estimate the effects of Partnership on untreated low-performing schools in Partnership districts. This approach allows us to estimate the effect of Partnership on students in all low-performing schools in Partnership districts, including schools identified for Partnership and students in near-selected schools.

Estimation Strategy

We estimate event study models separately for the two cohorts of treated and comparison schools. The model for student outcomes (i.e., Grade 4–8 test scores) in Cohort 1 takes the form

y i g s t = α 0 + ∑ r = − 3 2 I 2016 + r + ∑ r = − 3 2 I 2016 + r × P a r t n e r s h i p i s t + X i s t θ + δ g + ψ i + ε i g s t .

estimating outcome y (i.e., test score level or test score gains) for student i in grade g in school s in year t. I_2016+r represents a series of year indicators from 2013–2014 through 2018–2019, with 2016–2017 (the year of Partnership identification for Cohort 1) as the omitted reference year. We interact these year indicators with a binary indicator of treatment status, Partnership_ist, that takes a value of 1 for students who were assigned to Partnership schools at the time of identification. The coefficients on these interactions provide the estimated difference in outcomes between students in Partnership and comparison schools relative to the omitted 2016–2017 school year. The interactions for 2017–2018 and 2018–2019 provide the estimated effects of the Partnership reform for Cohort 1 in the first 2 years of implementation. X _ist is a vector of time-varying student and school covariates. Student covariates include indicators denoting whether the student was economically disadvantaged, ⁸ an English learner, and had a special education designation. School-level covariates include the school-level proportions of each race/ethnicity, economically disadvantaged, English learner, and special education, as well as logged school enrollment. We also include grade-level fixed effects ( δ g ) with fourth grade as the reference category for test score outcomes ⁹ and student fixed effects ( ψ i ), which allow us to estimate each student’s shift from their own pretreatment performance. ε is an idiosyncratic error term clustered at the school level. For fourth- through eighth-grade achievement, we estimate two separate models—one predicting the effect of Partnership on achievement levels and one predicting the effect on one-year achievement gains. We interpret these estimates as bounding the true effect of Partnership (Clotfelter et al., 2007). The model predicting student outcomes for Cohort 2 follows the same format as the model for Cohort 1 except it omits 2017–2018 (the Cohort 2 identification year) as the reference year and therefore provides only 1 year of effect estimates. In this case, the I_2016+r indicators become I_2017+r and r indexes to 1 instead of 2. ¹⁰

To estimate heterogeneous effects by lagged achievement, we estimate separate models for low- and high-achievers based on their test scores in a particular subject in the identification year. Estimates of school-level outcomes (i.e., SAT scores, graduation rate, dropout rate) largely follow the model above, but exclude student covariates and substitute school fixed effects for student fixed effects. The school-level models use treatment assignment based on the school’s status (treatment or comparison) in the relevant identification year. In other words, a school identified for Partnership in year t would be assigned to the treatment group and that assignment retained throughout the panel, and the graduation rate or other school-level outcome observed for that school in year t + 1 is based on all students in that school in that year regardless of the school those students attended in year t.

There are two key identifying assumptions in our event study design. The first is that in the absence of Partnership identification, outcomes in Partnership schools would have followed the same pattern as those observed in our comparison schools, conditional on covariates. We evaluate this assumption by interpreting the coefficients and corresponding p-values for ∑ r = − 3 2 I 2016 + r × P a r t n e r s h i p i s t terms in the years leading up to Partnership identification. Here, coefficients that are small and not statistically significant suggest that outcomes in Partnership and comparison schools were similar leading up to Partnership identification (see, for example, the rows of Table 2 that correspond to the years prior to Partnership identification, indicated by “ref.”). The second is the conditional no anticipation assumption, which assumes that students in treated and comparison groups did not respond differentially in anticipation of the treatment, again conditional on covariates. To address concerns related to this assumption, we also estimate models with the year prior to identification as the excluded reference year. Again, we do not observe significant differences between treated and comparison schools in the pretreatment years (these estimates are provided in Supplemental Appendix Table A-7 in the online version of the journal). These models can provide further evidence that the estimates are not being driven by mean regression following significant identification year dips.

OPEN IN VIEWER

Table 2

Partnership Effects on Student Outcomes

	(1)	(2)	(3)	(4)	(5)	(6)	(7)	(8)
	M-STEP Math Levels	M-STEP MathGains	M-STEP ELALevels	M-STEP ELAGains	SATMath	SATELA	Grad rate	Dropout rate
Panel A. Cohort 1
Partnership School 2013–2014	0.070 (0.050)	0.053 (0.047)	0.063 (0.053)	0.020 (0.046)	0.044 (0.040)	−0.001 (0.035)	0.045 (0.032)	0.028 (0.031)
Partnership School 2014–2015	0.020 (0.048)	−0.023 (0.046)	0.004 (0.050)	−0.055 (0.043)	0.117* (0.057)	0.020 (0.045)	0.061 (0.039)	−0.014 (0.028)
Partnership School 2015–2016	0.037 (0.033)	0.027 (0.037)	0.032 (0.035)	0.058 (0.052)	−0.023 (0.052)	0.060 (0.053)	−0.001 (0.032)	0.034 (0.021)
Partnership School 2016–2017	ref.	ref.	ref.	ref.	ref.	ref.	ref.	ref.
Partnership School 2017–2018	0.047 (0.036)	0.121* (0.054)	0.066* (0.030)	0.112* (0.045)	0.000 (0.057)	0.038 (0.047)	0.055 (0.039)	−0.021 (0.038)
Partnership School 2018–2019	0.071 (0.048)	0.075 (0.049)	0.103* (0.040)	0.075 † (0.042)	−0.002 (0.058)	0.064 (0.050)	0.058 (0.043)	−0.009 (0.036)
School covariates	X	X	X	X	X	X	X	X
Student covariates	X	X	X	X
Grade FE	X	X	X	X
N	81,830	81,830	81,990	81,990	211	211	215	215
Adjusted R2	0.678	−0.223	0.706	−0.214	0.803	0.870	0.884	0.784
Within R2	0.026	0.020	0.027	0.017	0.254	0.078	0.218	0.087
Panel B. Cohort 2
Partnership School 2013–2014	0.042(0.047)	0.043(0.043)	0.008(0.045)	0.004(0.042)	0.039(0.052)	−0.047(0.051)	0.001(0.034)	−0.002(0.030)
Partnership School 2014–2015	−0.025(0.047)	−0.067(0.043)	−0.047(0.046)	−0.074(0.048)	0.055(0.049)	−0.033(0.050)	−0.017(0.033)	0.004(0.024)
Partnership School 2015–2016	−0.008(0.035)	0.023(0.035)	−0.009(0.039)	0.046(0.031)	0.048(0.041)	0.074(0.045)	0.005(0.023)	0.000(0.025)
Partnership School 2016– 2017	0.029(0.028)	0.049(0.035)	0.025(0.031)	0.040(0.037)	0.033(0.049)	0.055(0.047)	0.010(0.023)	−0.005(0.021)
Partnership School 2017–2018	ref.	ref.	ref.	ref.	ref.	ref.	ref.	ref.
Partnership School 2018–2019	0.034 † (0.020)	0.055(0.036)	0.018(0.016)	0.035(0.038)	−0.050(0.055)	−0.072(0.052)	−0.016(0.031)	0.005(0.033)
School covariates	X	X	X	X	X	X	X	X
Student covariates	X	X	X	X
Grade FE	X	X	X	X
N	154,866	154,866	155,015	155,015	496	496	505	505
Adjusted R2	0.642	−0.216	0.667	−0.212	0.636	0.709	0.899	0.750
Within R2	0.016	0.014	0.016	0.010	0.205	0.052	0.143	0.080

Note. Student-level models (i.e., Columns 1–4) include student fixed effects and time-invariant treatment assignment based on school assignment in the identification year; robust standard errors clustered at the time-invariant school level in parentheses. School-level models (i.e., Columns 5–8) include school fixed effects and time-variant treatment assignment; robust standard errors clustered at the time-variant school level in parentheses. SAT ELA = the evidence-based reading and writing section of this assessment; M-STEP = Michigan Student Test of Educational Progress; ELA = English language arts; FE = fixed effects.

†

p < .10. *p < .05. **p < .01. ***p < .001.

We conduct several additional analyses to examine the robustness of our results. First, estimates from the two-way fixed-effects approach can be sensitive to violations of strict exogeneity of covariates (Baker et al., 2022; Wooldridge, 2021). We therefore re-estimate our models using covariates measured at baseline, interacted with a linear year trend, rather than time-varying covariates. Second, while our estimates will not be biased by staggered intervention timing, our main analyses do combine two identification rounds (Rounds 2 and 3) into a single implementation cohort (Cohort 2) because they are implemented on the same timeline. Due to differences in how these schools were identified, there may be concerns that a common treatment effect could obscure meaningful differences in outcomes. We therefore re-estimate our Cohort 2 models allowing for flexible effects in Rounds 2 and 3.

Three additional analyses are focused on the district-level nature of the intervention. Specifically, while treatment is assigned at the school level, it is implemented through the district and could therefore be considered as a district-level treatment. First, we estimate three different models to assess the extent to which the district-level nature of the treatment may be affecting (a) outcomes in low-performing non-Partnership schools in Partnership districts, (b) estimates for treated schools, and (c) inference on the coefficient estimates in our main models. To estimate the effects of the Partnership Model on low-performing non-Partnership schools in Partnership districts, we run a parallel set of event studies in which we move all students in non-Partnership schools in Partnership districts from the comparison to treated group. Specifically, we count a student as being treated if they are in a near-selected school in a Partnership district, regardless of that school’s Partnership status. In the post-treatment years, we then estimate two separate treatments—one for Partnership schools as in our main models and another for near-selected schools in Partnership districts. These models provide the estimated effects of being assigned to a Partnership school and being assigned to another low-performing school in a Partnership district. If Partnership improved district processes and systems or allocated resources across Partnership and other low-performing schools, these estimates would be positive. Second, we estimate a set of models that drop non-Partnership schools in Partnership districts from the comparison group. To the extent that Partnership affects outcomes in these schools, excluding them from the comparison creates a cleaner counterfactual. Finally, to assess the robustness of our inferences, we re-estimate our main models with standard errors clustered at the district level.

Survey Data and Analysis

To understand the elements of Partnership that made it more or less effective, we draw on surveys of teachers in Partnership districts and interviews with Partnership district leaders. Analyses of survey data provide a window into how Partnership impacted classrooms as teachers worked to implement their district’s Partnership Agreement. We surveyed all teachers in Partnership districts (in Partnership and non-Partnership schools) in fall 2018 and fall 2019. The survey garnered response rates of 38% (N = 2,718) in 2018 and 49% (N = 3,324) in 2019. Panel A of Supplemental Appendix Table A-3 (in the online version of the journal) shows response rates by cohort, year, and treatment status. As expected, response rates were higher for educators in Partnership schools. Panel B compares the characteristics of teachers who did and did not participate in this survey, by Partnership status. In Partnership schools, Black and male teachers responded at a slightly lower rate and White teachers responded at a slightly higher rate. In non-Partnership schools, male teachers responded at a lower rate and elementary-certified teachers responded at a slightly higher rate. These differences are descriptively small and fall below the threshold for baseline equivalence of 0.25 standard deviation units recommended by the What Works Clearinghouse (2022). However, we repeat our analyses using inverse probability weights generated using teacher demographics, certification, and school fixed effects.

Survey items focused on a range of constructs related to Partnership implementation and teachers’ perceptions of conditions within their school. In this study, we focus on constructs related to the mechanisms of improvement outlined in the Partnership Model’s theory of change. To generate constructs to measure implementation, we conducted factor analyses within sets of items from the larger instrument that had been asked in both years of survey administration. We conducted separate factor analyses on all items within a single construct by first determining the number of factors using parallel analysis (Horn, 1965) and then creating principal components factors. Where the parallel analysis pointed to multiple factors within a set of items, we applied a varimax rotation to identify orthogonal factors, removing items that did not fit statistically with other items in the factor. This yielded three factors: (a) Staffing, teachers’ perceptions of teacher and staff retention and teacher attendance; (b) School accountability, teachers’ perceptions that consequences will follow if their school fails to meet its improvement goals; and (c) Perception of school leadership, the extent to which teachers found their school leader to be effective at communicating school mission, strategies, and goals; working with staff to meet curriculum standards; engaging family and community members; and establishing discipline policies. Scale reliability coefficients range from .822 (school accountability) to .957 (school leadership). Supplemental Appendix Table A-4 (in the online version of the journal) provides descriptive statistics for the items used to create these factors, along with the text of items and their response scales. Factor loadings are presented in Supplemental Appendix Table A-5 (in the online version of the journal). We conduct t-tests on these factor scores to compare the perceptions of teachers in Partnership schools with teachers in non-Partnership schools within Partnership districts. We focus on the differences between these sets of schools to help us understand what Partnership districts may be doing differently in their Partnership schools relative to their other schools that are not specifically identified for turnaround.

Qualitative Data

A key characteristic of the Partnership Model is that it provided districts with significant flexibility around the improvement goals set for Partnership schools and the strategies for achieving those goals. In addition, the Partnership Model provided a range of supports from which districts could draw to improve their identified schools. This flexibility allowed variation in how Partnership was implemented in practice across districts that could not be captured with administrative data or surveys. To understand how districts approached the development and implementation of their Partnership Agreements, we recruited leaders in all Partnership districts/charters who were primarily responsible for the creation or implementation of their Partnership Agreement. We interviewed Partnership district leaders in 2018–2019 (n = 22/36 Partnership districts, or 61%) and in 2019–2020 (n = 22/29 Partnership districts, or 76%). In 2018–2019, we conducted these interviews when Cohort 1 schools were generally near their 18-month interim evaluation and in 2019–2020 when Cohort 2 schools were at the midpoint of implementing their Partnership Agreements.

In these 60-minute semi-structured interviews, we sought to understand how leaders perceived the Partnership Agreement and its design and implementation as well as overall perceptions of the successes, challenges, and intent of the Partnership Model. We began with open-ended questions asking them to describe what they put in their Partnership Agreement, what they did to implement the Agreement, and why. We also asked about the successes and challenges they experienced with design and implementation. Depending on the depth of responses, we asked probing questions aligned to the literature on turnaround and policy implementation such as whether or not turnover, relationships between stakeholders, and district capacity influenced implementation. Starting with open-ended questions allowed themes to first emerge primarily from the participants’ experience rather than from the interviewers signaling the importance of specific factors. Following up with nonleading, probing questions allowed us to check to make sure the participant did not accidentally omit or overlook a theme that might have been important. To help maintain confidentiality, we assigned hockey team name pseudonyms to interview districts, and we describe interviewee roles with general categories, such as “superintendent” or “leader.”

Team members coded the interviews using a deductive coding scheme in Dedoose software to condense data in alignment with the theory of change and key themes related to implementation, including codes such as “role of [district leader],” “district capacity,” “human capital initiatives,” and “role of MDE.” The coding scheme was developed by three members of the qualitative team, with Year 2’s coding scheme building from the original Year 1 coding scheme. For Year 1, coders independently coded three interviews to check for reliability, and in Year 2, coders re-established reliability by coding one shared interview and finding agreement. Coders also met regularly to share findings and ensure reliability in coding. In Year 1, after an initial round of coding, we organized key quotes and observations in a matrix to compare findings within and across interviews. Matrices facilitate comparisons and allow trends to emerge within and across cases as researchers group findings together (Bush-Mecenas, & Marsh, 2018). Columns of the matrix included response to the reform, perceived benefits, perceived challenges, and conditions impacting coherence, and each row contained one interview’s data. The matrices allowed us to categorize and better understand the variety of responses within and across Partnership districts. In Year 2, we used a similar process to identify key themes related to implementation as districts continued to refine their practices and plans to meet Partnership goals. We organized quotes by theme across interviews to better understand the scope and variation within the data. Finally, when possible, we counted mentions of key themes that emerged from district leader responses to the structured elements of our interviews. When appropriate, we present these counts to show similarities and differences across Partnership districts. Examples include mentions of specific ways districts utilized partners, positive impressions of working with the state education agency and ISDs around teacher recruitment and retention, and/or using funding associated with the reform.

In the following section, we first discuss the results of our event study analyses that estimate changes in student outcomes attributable to the Partnership Model. We then turn to our qualitative and survey data to examine how Partnership implementation shaped these outcomes. By triangulating across these findings, we are able to explore how the ways in which Partnership schools and districts approached their reforms have shaped the reform’s effectiveness.

The Partnership Model’s Estimated Effects on Student Outcomes

We show that the Partnership Model improved student outcomes in Partnership schools overall, although these effects varied by cohort, subject, and year. We also find evidence that Partnership had positive effects on other low-performing schools in Partnership districts, as suggested by the Partnership Model’s theory of change. In both cohorts, these effects are larger when excluding students in non-Partnership schools in Partnership districts from the comparison group, pointing to possible positive effects of the district-level intervention components. We also find smaller but positive effects for students in near-selected schools in Partnership districts, providing additional evidence for district-level effects across low-performing schools in Partnership districts. In addition, Partnership effects are larger for low-achieving students.

Full Sample

Cohort 1

Table 2 provides the effect estimates from our main event study models. For Cohort 1, we find evidence that Partnership led to increased student achievement in both math and ELA in the first year of implementation, with the ELA effects continuing into the second year. We provide results from the Cohort 1 event study models in Panel A of Table 2 and the top row of Figure 2. Columns 1 to 8 of Table 2 show results for each outcome, and the rows provide coefficients for the Partnership × Year interaction term in our event study model, which estimates the difference between our treatment and comparison groups, relative to the year of Partnership identification, in each year. We interpret the coefficients for the years after Partnership identification as the causal effect of Partnership on the relevant outcome and use the coefficients for the years leading up to Partnership identification to test the assumption of parallel trends between these groups prior to their selection for Partnership. For Cohort 1, we observe outcomes for 2 years of treatment: the 2017–2018 and 2018–2019 school years.

OPEN IN VIEWER

Figure 2.

Event study estimates by subject and cohort.

Columns 1–2 and 3–4 provide the levels and gains models for Grade 4 to 8 math and ELA, respectively, which should bound the true effect of Partnership in the years after identification (Imberman, 2010). Here, significant estimates in either the gains or levels specification is suggestive of a significant effect, while significant estimates for both specifications provide stronger evidence. ¹¹ Coefficients for the years leading up to Partnership identification, which are also illustrated in Figure 2, are generally small and none are statistically significant, which supports the assumption of conditional parallel trends. Columns 1 and 2 suggest math improvements in Grades 4 to 8 of 0.047 to 0.121 standard deviations in 2017–2018, although the estimate on the lower bound is not statistically significant. In 2018–2019, we see descriptively positive but insignificant estimates of 0.071 to 0.075 standard deviations. Columns 3 and 4 show that fourth- through eighth-grade students in Partnership schools experienced significant and sustained achievement gains in ELA in the first 2 years of Partnership. In the first year of implementation (2017–2018), the ELA achievement of students in Partnership schools improved by between 0.066 and 0.112 standard deviations. Improvements continued into 2018–2019, when students assigned to Partnership performed by between 0.075 and 0.103 standard deviations higher relative to comparison students. We do not find evidence in our school-level models of significant improvements in SAT scores (Columns 5 and 6), graduation (Column 7), or dropout (Column 8).

Cohort 2

We find little evidence in our main models, shown in Panel B of Table 2 and the second row of Figure 2, that Partnership impacted the Cohort 2 student and school outcomes under study. Only 1 year of post-Partnership outcomes are available for Cohort 2 as this group of schools was identified for treatment in the 2017–2018 school year and began implementing their respective Partnership Agreement in the 2018–2019 school year. Here, too, estimates for the years prior to Partnership identification are, overall, small and none are significant.

Columns 1 to 4 show that the coefficient estimates for Grade 4 to 8 math and reading are positive, but none are significant at conventional levels. Estimates for school-level SAT scores, graduation rate, and dropout rate, shown in Columns 5 to 8, are highly imprecise. Together, these results suggest that the Partnership Model was more effective in improving Grade 4 to 8 student achievement in Cohort 1 than Cohort 2, and had no effect on SAT achievement, on-time high school graduation, and dropout. Supplemental analyses provide some evidence of small positive effects in Grade 4 to 8 math and ELA in Round 2 but not Round 3 schools (Supplemental Appendix Table A-6 in the online version of the journal). ¹²

Robustness tests

We test the robustness of these findings in several ways. For both cohorts, our estimates are qualitatively similar, though less precise, when we move the reference year back to the year prior to identification (see Supplemental Appendix Table A-7 in the online version of the journal). Results are also similar using the “all starters” approach—though somewhat larger and generally more precise. These results (Supplemental Appendix Table A-8 in the online version of the journal) provide some evidence that Partnership did not induce transfers out by the highest performing students. We find similar effects using baseline rather than time-varying covariates (Supplemental Appendix Table A-9 in the online version of the journal) and can make similar inferences when we cluster standard errors at the district rather than school level (Supplemental Appendix Table A-10 in the online version of the journal).

The Effect of Partnership on Other Low-Performing Schools in Partnership Districts

Using the full sample of Partnership and comparison schools may lead to downward-biased estimates if the intervention leads to improvements in Partnership districts’ operations that enable them to more effectively support all of their schools and not just those identified for Partnership. We therefore present a set of estimates in which the treatment group includes all students in Partnership schools and the comparison group includes only students in schools within districts never identified for Partnership. Using this sample, estimates for Grade 4 to 8 math and ELA are both larger in magnitude and more precise for both cohorts of Partnership schools.

The left panel of Table 3 presents these results for Cohort 1 Partnership schools. We find that student math achievement increased by 0.087 to 0.174 standard deviations after the first year of Partnership implementation and by 0.123 to 0.124 standard deviations after the second, with each estimate significant at the p < .05 level. Estimates for ELA range from 0.092 to 0.137 standard deviations after the first year of Partnership and 0.10 to 0.133 standard deviations after the second, all significant at the p < .05 level. These estimates are meaningfully larger in magnitude, especially in math, and more significant when compared with our estimates from the full sample.

OPEN IN VIEWER

Table 3

Partnership Effects on Student Achievement in Grades 4-8, Non-Partnership Schools in Partnership Districts Removed From the Comparison Group

	Cohort 1				Cohort 2
	(1)	(2)	(3)	(4)	(5)	(6)	(7)	(8)
	M-STEP Math Levels	M-STEP MathGains	M-STEP ELALevels	M-STEP ELAGains	M-STEP MathLevels	M-STEP MathGains	M-STEP ELALevels	M-STEP ELAGains
Partnership School 2013- 2014	0.092(0.057)	0.057(0.050)	0.055(0.063)	−0.011(0.048)	0.049(0.050)	0.019(0.047)	0.021(0.049)	−0.013(0.044)
Partnership School 2014–2015	0.021(0.053)	−0.034(0.050)	−0.022(0.055)	−0.083 † (0.046)	−0.016(0.051)	−0.086 † (0.048)	−0.034(0.049)	−0.088 † (0.049)
Partnership School 2015–2016	0.052(0.036)	0.057(0.047)	0.033(0.035)	0.082(0.052)	−0.014(0.040)	−0.010(0.040)	−0.000(0.043)	0.040(0.035)
Partnership School 2016–2017	ref.	ref.	ref.	ref.	0.030(0.032)	0.049(0.039)	0.042(0.033)	0.061(0.042)
Partnership School 2017–2018	0.087*(0.038)	0.174** (0.057)	0.092** (0.029)	0.137** (0.046)	ref.	ref.	ref.	ref.
Partnership School 2018–2019	0.123* (0.050)	0.124* (0.052)	0.133*** (0.039)	0.100* (0.044)	0.051* (0.021)	0.088* (0.039)	0.021(0.019)	0.059(0.041)
Constant	0.061(0.193)	1.076*** (0.214)	0.085(0.107)	1.001*** (0.152)	−0.149(0.125)	0.799*** (0.122)	−0.087(0.120)	0.843*** (0.130)
School covariates	X	X	X	X	X	X	X	X
Student covariates	X	X	X	X	X	X	X	X
Grade FE	X	X	X	X	X	X	X	X
N	59,708	59,708	60,003	60,003	116,065	116,065	116,379	116,379
Adjusted R2	0.692	−0.224	0.721	−0.214	0.633	−0.218	0.663	−0.214
Within R2	0.026	0.020	0.028	0.017	0.016	0.015	0.017	0.010

Note. All models include student fixed effects and time-invariant treatment assignment based on school assignment in the identification year; robust standard errors clustered at the time-invariant school level in parentheses. M-STEP = Michigan Student Test of Educational Progress; ELA = English language arts; FE = fixed effects.

†

p < .10. *p < .05. **p < .01. ***p < .001.

Results for Cohort 2 Partnership schools are shown in the right panel of Table 3. Here, too, estimates for M-STEP ELA and math are larger than the corresponding estimates when using the full sample. In math, student achievement improved by 0.051 to 0.088 standard deviations, both significant at conventional levels. ELA estimates are likewise larger in magnitude using this sample, at 0.021 to 0.059 standard deviations, but imprecise. For the other outcomes of SAT math and ELA achievement, graduation rate, and dropout (not shown here), estimates are similar to the main specification and not significant. These results show that the Partnership model has stronger effects when students in identified Partnership schools are compared with students in schools that were neither directly nor indirectly impacted by Partnership. Consistent with the Partnership Model’s theory of change, Partnership may have district-level effects that benefit students in other low-performing schools, thus attenuating estimates in our main models.

Next, we answer this question more directly by estimating the effects of being in a low-performing school in a Partnership district in addition to the effects of being in a Partnership school. In both cohorts, we find effects in Grade 4 to 8 math in near-selected Partnership district schools that are statistically significant and similar in magnitude to the effects on Partnership schools (Supplemental Appendix Table A-11 in the online version of the journal). Although we find positive and significant effects in ELA for Cohort 1 Partnership schools, we do not detect ELA effects for near-selected Partnership district schools in either cohort. Thus, consistent with the Partnership Model’s theory of change, enhanced district-level systems may have driven improvements not just in Partnership schools but also in other low-performing schools in Partnership districts. It is also possible that Partnership districts allocated resources and/or directed efforts to Partnership schools as well as other low-performing schools. We explore these possibilities in the section below where we discuss how Partnership district leaders approached the strategic planning element of the Partnership Model.

Partnership Effects by Prior Achievement

Table 4 provides estimates for Grade 4 to 8 math and ELA achievement by prior achievement, with Cohort 1 in Panel A and Cohort 2 in Panel B. Columns 1 to 2 of Panel A show that Cohort 1 students in the bottom quartile of achievement improved math scores by 0.092 to 0.183 standard deviations in the first year, with both estimates being significant, and 0.096 to 0.126 standard deviations in the second year, although only the upper bound is significant. Estimates for students in the top three quartiles (Columns 3 and 4) are descriptively positive but insignificant. Estimates for bottom quartile students in ELA are 0.102 to 0.195 standard deviations in Year 1 and 0.079 to 0.093 standard deviations in the second year (Columns 5 and 6). However, the Partnership × 2015–2016 term for this sample (also in Columns 5 and 6) is also positive and significant, indicating that the positive estimates for low-achieving students in ELA may in part stem from mean regression following an identification year dip for these students—although the estimates are descriptively larger in the first year of implementation and sustain into the second year. In ELA, we also find some evidence that comparatively high-achieving students experienced achievement gains following Partnership identification—although these estimates are less consistent. As shown in Columns 7 and 8, estimates range from 0.051 to 0.087 standard deviations in Year 1 and 0.07 to 0.102 standard deviations in Year 2, with the upper bounds of those estimates significant at the p < .10 and p < .05 levels, respectively.

OPEN IN VIEWER

Table 4

Grade 4 to 8 Math and ELA Achievement by Lagged Performance

	M-STEP Math				M-STEP ELA
	Bottom quartile		Top 3 quartiles		Bottom quartile		Top 3 quartiles
	(1)	(2)	(3)	(4)	(5)	(6)	(7)	(8)
	Levels	Gains	Levels	Gains	Levels	Gains	Levels	Gains
Panel A. Cohort 1
Partnership School 2013–2014	0.157* (0.071)	0.095(0.084)	0.095(0.057)	0.113* (0.052)	0.102 † (0.058)	0.084(0.068)	0.102(0.065)	0.026(0.054)
Partnership School 2014–2015	0.054(0.057)	−0.007(0.061)	0.021(0.051)	−0.046(0.054)	−0.010(0.049)	−0.019(0.061)	−0.033(0.055)	−0.136** (0.045)
Partnership School 2015–2016	0.057(0.046)	0.022(0.058)	0.015(0.034)	0.015(0.043)	0.068 † (0.039)	0.129* (0.062)	0.021(0.036)	0.061(0.057)
Partnership School 2016–2017	ref.	ref.	ref.	ref.	ref.	ref.	ref.	ref.
Partnership School 2017–2018	0.092* (0.045)	0.183* (0.076)	0.028(0.037)	0.089 † (0.053)	0.102** (0.036)	0.195** (0.062)	0.051(0.033)	0.087 † (0.045)
Partnership School 2018–2019	0.126* (0.055)	0.096(0.066)	0.052(0.049)	0.074(0.051)	0.093* (0.045)	0.079(0.050)	0.102* (0.042)	0.070(0.045)
Constant	−1.233*** (0.192)	0.363 † (0.213)	0.391 † (0.223)	1.282*** (0.208)	−1.296*** (0.174)	0.397 † (0.232)	0.417** (0.135)	1.156*** (0.171)
N	16,095	16,095	52,642	52,642	15,903	15,903	53,052	53,052
Adjusted R2	0.411	−0.063	0.684	−0.185	0.418	−0.030	0.703	−0.191
Within R2	0.189	0.148	0.032	0.037	0.213	0.166	0.035	0.030
Panel B. Cohort 2
Partnership School 2013–2014	0.113* (0.055)	0.067(0.064)	0.049(0.050)	0.038(0.043)	0.007(0.060)	−0.024(0.061)	0.040(0.049)	0.026(0.048)
Partnership School 2014–2015	−0.005(0.053)	−0.062(0.059)	−0.031(0.050)	−0.080(0.051)	−0.033(0.051)	−0.028(0.059)	−0.056(0.050)	−0.107 † (0.059)
Partnership School 2015–2016	0.038(0.038)	0.079(0.048)	−0.025(0.036)	0.007(0.038)	−0.012(0.036)	0.037(0.037)	−0.003(0.040)	0.052(0.035)
Partnership School 2016–2017	0.073* (0.033)	0.081(0.053)	0.023(0.029)	0.047(0.036)	0.022(0.032)	0.035(0.044)	0.029(0.032)	0.042(0.040)
Partnership School 2017–2018	ref.	ref.	ref.	ref.	ref.	ref.	ref.	ref.
Partnership School 2018–2019	0.075** (0.026)	0.123* (0.048)	0.021(0.021)	0.040(0.036)	0.054** (0.021)	0.073 † (0.042)	0.005(0.018)	0.026(0.039)
Constant	−1.244*** (0.158)	0.240(0.172)	0.242 † (0.137)	0.776*** (0.128)	−1.255*** (0.148)	0.056(0.146)	0.279 † (0.147)	0.865*** (0.146)
N	27,060	27,060	90,992	90,992	26,308	26,308	91,984	91,984
Adjusted R2	0.334	−0.047	0.643	−0.170	0.326	−0.041	0.666	−0.182
Within R2	0.173	0.165	0.037	0.030	0.166	0.169	0.036	0.021

Note. Prior achievement measured in identification year. All models include school, student, and grade fixed effects. Bottom quartile (Columns 1–2 and 5–6) are those in the bottom quartile of their school in the identification year. Top 3 (Columns 3–4 and 7–8) quartiles are those above the 25th percentile of their school in the identification year. Note the sample is smaller than the main sample because to be in this sample, students need to have an M-STEP score in the identification year. M-STEP = Michigan Student Test of Educational Progress; ELA = English language arts; FE = fixed effects.

†

p < .10. *p < .05. **p < .01. ***p < .001.

Panel B provides these estimates for Cohort 2. The significant positive estimates on the Partnership × 2016–2017 indicator (i.e., the year prior to identification) point to an identification year dip for the lowest achievers in math. We thus interpret these findings with caution. Following that dip, we find that estimates for low-achieving students in math, shown in Columns 1 and 2, rebounded to preidentification levels (Column 1) and were descriptively higher than preidentification gains (Column 2). Columns 5 and 6 of Table 4 show that ELA achievement for low-achieving students increased by 0.054 to 0.073 standard deviations. Collectively, these results suggest that Partnership was particularly beneficial for the lowest achieving students in Partnership schools, with gains in math and ELA evident in both cohorts, and that the overall effects of Partnership are driven by improvement in this group.

How Did Partnership Implementation Contribute to Outcomes?

In this section, we draw on our interview and survey data to explore how Partnership districts’ implementation of the Partnership Model contributed to the policy’s positive effects on ELA and math achievement in Grades 4 to 8. The Partnership Model’s theory of change posits several mechanisms through which the policy facilitates improvement in low-performing schools. Our study of Partnership implementation identifies four such mechanisms that shaped these positive outcomes: strategic planning, external supports, the threat of accountability, and improving human capital.

The Threat of Accountability

Although the focus of the Partnership Model is on providing supports to improve student outcomes in low-performing schools and districts, it creates pressure for schools to improve by requiring Partnership Agreements to include next-level accountability (NLA) measures to be implemented if a school’s 36-month improvement goals are not met. We find that the threat of accountability is salient for educators in Partnership schools and that district leaders have used the threat of NLA to build momentum for their reform agenda.

Table 5 compares teachers’ perceptions of school accountability if their school’s improvement goals are not met between teachers who work in Partnership and non-Partnership schools. Comparisons using nonresponse weights are very similar and are provided in Supplemental Appendix Table A-12 (in the online version of the journal). Teachers working in Partnership schools have a much stronger sense of accountability than teachers who work in non-Partnership schools, and this difference is significant at the p < .001 level. Some items that make up this school accountability composite score illuminate teachers’ perceptions around accountability for failing to improve. For instance, teachers in Partnership schools, compared with teachers in non-Partnership schools, see it as much more likely that their school leader or other staff will be removed if their school does not meet its improvement goals (see Supplemental Appendix Figure A1 in the online version of the journal). This suggests that teachers in Partnership schools see the threat of NLA as real, which may motivate them to more fully support their school’s improvement efforts.

OPEN IN VIEWER

Table 5

Teachers’ Perceptions in Partnership Schools and Non-Partnership Schools in Partnership Districts.

	Partnership	Non-partnership	Difference (Partnership − non-partnership)	p value
Staffing	−0.182	0.128	−0.311***	.000
School leadership	0.064	−0.043	0.107**	.001
School accountability	0.320	−0.236	0.556***	.000
Observations	5,474

Note. Factor scores are based on responses from when teachers were in the first year of their school’s Partnership reform.

†

p < .10. *p < .05. **p < .01. ***p < .001.

Partnership teachers’ heightened perception of accountability likely comes at least in part from the messaging they received around Partnership from their school and district leadership. In interviews, multiple leaders reported using their Partnership status to build a sense of urgency for their reform agenda and used this to enact changes that otherwise would have been more challenging. Describing a debate within the district over when to implement a new reading program, the superintendent of Blues used their Partnership status to push for more immediate action, noting, “How do we help the teachers feel this urgency? That’s where I think the power of the Partnership can come in if you capitalize on it, like, look [we’ve only got so much time].” Some leaders identified specific actions they were able to take using the urgency they built around their Partnership status, such as the superintendent of Penguins, who at one of their Partnership schools changed the grade configuration, replaced the principal, and also replaced a large share of the teachers. They noted that such drastic changes would normally be quite difficult, but that Partnership created a space to implement this change. Another Partnership district leader explained this strategy as “I would just say a leader knows how to use extra accountability as a lever for change and creating a sense of urgency.” They further elaborated,

Having come in knowing that these were the lowest performing schools, I’ve been able to, more over the last couple of years, use the Partnership Agreement as a lever to get a greater sense of urgency among the principal and teachers to say “We have to do things differently here. Why? Because we’re a Partnership school.”

These accounts illustrate how some Partnership district leaders used Partnership, and in particular the threat of high-stakes accountability, as a warrant to make changes that aligned with their vision for improvement in identified schools.

Improving Human Capital

Consistent with earlier research on school turnaround (Harbatkin, 2022; Heissel & Ladd, 2018; Henry et al., 2020; Strunk, Marsh, Hashim, et al., 2016; Zimmer et al., 2017), the Partnership Model’s theory of change recognizes human capital as a key driver of improvement in turnaround schools. There are two important elements of human capital to consider—effective school leaders (Aladjem et al., 2010; Hitt et al., 2018) and high-quality teachers (Henry & Harbatkin, 2020; Henry et al., 2020; Pham, 2022). We find that Partnership districts had mixed success in improving human capital in their Partnership schools across these two categories. Overall, it appears that Partnership districts were able to bring in more effective principals to lead Partnership schools. However, while some leaders reported improvements in their ability to attract and retain teachers in Partnership schools compared with prior years, they still cited maintaining a supply of high-quality teachers as among their most significant challenges.

In our interviews, Partnership district leaders noted some areas of concern around their leaders but identified principals as key players in their reforms. Speaking about some of the specific challenges in their Partnership schools, the superintendent of Canadiens lamented that many of their leaders were not well-equipped to drive instructional change, saying,

I think a lot of our Partnership principals are not instructional leaders, defined as individuals that know how to recruit, retain teachers, know how to give constant feedback to teachers, can problem-solve with teachers, and create a culture of instructional focus where the principal is truly the instructional leader. Can go in as a generalist in specific content areas, provide feedback on how to improve instruction, can lead professional development, can analyze data, and really have credibility with teachers to improve practice.

Acknowledging issues with leadership, several Partnership district leaders replaced principals to create an environment more conducive to growth and improvement. The superintendent of Penguins, for example, strategically selected a principal from another school to lead a Partnership school and anchored their reforms there around this new leadership, saying,

That school has been a pain in the whatever of anybody who’s been sitting in this chair, because it just has been a mess. I mean a total mess. No buy in or whatever. [. . .] We turn it over to this new principal—not new but [principal name], and we say, “Go for it.” [. . . ] and it’s like the sun has risen on that place. It’s a new environment, it’s a new place, different staff, kids are happy, not a peep out them, community is happy.

Teachers in Partnership schools seemed to understand district leaders’ efforts to improve the quality of leadership in their schools. Table 5 shows that, on average, teachers in Partnership schools gave their principals significantly higher ratings than teachers in non-Partnership schools, although this difference is not significant when using weighted responses (see Supplemental Appendix Table A-12 in the online version of the journal). Within this construct, Partnership teachers gave their principals significantly higher ratings for facilitating and encouraging professional development (see Supplemental Appendix Figure A1 in the online version of the journal). This suggests that the types of efforts by district leaders noted above improved leadership quality in their Partnership schools, which in turn served as a catalyst for positive change.

Some district leaders cited improvements in their ability to recruit and retain teachers relative to prior years, which could help explain some of the academic growth in Partnership schools. For example, the leader of Senators noted they had “increased the number of certified teachers [in Partnership schools].” The Avalanche leader said they were able to use funds from Partnership to improve their hiring process, which increased the fit between candidates and Partnership schools. The Ducks superintendent said, “we’ve seen some improvement in recruitment and retention.” Others were able to use modest financial incentives to help stabilize staffing, and the leader of Flyers explained that they were able to change “job descriptions to attract a different level of talent.” Despite these perceived improvements, these and other leaders still felt greater staffing challenges in their Partnership schools compared with other schools.

These improvements in staffing are best characterized as making the problem somewhat less acute; retaining and developing high-quality teachers in Partnership schools remained a significant challenge for most. Partnership schools experience much greater teacher turnover than other schools and struggle to replace them with the high-quality teachers they believe are necessary for their reforms, which hampered their efforts to improve instruction. In fact, 23% to 29% of teachers exit Partnership schools each year, roughly twice the statewide average, a trend that predated Partnership and persisted into the years of Partnership implementation (authors’ calculation, 2021). It is perhaps unsurprising then that, as Table 5 shows, teachers in Partnership schools give their school lower ratings for staffing (robust to using survey weights, see Supplemental Appendix Table A-12 in the online version of the journal). Within this construct, teachers in Partnership schools gave their schools particularly low marks for teacher retention (Supplemental Appendix Figure A1 in the online version of the journal). Moreover, leaders reported significant challenges in attracting the high-quality educators they need, and sometimes challenges even in attracting educators with basic credentials. The superintendent of Devils noted,

For every opening, you used to get 40 applicants, ten of whom were very hirable, five of whom were probably great. I’m talking up ‘till five about five years ago. Now, you are lucky to get one (applicant), extremely lucky to get two or three, and incredibly lucky if one of them is really a hirable candidate.

Of the 22 Partnership district leaders interviewed in the second year of data collection, 20 raised the supply of teachers as a concern in their turnaround work, which complicated Partnership districts’ efforts to reform the instructional core of their Partnership schools.

Issues around teacher recruitment and retention made it more challenging for Partnership schools to implement their Partnership Agreement by making it difficult to build momentum around instructional reform. Recalling that turnover forced their district to keep “starting over,” the superintendent of Flyers described,

We’ve been unable to have one program model implemented more than one year due to turnover in either leadership and/or teachers. High-quality certified teachers is another very large obstacle that we continue to face. In a building with 19 total teachers, four of them are certified teachers, so we have a serious shortage in the availability of certified teachers. [. . .] There’s the consistency issue, which prevents the model from being able to really take hold, and then there’s the lack of talent.

In addition to the challenge of sustaining reform in the face of turnover, this quote refers to a reliance on teachers who are less qualified, and in some cases not certified. Although it was not typical for a majority of teachers in Partnership schools to be noncertified, in the first year of our Partnership superintendent interviews, one third described using long-term substitute teachers to fill teaching positions in their Partnership school(s). These examples highlight the challenges of successfully implementing a turnaround plan in the face of instability among the teaching staff and suggest that Partnership may have been more effective if Partnership schools had greater access to certified and effective teachers.