Educational Evaluation and Policy Analysis

Middle school transitions are increasingly required, despite documented negative effects on general education students (GENs). We explore if and how the move to middle school differentially affects students with disabilities (SWDs), a large and low-performing group. Using an instrumental variables strategy and New York City (NYC) data on nine student cohorts, we find the middle school transition causes a 0.29 SD decline in SWD math performance, a 0.16 SD decline in English language-arts (ELA) performance, and a one percentage point increase in grade retention. Accounting for potential mediators (e.g., peer cohort stability) does not substantially explain estimated effects on SWD performance. However, the middle school transition has similarly large negative effects on GENs, suggesting the need to ease this transition for all students.

While there is a growing body of literature related to the benefits of participation in career and technical education (CTE) in high school, there remains a dearth of causal work in this area. Relying on administrative data from Baltimore City Public Schools, which uses a ranking system with a strict cut score to assign students into CTE-focused high schools, we employ a regression discontinuity design to estimate the causal effects of attending one of these high-quality CTE schools on attendance, career pathway completion, high school GPA, and SAT scores. We find that students who are offered admission to a program within one of the CTE centers are significantly more likely to complete a CTE pathway, though they present significantly lower GPAs. Implications are discussed.

We develop a conceptual framework for understanding and predicting teacher shortages at the state, region, district, and school levels. We then empirically examine how teacher shortages vary geographically and by subject using data on unfilled teaching positions in Tennessee traditional public schools during Fall 2019. We find that teacher staffing challenges are highly localized, causing shortages and surpluses to coexist. Aggregate descriptions of staffing challenges mask considerable variation between schools and subjects within districts. Schools with fewer local early-career teachers, smaller district salary increases, worse working conditions, and higher historical attrition rates have higher vacancy rates. Our findings illustrate why viewpoints about, and solutions to, shortages depend critically on whether one takes an aggregate or local perspective.

Twenty-nine states require or allow all 11th graders to take the ACT or SAT in school, for free, eliminating access to testing as a barrier to college entry. I examine whether this affects postsecondary outcomes using state-aggregated panel data and time-varying difference-in-differences methods. I find policy adoption led to 2% increases in selective 4-year enrollments by students from treated states. However, adoption had no effect on enrollments within adopting states’ selective institutions, in part because institutions enrolled fewer students from out-of-state after adoption. I also find null but directionally negative effects on the number of graduates from state institutions. Therefore, impacts are small but positive for students from adopting states, but null or negative for adopting states’ institutions.

AB705 is a landmark higher education policy that has changed approaches to developmental/remedial education in the California Community College system. We study one district that implemented reforms by placing most students in transfer-level math/English courses and encouraging enrollment in support courses based on multiple measures of academic preparation (e.g., grade point average [GPA]). We use regression discontinuity designs to examine the impact of these new placement recommendations, finding some benefits to English support course recommendations for low GPA students and increased math support course enrollment. We use matching methods to explore the relationship between support course enrollment and subsequent outcomes; finding enrollment in concurrent math and English support courses was associated with increased likelihood of completing transfer-level math and English.

While decades of research document inequity in the distribution of teachers across schools, few studies consider the distribution of teacher vacancies and other school hiring needs. We use teacher job postings, a direct proxy of school hiring needs, to document how these hiring needs vary across districts, schools, and subject areas. We first validate job postings as an indicator of school hiring needs by showing that filled postings closely line up with eventual teacher hires. We then show that schools serving more students of color had greater hiring needs throughout the year and that hiring needs for special education and science, technology, engineering, and math positions were considerably higher than for elementary positions according to both the number and duration of job postings.

Using administrative data of center-based child care providers in North Carolina from 2005 to 2018, we provide the first direct evidence on the effects of competition on provider quality and student outcomes in the context of early care and education, taking advantage of quality measures from the state’s Quality Rating and Improvement System (QRIS). We found that competition was associated with higher quality ratings and a higher probability to achieve a five-star rating—the highest tier in the QRIS. More competition increased providers’ probability to improve their rating and reduced the time to improve. Compared to public schools, private providers were responsive to competition. However, we did not find any effects of competition on district-level student third-grade academic performance.

Researchers and policymakers have often debated whether urban schools of choice enroll students who are relatively advantaged compared to their traditional public school peers. Existing research has not adequately answered this question due to a reliance on inadequate quantitative measures of socioeconomic status and an emphasis on differences between racial or class groups rather than within them. This mixed-methods study contributes new evidence based on novel survey data and interviews with parents and educational leaders in Detroit. Detroit charter schools enroll significantly fewer students living in deep poverty than neighborhood schools, and selective schools enroll a distinctly advantaged population. These stratified enrollment patterns result from differences in geographic constraints, the influence of social networks, school type reputations, and school practices.

This paper examines how relationships among government and “outside” organizations influenced policy implementation of new dropout prediction data systems. Using comparative historical and network analyses of three cities, I suggest the concept of interorganizational coupling—highlighting how the dependence and (in)formal collaborations among local school improvement organizations affected implementation speed, variation, and constraint. In Chicago, the loosely coupled system influenced slow and varied implementation, sustained by interpersonal relations and challenged by unclear division of labor. In Philadelphia, the tightly coupled system shaped swift and uniform changes, constrained by questions of sustainability. In New York, the tightening system led to fast yet variable transformations, limited by competition among organizations. Broadly, the article contributes to studies of education policies, interorganizational networks, and school improvement.

Although the COVID pandemic dramatically expanded K–12 remote learning in its first years, little is known about the lasting effects on virtual schooling policies and practices. Drawing on evolutionary theories of change and qualitative data from 2019 to 2022, we explore this topic in Oregon, a state with a long history of virtual schools. We find that the pandemic contributed to limited change in state virtual schooling policy and some shifts in local organizational practices. Our data also raise questions about the durability of any changes and the implications for marginalized students. Ultimately, this study provides much-needed insights on the impact of the pandemic on policy and practice, and the utility of multi-disciplinary, equity-oriented theory to understand change processes in K–12 education.

Little research has examined how K–12 fiscal accountability policies and practices intersect with district finances and student outcomes during periods of economic crises. Employing a critical policy analysis perspective that distinguishes between the concepts of fiscal accountability and fiscal austerity and differences-in-differences and event studies approaches, this study examines the practice of early fiscal intervention in California’s AB 1200 fiscal accountability system during and after the 2008 financial crisis and recession. In particular, the study estimates the relationship between early fiscal intervention, district finances, and student academic outcomes in mathematics and reading and language arts. Findings suggest that intervention led to significant cuts in per-pupil expenditures, resulted in no increase in state or federal revenues and an increase in local revenues through property taxes, and was negatively associated with student outcomes, especially in reading and language arts and for Hispanic/Latinx students. When taken together, the findings imply K–12 fiscal accountability in California was leveraged to promote fiscal austerity in educational spending during and after the 2008 financial crisis. Implications and policy recommendations are discussed.

Limited literature has investigated the effects of state and institutional merit-based financial aid on student choice of science, technology, engineering, and mathematics (STEM) major fields, an unintended consequence with important implications. By leveraging nationally representative longitudinal data from the Beginning Postsecondary Students, we examined these effects—respectively and jointly—with logistic regressions (LR) and propensity score matching (PSM). Both the LR and PSM results showed that students who receive both state- and institution-awarded merit aid were more likely to major in STEM. For students who only received state-awarded merit aid, the PSM presented significant and positive effects while the LR did not. Institution-only merit aid had no statistically measurable effect. We discuss implications for research, policy, and practice for state- and institution-based financial aid.

Educational researchers often report effect sizes in standard deviation units (SD), but SD effects are hard to interpret. Effects are easier to interpret in percentile points, but converting SDs to percentile points involves a calculation that is not transparent to educational stakeholders. We show that if the outcome variable is normally distributed, we can approximate the percentile-point effect simply by multiplying the SD effect by 37 (or, equivalently, dividing the SD effect by 0.027). For students in the middle three-fifths of a normal distribution, this rule of thumb is always accurate to within 1.6 percentile points for effect sizes of up to 0.8 SD. Two examples show that the rule can be just as accurate for empirical effects from real studies. Applying the rule to empirical benchmarks, we find that the least effective third of educational interventions raise scores by 0 to 2 percentile points; the middle third raise scores by 2 to 7 percentile points; and the most effective third raise scores by more than 7 percentile points.

Many studies rely on teachers’ reported career intentions instead of measuring actual turnover, but research does not clearly document how these variables relate to one another. We test how measures of teacher intentions relate to turnover. Using nationally representative data on 102,970 public school teachers, we conduct a descriptive and regression analysis to probe how teachers’ turnover intentions are and are not associated with moving schools and leaving teaching. While there is some variation across measures of intent, we find evidence that intention is distinct from, but strongly related to, turnover. We advise that surveys continue to capture intention as it provides meaningful information, but we recommend intention not be used as proxy for turnover.