Abstract
A pervasive issue in the school choice literature is whether schools of choice cream skim students by enrolling high-achieving, less-challenging, or less-costly students. Similarly, schools of choice may “push out” low-achieving, more-challenging, or more-costly students. Using longitudinal student-level data from Indiana, we created multiple measures to examine whether there is evidence consistent with the claims of voucher-participating private schools cream skimming the best students from public schools or pushing out voucher-receiving students. We do not find evidence consistent with the claim of cream skimming. However, we find evidence consistent with the claim of private schools pushing out the lowest-achieving voucher students. This is the first study to examine these two issues within a statewide private school voucher program.
Keywords
The best research with implications for the cream skimming debate is by Fleming and colleagues (2015), who examined data of Milwaukee TPS students who used vouchers and found that parents of voucher students had higher educational attainment, but lower incomes. Meanwhile, students remaining in TPSs had higher initial test scores and were more likely to be special education students. While informative, it only indirectly addresses the central issue of the cream skimming claim—whether vouchers lead private schools to attract the best students from TPSs. To examine this issue more directly, it is important to examine whether the students exiting a TPS to attend a private school using a voucher are the highest-performing, least-challenging, or least-costly students relative to all their peers in the exiting school.
For the pushout question, the research is even thinner, with only two studies on voucher students in Milwaukee. Cowen and colleagues (2012) examined the characteristics of voucher students who exited a private school to return to a TPS and found that lower-performing students are more likely to exit private schools. Subsequently and using math and English language arts (ELA) achievement data on a statewide assessment, Carlson et al. (2013) found that the performance of voucher students who move out of a private school improved once they entered a TPS. However, neither analysis made comparisons to the likelihood of low-performing students exiting schools in general (including TPSs) or the achievement of students after making other school transitions, which limits the ability to speak to whether these are general trends among all schools or specific to voucher students exiting private schools.
We expand on existing work by examining claims of cream skimming and pushout in the context of Indiana’s voucher program. The Indiana Choice Scholarship Program is one of the largest statewide private school voucher programs, enrolling nearly 45,000 students in the 2021–2022 school year (Indiana Department of Education, 2022). We use 8 years (2010–2011 through 2017–2018) of statewide longitudinal data from the Indiana Department of Education. As all voucher-participating private schools report data to the state and take the same assessments as public schools, we can examine transition rates of voucher recipients to and from voucher-participating private schools. We build off empirical approaches of studies that have examined the cream skimming and pushout issues in charter schools across several states (Kho et al., 2022; Winters, 2017; Zimmer et al., 2011; Zimmer & Guarino, 2013) and draw comparisons with the transition rates of voucher-eligible peers in TPSs. We address two main questions:
We cannot prove definitively whether voucher-participating private schools are cream skimming or pushing out students using only administrative data. Administrative data do not allow us to discern the action of the schools (supply side) from the decision-making of families (demand side). Therefore, our analysis focuses on whether we can find patterns in the data consistent with the claims of cream skimming or pushout behaviors, which as suggested in the charter literature (Kho et al., 2022), is a necessary, but not sufficient condition, for identifying cream skimming and pushout behaviors by schools. From the TPS’s perspective, it does not matter whether a voucher student’s move is the result of the private school’s behavior or the choice of the family. In either case, the moves create a greater burden on TPSs, which are left educating the most challenging and costly students.
Empirical Approach
We provide a condensed overview of our empirical approach here. The full technical details of our data set, sample description, and robustness checks are available in the Supplementary Appendix in the online version of the journal. For the cream skimming question, we examine whether students entering the voucher program had above-average test scores, had fewer disciplinary incidents, or were less likely to be classified as either an English language learner (ELL) or a special education student relative to their previous voucher-eligible peers in the TPSs. For the pushout analysis, we examine whether students exiting the voucher program had below-average test scores or were more likely to be classified as either an ELL or a special education student relative to their former voucher peers in the private school they exit and the broader population of voucher-eligible students in TPSs. Private schools do not report disciplinary data to the state, so we cannot assess movement based on these characteristics for the pushout analysis. In both the cream skimming and pushout analyses, we compare the probabilities of voucher students changing schools with their respective comparison groups. For the cream skimming question, we include all school transitions, while for the pushout question, we only examine nonstructural school changes (i.e., students moving before the school’s terminal grade), as schools have little motivation to push out students upon reaching the maximum grade level offered by the school.
For our formal models, we do not include any controls. From a perspective of whether there is cream skimming, student characteristics (e.g., sex, race/ethnicity) may not be important as we want to know whether the voucher-participating private schools are skimming off higher-performing students or pushing out lower-performing students regardless of other student characteristics. Therefore, we run simplified linear probability models with the discrete outcome of whether the student exits their school as a function of a series of key independent variables. As a sensitivity analysis, we include student characteristics such as sex, race/ethnicity, ELL status, and special education status for both cream skimming and pushout and display the results below our main results in Tables 1 and 2. Our analyses do not yield causal estimates, but rather descriptive information about patterns in the data consistent with the cream skimming and pushout claims.
Cream Skimming Main Results
Note. Standard errors, clustered by school, in parentheses. Separate analyses conducted for each subject and achievement threshold as well as for the disciplinary, ELL status, and special education status indicators. Analyses include all voucher-using and voucher-eligible students in traditional public schools (TPSs). The dependent variable includes all structural and nonstructural transitions from public schools. The key estimates are the interaction terms, indicating the differential likelihood of high-achieving/low-discipline/lower cost to educate students use a voucher to transition to a private school relative to their similar peers to transitioning to any other school without a voucher. We do not display the results from the model of the parameter δ (the coefficient on the voucher private baseline transition rate). This parameter is necessary for estimation but not a useful to interpret (see Supplementary Appendix in the online version of the journal for more information). ELL = English language learner; TPSs = traditional public schools; ELA = English language arts.
p < .05. **p < .01. ***p < .001.
Pushout Main Results
Note. Standard errors, clustered by school, in parentheses. Separate analyses conducted for each subject and achievement threshold. Analyses include all voucher-using students in private schools and all voucher-eligible students in traditional public schools (TPS). The dependent variable includes only nonstructural transitions from public schools. Italic estimates represent statistically significant differences in the likelihood of making a nonstructural transition at p < .025 between below-test-score threshold/ELL/special education voucher students and below-test-score threshold/ELL/special education TPS students, derived from the sum of the voucher private transition main effect and the Voucher × Below Threshold interaction term. Bold estimates represent statistically significant differences in the likelihood of making a nonstructural transition at p < .025 between below-test-score threshold/ELL/special education voucher students and above-test-score threshold/non-ELL/non-special education voucher students, derived from the sum of the below-test-score threshold main effect and the Voucher × Below Threshold interaction term. ELL = English language learner; TPS = traditional public schools; ELA = English language arts.
p < .025. **p < .005. ***p < .0005 after adjustment for multiple comparisons.
Cream Skimming Estimation
Our sample for the cream skimming analysis consists of all voucher-eligible students in TPSs (for a sample description, please see Supplementary Appendix Table A1 in the online version of the journal). For an analysis of student movement patterns consistent with the claim of cream skimming, we assess the likelihood of voucher students moving to a private school (the move could be either structural or nonstructural move) in a given year t who are classified as high-achieving, low-discipline, or not an ELL or special education student in the previous year t − 1. We compare this likelihood to all other voucher-eligible students who may make any move without a voucher. We estimate this using achievement as a measure of performance in Equation 1:
Here,
In the cream skimming analysis, we want to know whether the rate in which high-achieving students transition from their public school to a private school after receiving a voucher (i.e., voucher users) is different from the rate in which high-achieving students—who are also voucher-eligible—transition from their public school to any other schools without a voucher. We capture this difference in the coefficient (γ) on the interaction term (
Using this same structural model, we also examine whether students with below-average number of discipline infractions, non-ELL, and non-special education students are more likely to transfer to private voucher schools. In these analyses, a low-discipline, ELL status, or special education status indicator replaces the high-achieving student indicator,
Pushout Estimation
Our sample for the pushout analysis consists of all voucher-using students in private schools and all voucher-eligible students in TPSs (for a sample description, please see Supplementary Appendix Table A2 in the online version of the journal). Following a similar approach to our cream skimming analysis, we assess the likelihood of voucher students moving to a TPS (in this case, only a nonstructural move) in a given year t who are classified as low-achieving, or a special education or ELL student in the previous year t − 1. We want to compare this likelihood to all other voucher-eligible TPS students with the same classifications who make a move. We estimated this using achievement as a measure of performance in Equation 2:
Here,
In the pushout analysis, the coefficient of interest (γ) on the interaction term (
First, we want to know whether the rate in which low-achieving, voucher-using students transition from their private school to a public school is different from the rate in which low-achieving students—who are also voucher-eligible—transition from their public school to any other schools without a voucher. Because not all voucher students change schools, we must also account for the baseline transfer rate of all voucher students (δ). A significant and positive linear combination of δ and γ compared with θ suggests evidence that lower-achieving voucher-using students are more likely to exit a private school above and beyond the baseline rate of low-achieving, voucher-eligible TPS students who make a transition (θ).
Second, we want to know whether the rate in which low-achieving, voucher-using students transition from their private school to a public school is different from the rate in which high-achieving, voucher-using students transition from their private school to a public school. Here, we must account for θ, which captures the baseline likelihood that low-achieving students are to transition schools, relative to high-achieving students. A significant and positive linear combination of θ and γ compared with δ suggests evidence that lower-achieving voucher-using students are more likely to exit a private school above and beyond the baseline rate of high-achieving voucher-using students who make the same transition (δ). Together, these estimates yield descriptive information about patterns in the data consistent with the claim of private schools pushing out lower-performing voucher students.
Using this same structural model, we also examined whether voucher-receiving ELL or special education students are more likely to transfer out of private voucher schools. In these analyses, an ELL or special education status indicator replaces the low-achieving student indicator,
Results
Cream Skimming Results
We display our cream skimming results in Table 1. The coefficient of interest is the interaction term with statistical significance for these estimates denoted with asterisks. Overall, we find little evidence consistent with the claim that private schools are cream skimming higher-performing, less disruptive, or less-costly-to-educate students from TPS. While all high-achieving students are less likely to transition schools, the differential transition rates for high-achieving voucher students are mostly null across all subjects and achievement thresholds. The other measures we use to assess cream skimming students yield no evidence consistent with the claims of private schools cream skimming less disruptive or less-costly-to-educate students.
Pushout Results
We display our pushout results in Table 2. Italicized estimates indicate a statistically significant difference in the likelihood of low-achieving/more-costly-to-educate voucher students moving from their private school (the sum of the voucher private baseline transition and the interaction term) compared with their low-achieving/more-costly-to-educate voucher-eligible peers who move from a TPS. Bold estimates indicate a statistically significant difference in the likelihood of low-achieving or more-costly-to-educate voucher students moving from their private school (the sum of the low-achieving/ELL/special education main effect and the interaction term) compared with their high-achieving or less-costly-to-educate voucher peers.
Overall, we find that lowest-achieving voucher students are exiting private schools at a modestly higher rate than their similarly low-achieving voucher-eligible TPS peers as well as their higher-achieving voucher private school peers. The transition rate for low-achieving voucher students is higher than their low-achieving, voucher-eligible TPS peers by 1 to 3 percentage points. Considering that voucher students in general are a percentage point less likely to leave private schools, a statistically significant differential transition rate of 2 percentage points only holds for the lowest-achieving students (below the 10th percentile statewide) when looking at the linear combination of these two terms. Low-achieving voucher students are also 3 to 9 percentage points more likely to move from a private school than their higher-achieving voucher peers. This evidence is consistent across all achievement thresholds.
When looking at the potential pushout of ELL and special education voucher students, the evidence is mixed. Voucher ELL students are 2 percentage points less likely to move from their private school as compared with voucher-eligible ELL peers who move from their TPS. Voucher ELL students are also 4 percentage points less likely to make a transition from their private school than their non-ELL voucher peers. Meanwhile, voucher special education students are 3 percentage points more likely to move a transition from their private school than their non-special education peers, though this rate is similar to the rate at which special education students move from a TPS.
Conclusion
While our analysis does not provide evidence consistent with the claim of voucher-participating private schools cream skimming the best students from TPSs based on ability, disciplinary background, or cost to educate, it raises concerns as to whether voucher programs are creating effective educational opportunities for the lowest-achieving students. As Indiana invests more than US$240 million of public funds annually in vouchers for students to attend private schools (Indiana Department of Education, 2022), policymakers should be wary of potential exacerbated educational inequalities and the challenges that low-performing students exiting out of private schools create for TPSs.
Supplemental Material
sj-pdf-1-epa-10.3102_01623737231183397 – Supplemental material for Cream Skimming and Pushout of Students Participating in a Statewide Private School Voucher Program
Supplemental material, sj-pdf-1-epa-10.3102_01623737231183397 for Cream Skimming and Pushout of Students Participating in a Statewide Private School Voucher Program by R. Joseph Waddington, Ron Zimmer and Mark Berends in Educational Evaluation and Policy Analysis
Footnotes
Acknowledgements
Author order determined randomly; each contributed equally to this article. This research was supported by the University of Notre Dame Center for Research on Educational Opportunity (CREO) and Institute for Educational Initiatives. We are grateful to the Indiana Department of Education for providing access to the state administrative records and for supporting independent analyses. We are also grateful for the helpful feedback we received from colleagues, including J. S. Butler, Stephen Cornman, Rajeev Darolia, Joseph Ferrare, Joshua Goodman, Alison Grantham, Adam Kho, Jodi Moon, Rob Olsen, Umet Ozek, Barbara Steel-Lowney, Marsha Silverberg, Molly Stewart, Genia Toma, and anonymous reviewers. All opinions expressed in this article represent those of the authors and not necessarily the institutions with which they are affiliated. All errors in this article are solely the responsibility of the authors.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported by a Lyle Spencer Research Award (#201600089) from the Spencer Foundation.
Authors
R. JOSEPH WADDINGTON, PhD, is an associate professor of educational policy studies and evaluation at the University of Kentucky College of Education and Martin School of Public Policy and Administration. His primary line of research focuses on the variation in impacts of school choice programs and policies on student outcomes.
RON ZIMMER, PhD, is a professor of public policy at the University of Kentucky, where he is the director of the Martin School of Public Policy and Administration. His research focuses on school choice and school reforms.
MARK BERENDS, PhD, is a professor of sociology at the University of Notre Dame, where he is the Hackett Family Director of the Institute for Educational Initiatives and director of the Center for Research on Educational Opportunity (CREO). His mixed-methods research focuses on how school organization and classroom instruction are related to student outcomes, with special attention to marginalized students and educational policies and reforms aimed at reducing educational inequalities. He is an elected member of the National Academy of Education and a fellow of the American Educational Research Association (AERA).
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
