Informing School-Choosing Families About Their Options: A Field Experiment From New Orleans

Introduction

The most prominent theory for how school choice could improve schools is straightforward. In a market for tuition-free, autonomous schools, the schools that provide the best options to families should thrive and proliferate (Chubb & Moe, 1990a, 1990b; Friedman, 1955). In practice, this has not happened as straightforwardly as the theory suggests. The potential reasons why are numerous, from structural barriers such as inadequate transportation offerings to questions about what parents seek in schools. These issues have captured the attention of researchers, who have sought to understand parents’ decision-making processes (e.g., Bell, 2009; Glazerman & Dotter, 2017) and how various policies and supports affect their choices (e.g., Hastings & Weinstein, 2008; Trajkovski et al., 2021).

With this motivation, we conducted a randomized controlled trial (RCT) in partnership with the Orleans Parish School Board (OPSB). Before families requested school placements for the 2019–2020 school year, we randomly assigned potential applicants to one of three groups. A “growth” group received lists—via flyers, text messages, and emails—of the highest performing schools they could request. For K–12, these were schools that earned top grades on new state measures of academic growth. For pre-kindergarten (pre-K), these were programs that scored highest on a quality measure (the Classroom Assessment Scoring System, or CLASS) the state had recently adopted for early childhood education (ECE). A “distance” group received lists—also via flyers, text messages, and emails—of all of the schools in the applicants’ home geographic zones. A “control” group received a parallel set of communications that did not highlight any particular schools. ¹ New Orleans is an especially suitable setting for this work. Nearly all of its public schools and publicly funded ECE programs participate in a unified enrollment system, OneApp, that uses a placement algorithm designed to elicit applicants’ true rank-ordered preferences. ²

Our findings indicate that the growth treatment affected applicants’ school choice behaviors. Those effects are concentrated among certain subgroups of applicants, and more evident on some outcomes than others. For example, we see stronger effects for high school applicants than pre-K or elementary school applicants, and the effects are especially large for families of students with disabilities (SWDs). The information provided was relatively unlikely to change applicants’ first-choice requests and more likely to lead applicants to request more (high-growth) schools. The distance treatment, meanwhile, had only modest effects.

The intervention was designed in ways that enhance its contribution to the literature. For one, due to fortunate timing with the state’s first-ever release of student progress letter grades, we shared information that was relevant to most families (measures showing how much students learn) and not previously well known to the public. By partnering with the district and communicating via flyers, text messages, and emails, we reached a large population, often in multiple ways. By running the study in New Orleans, we could see how applicants ranked their options—and administer similar interventions across the ECE, elementary/middle, and high school spectrum. As a result, we believe the study’s findings show the effects of a low-cost, well-administered intervention that reached a broad population of families at the time they were choosing schools. With respect to the findings, we think the magnitude of the effects on families of SWDs suggests a specific group that is not adequately supported during the school choice process—and that the stronger effects on applicants’ lower ranked schools have implications for understanding how information affects requests and placements.

The article proceeds with a description of prior research on parental decision-making and how the provision of information affects that decision-making, followed by a description of the policy context, data, study design, methods, and results. We conclude by discussing the implications of these results for school choice policy, focusing on the potential impact of information provision to change school choice behaviors and outcomes.

Prior Literature

Friedman (1955) authored the most prominent argument for school choice, applying the logic of markets to schools. He proposed a system in which parents (consumers) could select from a collection of government-funded schools led by private school operators (producers). Government funding was justified, he argued, by the positive externalities of schooling, as a society benefits when its children are well educated. School choice policies would improve efficiency through the market interactions between parents (who deeply care about and know their children) and school leaders (who must attract parents to receive state funding). Parents would choose high-quality schools that fit their children well, and then both those children and society more generally would benefit.

This theory, with roots in neoclassical economics, relies on assumptions about parents’ decision-making processes and the absence of barriers to interfere with that process. It implies that families carefully examine many schools and choose the ones they deem utility-maximizing. However, research on how families choose schools—and on decision-making in general—suggests a more boundedly rational process (Simon, 1956). Operating with limited time and resources, parents tend to examine only a few schools, often with vague and malleable criteria, and might “satisfice” (choose once they have found an option they find satisfactory) rather than optimize (Bell, 2009; Ben-Porath, 2009). This could mean, for example, that parents asked to rank many school options might be relatively uninformed about their lower ranked choices.

Even if families conduct thorough searches and make informed choices, a variety of factors could prevent them from enrolling in seemingly high-performing schools. First, parents might not prioritize school quality as it is commonly conceived in education research and policy. Studies of parents’ school requests indicate that parents value academic performance but that other considerations—such as proximity to home and student demographics—can receive greater weight (Denice & Gross, 2016; Frankenberg, 2018; Glazerman & Dotter, 2017; Harris & Larsen, 2019; Hastings et al., 2009; Ladd & Turaeva, 2020). Insofar as parents consider school performance, they may lean more heavily on achievement levels than on growth measures that are more directly attributable to schools (Abdulkadiroğlu et al., 2020; Glazerman & Dotter, 2017; Imberman & Lovenheim, 2016; Rothstein, 2006), although the evidence on this question is mixed (Houston et al., 2020; Houston & Henig, 2019). Moreover, some parents surely have altogether different understandings of what makes a “good school” for their child.

A second possibility is that parents want to send their children to seemingly high-performing schools but confront barriers to enrollment, such as complex application processes (Gross et al., 2015), a lack of transportation availability (Jochim et al., 2014), capacity constraints in desirable schools (Lincove et al., 2018), and logistical and financial considerations that supersede academic considerations (Harris & Larsen, 2019). These barriers may be especially daunting for families in poverty—a possible explanation for why disadvantaged families’ school requests imply less weight on academic quality criteria (Hastings et al., 2009).

A third possibility is that many families lack relevant information while choosing schools. If this is the case, then studies using school requests to examine parents’ revealed preferences could underestimate how much they value academic quality (Bergman et al., 2020; Yettick, 2016). The potential reasons behind information shortcomings are plentiful. Parents might struggle to find information or, on the other hand, could miss useful information amid a “smog” of data (Shenk, 1997). The information they find might be already known to them or seem irrelevant to their children. Relevance could be a particular issue for families with particular needs, such as parents of SWDs (McKittrick et al., 2020). Parents’ use or interpretation of school information may depend on how that information is formatted (Glazerman et al., 2020; Jacobsen et al., 2014). Furthermore, many parents prefer word-of-mouth information—the quality of which varies across social networks (Schneider et al., 1997)—and may not see much value in formal information reports (Schneider et al., 2000; Valant & Newark, 2020).

A few experimental studies have sought to identify the effects of providing school choice information to families. Hastings and Weinstein (2008) randomly assigned families in Charlotte-Mecklenburg to receive, in hard copy, lists of schools ordered by their average math and reading scores (along with, for some families, odds of admission). The information modestly increased the probability of choosing a high-scoring school, at least for some applicants. Corcoran, Jennings, Cohodes, and Sattin-Bajaj (2018) conducted a field experiment in New York City in which they randomly assigned eighth-grade students in high-poverty middle schools to receive lists of nearby high schools with relatively high graduation rates (with some students receiving additional information). The treatment led students to request more schools from the lists provided. Treatment group students did not request schools with higher graduation rates, on average, but were less likely to match to schools with low graduation rates and more likely to match to their first-choice schools. Allende, Gallego, and Neilson (2019) found that presenting families of public pre-K students in Chile with an informational video and materials led them to choose higher performing elementary schools. In Romania, Ainsworth, Dehejia, Pop-Eleches, and Urquiola (2020) found that providing information about schools’ value-added performance (and an explanation of those measures) improved the accuracy of families’ beliefs about schools, and in some cases, reshaped their criteria for judging schools. Quasi-experimental studies from the Netherlands (Koning & van der Wiel, 2013) and Chile (Mizala & Urquiola, 2013) have found only modest effects of school ratings on choice behavior.

Research from ECE choice is more limited, but recent work suggests potential for information interventions to have impact. Information shortcomings and asymmetries shape many parents’ perceptions of ECE program quality, sometimes even after enrolling in a program (Mocan, 2007). Over the last decade, most states have developed a Quality Rating and Improvement System (QRIS) to monitor the performance of ECE programs and communicate programs’ ratings to parents (Workman, 2017). Studies of these systems show that programs receiving lower ratings lose enrollment (Bassok et al., 2019) and that providing rating information to parents can lead them to switch from parental to nonparental care (Herbst, 2018) and choose higher performing options (Dechausay & Anzelone, 2016). Notably, these studies focus on private childcare programs, not school-based pre-K. Choosing a school-based pre-K program is often a choice to attend not only a pre-K program but also the elementary school affiliated with that program.

Looking across these studies, we see reason to believe that families want and use information about schools, with potentially varied responses from different groups of families (Corcoran et al., 2018; Hastings & Weinstein, 2008). We see less reason to believe that providing information is a surefire way to change behaviors or outcomes. Notably, though, how interventions are designed and administered could shape their effects. For instance, the effects from Charlotte-Mecklenburg may have been mitigated if many families did not pay attention to their mail or if reports of schools’ average test scores—which, compared with student growth measures, tend to be more widely available, more highly correlated with student composition (Reardon, 2019), and less informative about school quality (Meyer, 1997)—did not lead to much updating of school choosers’ opinions. The effects from New York City may have differed if the information had been directed to the parents of eighth-grade students, whose priorities might differ from those of their children (Ajayi et al., 2017; Steinberg et al., 2009). It also could be that information affects families’ preferences, but those effects are not evident in data available to researchers (which may not show rank-ordered requests) or do not result in different placements or outcomes (e.g., because of capacity constraints in desirable schools).

We designed this study with hopes of approximating the strongest impact that one might realistically expect from an authentic, low-cost, school choice information intervention. We partnered with a school district and provided information to all families for whom we had contact information—in a city in which choice is not restricted to the most engaged families. We contacted families via U.S. mail, text message, and email to increase the likelihood that we reached them and the number of ways in which they heard from us. We targeted materials to parents/guardians, in case adolescents’ school preferences are erratic or unpredictable. For the growth group, we provided simplified information with a useful metric (growth) that hardly any recipient likely knew before receiving our materials. By incorporating a distance treatment along with that growth treatment, we provide context for the effects of the growth treatment. Finally, we examined applicants’ rank-ordered preferences, which provides a fuller picture of the interventions’ effects on applicant behaviors and placements.

Policy Context

In the aftermath of Hurricane Katrina in 2005, the New Orleans public school system underwent drastic changes. The state of Louisiana quickly accelerated its efforts to take control of low-performing public schools in New Orleans, transferring control of the vast majority of schools from the city-controlled OPSB to the state-controlled Recovery School District (RSD). The RSD opted for a portfolio model whereby it would authorize, oversee, and provide support to schools while leaving school leaders with substantial operational autonomy. Rather than being assigned to schools based on where they live, families could request seats in schools across the city. Within a few years, what had been a conventional urban school district became the country’s most decentralized, charter-based system.

The radical decentralization of the immediate post-Katrina school system created an assortment of problems, many of which related to the application and enrollment processes (Buerger & Harris, 2015). For example, families struggled to navigate a complex set of school-specific application processes. To address these problems, the city adopted a common application system that later evolved into the OneApp. This system centralized both the application process, by allowing parents to apply to multiple schools with one form, and the enrollment process, by assigning students to schools based on a placement algorithm. One role for government in this system has been to provide information about school offerings and performance to the public in hopes that families will make informed choices for their children.

The OneApp, like the broader school system in New Orleans, has evolved over time. At the time of this RCT (requests made for the 2019–2020 school year), families could request up to 12 schools for K–12 grades and eight for early childhood, ranked in order of preference. The system had two rounds: the Main Round, when most requests and placements occurred (including virtually all placements in the most popular schools), and Round 2 (K–12 only) for families who did not get a placement or were not satisfied with the placement they received. After the OneApp was complete, OPSB also managed a process called Late Enrollment as a first-come, first-served option for schools with seats still available.

For K–12, families could request seats in virtually any public school in New Orleans through OneApp. ³ Income-eligible families could also request seats in private schools that participated in the Louisiana Scholarship Program, a statewide private school voucher program. Many schools gave priority access for a subset of their seats to students who lived in the school’s geographic zone or within a half-mile walk zone. However, at least half of the open seats in every school did not offer any geographic priority. The algorithm also considered other priorities, such as having a sibling enrolled in the school. Importantly for the purposes of this article, the algorithm was strategy-proof in the sense that applicants’ had incentive to rank schools in their true order of preference rather than attempt to game the algorithm in some way (e.g., by ranking less-preferred, higher probability schools first).

For early childhood, families could request seats in an assortment of publicly funded programs, including school-based pre-Ks, Head Start programs, and private learning centers. These seats were available tuition free only to families whose income did not exceed the qualifying limit. In nearly all schools, obtaining a seat in a school-based pre-K offered the additional perk of being guaranteed a seat in that school’s kindergarten program in a subsequent year by virtue of being a continuing student. This early childhood choice process also happened through OneApp, with the need to demonstrate eligibility creating additional steps for applicants (Weixler et al., 2020).

In the year before our intervention, our data show that about half of Main Round applicants to entry grades (pre-K, kindergarten, and grade 9) applied to three or fewer programs, with 12% of the entry-grade applicants going unmatched. It is possible that these applicants were well informed and, for example, would only enroll in a few OneApp schools before looking for other options (e.g., private schools). However, earlier research on the OneApp found that most applicants who did not receive Main Round placements subsequently requested schools that they could have requested in the Main Round but did not (Harris et al., 2015). Notably, too, although OneApp requests indicate overall preferences for schools closer to home and higher rated schools (Harris & Larsen, 2019), many applicants do not request any high-growth, high-rated, or in-zone schools. In the year prior to our intervention, only about one-fourth of pre-K applicants and half of kindergarten applicants requested at least one high-growth school, while only about half of kindergarten applicants requested at least one school in their home geographic zone. These, too, could be intentional decisions that reflect parents’ preferences. However, New Orleans has high rates of student mobility (a possible indicator of dissatisfaction), and applicants matched to a high-scoring or high-growth school in our data were less likely than other students to switch schools before October 1. ⁴ It stands to reason that families could benefit from receiving more information about high-performing and close-to-home schools.

Data

This study uses several datasets from OPSB and the Louisiana Department of Education (LDOE). With data from OneApp applications and school enrollments in prior years, the district identified the students who, with normal grade progression, would enter a 4-year-old pre-K program, kindergarten, or ninth grade in the fall of 2019. Prior OneApp submissions were our primary source of information about children’s ages and families’ contact information. For example, most of the kindergarten sample consisted of 4-year-olds enrolled in New Orleans pre-K programs who had applied for those programs via the OneApp in the prior year. We supplemented this sample with families that had applied for pre-K in that same cycle but did not enroll and families that applied for an ECE seat via the OneApp in an earlier year. We focused on applicants to pre-K, kindergarten, and grade 9 because these are the primary school entry grades in New Orleans (which has few standalone middle schools). Since the addresses and phone numbers in the district’s database were recorded exactly as entered by parents, we dropped unusable records (e.g., phone numbers missing a digit) and corrected seemingly obvious mistakes in other records (e.g., slightly misspelled street names). We discarded mailing addresses known to be more than 3 years old due to concerns that the family might have moved since the data were collected.

After administering the study, we obtained deidentified, applicant-level OneApp records for the 2019–2020 OneApp Main Round. These records included the schools that families in the study requested (ranked in order of preference), along with students’ subsequent placements. These files also showed applicants’ priority status, including any schools in which they were guaranteed a seat because they were enrolled in the prior year.

We merged these files with student-level records from LDOE from the 2018–2019 school year (concurrent with the time of treatment being administered). We used the files to obtain information on students’ background characteristics. Specifically, we observe race/ethnicity, eligibility for free or reduced-price lunch (FRPL), special education status, and gender for students who had been enrolled in a Louisiana public school. For applicants who had not previously enrolled in a Louisiana public school, we supplemented LDOE’s records with information available in the OneApp files. As part of the OneApp process, many parents reported their children’s special education status and their family income (primarily to determine eligibility for publicly funded early childhood seats). We used a binary variable for special education status that includes all classifications except for gifted status. ⁵ For family income, we used LDOE’s FRPL variable where available and supplemented it with FRPL eligibility that we calculated (as the district does) based on information that families reported about their household size and monthly income. The OneApp did not ask parents to report their children’s race, ethnicity, or gender, so our only information on those characteristics comes from LDOE records.

Table 1 shows demographic information for students in our analytic sample, disaggregated by grade level and treatment condition. We have background information available for a little more than 80% of the incoming ninth-grade students, as most of these students were enrolled in a Louisiana public school in the prior year. ⁶ Our background information is more limited for students entering kindergarten and (especially) pre-K, as many of these students were new to the public school system. Information on children’s race/ethnicity and gender, which come only from LDOE records, is available for only about 17% of pre-K applicants and 61% of kindergarten applicants. Information on family income and special education status is available for about three fourths of pre-K applicants and 90% of kindergarten applicants.

OPEN IN VIEWER

Table 1

Descriptive Statistics and Randomization Balance

	Full sample			Prekindergarten			Kindergarten			Grade 9
Variable	Control	Growth—control	Distance—control	Control	Growth—control	Distance—control	Control	Growth—control	Distance—control	Control	Growth—control	Distance—control
Free/reduced-price lunch
Yes	0.625	−0.017	0.003	0.656	−0.024	0.011	0.657	−0.040*	−0.019	0.578	0.011	0.017
		(0.015)	(0.015)		(0.030)	(0.029)		(0.024)	(0.023)		(0.023)	(0.023)
No	0.199	0.005	−0.004	0.096	0.026	−0.009	0.234	0.020	0.011	0.228	−0.018	−0.012
		(0.012)	(0.012)		(0.019)	(0.018)		(0.022)	(0.021)		(0.019)	(0.020)
Unknown	0.176	0.012	0.000	0.248	−0.002	−0.002	0.109	0.020	0.007	0.195	0.007	−0.005
		(0.011)	(0.011)		(0.027)	(0.027)		(0.016)	(0.015)		(0.018)	(0.019)
Special education
Yes	0.085	0.011	−0.008	0.068	0.004	−0.007	0.081	−0.007	−0.015	0.097	0.027*	−0.001
		(0.009)	(0.008)		(0.016)	(0.015)		(0.014)	(0.013)		(0.014)	(0.014)
No	0.752	−0.023*	0.007	0.699	−0.007	0.012	0.817	−0.013	0.005	0.723	−0.032	0.007
		(0.013)	(0.013)		(0.029)	(0.028)		(0.020)	(0.019)		(0.021)	(0.021)
Unknown	0.163	0.011	0.000	0.233	0.004	−0.005	0.101	0.019	0.010	0.180	0.004	−0.006
		(0.011)	(0.011)		(0.026)	(0.026)		(0.016)	(0.015)		(0.018)	(0.018)
Race/ethnicity
Black	0.486	0.003	−0.015	0.129	−0.002	−0.014	0.473	0.008	−0.014	0.703	−0.019	−0.007
		(0.015)	(0.015)		(0.021)	(0.020)		(0.025)	(0.024)		(0.021)	(0.022)
White	0.052	0.004	0.002	0.028	−0.006	−0.010	0.070	0.005	0.005	0.049	0.009	0.007
		(0.007)	(0.007)		(0.010)	(0.009)		(0.013)	(0.013)		(0.010)	(0.010)
Hispanic	0.042	0.002	0.008	0.006	0.000	0.002	0.051	0.006	0.015	0.054	0.000	0.007
		(0.006)	(0.006)		(0.005)	(0.005)		(0.011)	(0.012)		(0.010)	(0.011)
Other	0.013	0.003	0.002	0.006	−0.006*	0.002	0.017	0.002	0.006	0.013	0.008	−0.002
		(0.003)	(0.003)		(0.003)	(0.005)		(0.007)	(0.007)		(0.006)	(0.005)
Unknown	0.408	−0.012	0.003	0.832	0.013	0.020	0.388	−0.021	−0.013	0.182	0.002	−0.005
		(0.014)	(0.015)		(0.023)	(0.023)		(0.024)	(0.024)		(0.018)	(0.018)
Gender
Female	0.309	−0.001	−0.010	0.087	−0.017	−0.020	0.306	0.014	0.005	0.438	−0.017	−0.012
		(0.014)	(0.014)		(0.017)	(0.016)		(0.023)	(0.023)		(0.022)	(0.023)
Male	0.285	0.012	0.008	0.079	0.006	0.006	0.309	0.005	0.005	0.381	0.013	0.017
		(0.013)	(0.013)		(0.017)	(0.017)		(0.023)	(0.023)		(0.022)	(0.023)
Unknown	0.406	−0.011	0.002	0.834	0.011	0.014	0.385	−0.019	−0.009	0.181	0.003	−0.006
		(0.014)	(0.015)		(0.023)	(0.023)		(0.024)	(0.024)		(0.018)	(0.018)
Observations	2,346	4,733	4,680	541	1,082	1,094	860	1,664	1,717	945	1,987	1,869

Note. Standard errors appear in parentheses. Table shows results of OLS regressions conducted at student level with standard errors clustered by home address. All values reported as proportions (of students). Special education classifications include students with an Individualized Education Program or Individualized Family Service Plan but exclude those with gifted status. OLS = ordinary least squares.

*

p < .10. **p < .05. ***p < .01.

Across these grade levels, large majorities of the students observed are Black and eligible for FRPL. This reflects New Orleans having disproportionate numbers of students of color and students in poverty—disproportionate even to the city’s youth population, since a large share of white and nonpoor students in New Orleans attend private schools (Weixler et al., 2017). An especially large share of pre-K applicants comes from low-income families, since family income is a criterion for most early childhood seats available in the OneApp.

Table 1 also illustrates that the treatment and control groups are well balanced with respect to observable student characteristics. Columns showing the differences in the background characteristics between the growth and the control groups, and distance and control groups, reveal no significant differences at p < .05.

Study Design

The basic design of this study is an RCT in which we randomly assigned families to one of three conditions: growth (emphasizing high-performing schools), distance (emphasizing schools close to home), or control. We sent flyers, text messages, and emails to parents/guardians in accordance with their treatment or control group assignment. ⁷

Treatment Conditions

We opted for growth- and distance-related treatments to address questions of interest to district leaders in New Orleans and the broader education research and policymaking communities. Testing a growth-related intervention was important in the context of unanswered questions about why parents’ school requests do not suggest more interest in schools with strong academic records. District leaders were especially curious about the impact of the state’s newly released growth scores. These local leaders also encouraged us to test a distance condition, because of ongoing uncertainty about whether parents are more interested in—and would be better served by—schools close to home or higher performing schools that are farther away. ⁸ In New Orleans, keeping track of the public schools close to one’s home is harder than in many other cities, since students are not assigned to schools based on their home address and schools frequently open, close, and relocate. This meant that providing information about schools close to home was not necessarily just reminding families of schools they already knew. Furthermore, incorporating both a distance treatment and a growth treatment created an opportunity that their respective effect estimates could give context to one another.

The “growth” group received lists of the highest performing schools in New Orleans (for kindergarten and grade 9), as well as a note that enrolling in schools where students learn a great deal is very important to some families. For pre-K, we identified the 10 programs available tuition free to eligible families that received the highest scores according to the CLASS. LDOE has adopted CLASS as its primary early childhood assessment program. CLASS measures the quality of teacher–child interactions in early childhood classrooms, drawing from the assessments of trained observers (Mashburn et al., 2008). The decision to highlight the highest scoring schools rather than providing data on all schools reflected district leaders’ preferences and our desire to keep from overloading families with information. We included the note about families’ priorities—and a parallel note for the distance group—primarily because the school district wanted to give recipients of this information a sense of why the district was highlighting certain schools (without directly suggesting that parents should value the highlighted characteristics).

For kindergarten and ninth grade, we benefited from fortuitous timing in the state’s first-ever release of school letter grades based only on student progress. These grades showed how schools performed on state-defined measures of student growth. In prior years, the state had released a School Performance Score (SPS) and accompanying letter grade, but these grades emphasized students’ achievement levels more than the degree to which students learned in those schools. ⁹ Schools with students who entered below grade level could make good progress in student learning but still receive a poor SPS grade because their children started so far behind. LDOE developed a new “progress index” (and accompanying letter grades) based primarily on student growth, not achievement levels. In the year of our intervention, it released both progress grades and SPS grades, and it did so almost concurrently with our interventions. Parents had not seen these grades before, and we do not believe the grades were widely publicized in New Orleans immediately upon release (aside from this study’s communications with the growth group). LDOE used a formula to calculate this “progress index” that considered students’ growth using the state’s value-added model (VAM), attainment of an “Advanced” score in tested subjects, and student-specific growth targets for reaching an Advanced score (LDOE, 2017, 2019). For elementary and middle school, the state used students’ math and English language arts (ELA) scores on the Louisiana Educational Assessment Program. For high school, it used students’ math and ELA scores on end-of-course assessments. To better understand what this progress index measures, we checked its correlation with school value-added scores that our research team had created for prior studies using more conventional VAMs (see Harris & Liu, 2021, for a description of these VAMs). We found correlations of 0.80 for elementary/middle schools and 0.87 for high schools. We believe these grades largely captured school value-added gains in math and ELA.

Appendix A shows how the schools in this study performed on the state’s progress measure relative to its previously created measures. In general, schools received better grades on the new progress measure, and many schools looked considerably different on this measure (2018 progress score) than the measure that had been available prior to its release (2017 SPS). The growth group’s communications highlighted all schools that received an “A” grade in student progress. This was 17 schools offering kindergarten and six schools offering ninth grade. ¹⁰ Private pre-K programs received CLASS scores, so we included private pre-K programs that offered publicly funded tuition-free seats in the pre-K informational materials (for both the treatment and distance groups). Private schools did not receive LDOE student progress scores, so we did not include them in the kindergarten or ninth-grade materials (for either the treatment or the distance group).

The “distance” group received lists of schools in their home geographic zones. This information could affect families’ requests or placements in a variety of ways. It could persuade them to weigh distance from home more heavily in their decision-making. Of course, many families already valued proximity to home, and similar studies from New Orleans (Harris & Larsen, 2019) and Washington, D.C. (Glazerman & Dotter, 2017) suggest that New Orleans families might place especially strong emphasis on proximity. The distance information also could increase the salience of schools already known to families, or make families aware of schools they did not know or did not realize were so close to (or far from) home. This is especially plausible in New Orleans, where schools open, close, and relocate frequently.

For the distance group materials, we used the seven zones the district created to determine geographic preferences for the subset of schools—and the subset of seats within those schools—that had those preferences. Families in the distance group received a complete list of schools in their geographic zone. For the flyers, we identified their zone by the address that we had on file (the address to which we sent the flyers). For the text messages, we invited families to reply with their zip code to receive a message showing the schools in that zip code’s geographic zone. ¹¹ For the emails, we displayed a zone map and a list of all of the schools in each zone. We chose not to prepopulate the school lists on the text messages and emails because we expected that some of our mailing addresses were no longer current and we would be sending list of schools in a zone where the family no longer lives. ¹² In total, across the seven geographic zones, 78 pre-K, 55 kindergarten, and 27 ninth-grade programs appeared in the distance group’s materials. ¹³

The control group received a set of flyers, text messages, and emails parallel to those sent to the growth and distance groups, but the control group materials did not identify or highlight any specific schools. Rather, these materials reminded families of the OneApp deadline and website. We did this to neutralize effects from the growth or distance treatments that might arise from just reminding (or notifying) people of the deadline.

Of course, members of the control group—like members of the growth and distance groups—might have utilized information other than what we provided through this study. Many families likely used “informal” sources of information, such as word-of-mouth recommendations from friends. In New Orleans, “formal” sources of school information included the OPSB and LDOE websites and the OneApp itself. The OPSB website showed school profiles with an assortment of information that included the geographic zone, state letter grade (not the student progress grade shown on the intervention materials), and lists of school offerings. The LDOE website had profiles with more detail on the letter grades, including progress scores, and less detail on the school programs. The Urban League of Louisiana published guides to support families choosing ECE programs and high schools and hosted its annual school exposition. The OneApp application portal—which applicants saw, although sometimes only to enter already-determined choices—contained information as well. All applicants saw a list of schools within their geographic zone and, if the applicant added a school to her list of requests, a few details about that school, including its growth letter grade and overall letter grade. Although most applicants only saw this information from the OneApp portal very late in the process, it could have mitigated the effects of the treatments. More generally, New Orleans is perhaps a more information-rich school choice environment than most U.S. cities, which is important for purposes of generalizability. Notably, however, this intervention provided information about specific schools to parents in very direct ways that likely stood out even in a relatively information-rich setting.

Figure 1 displays examples of the flyers. These two-sided, color flyers were identical on one side for the growth, distance, and control groups (within the same grade). The other side contained group-specific information. This includes a list of high-performing schools and their scores for the growth group, and a map and list of in-zone schools for the distance group. Figure 2 displays the emails. We designed these emails to mirror, in style and content, the flyers. Figure 3 displays the text messages. These, too, align with the content of the other materials, although the limitations of text message communication kept us from mirroring the style and color. All of these materials were addressed from the district, not the research team.

OPEN IN VIEWER

Figure 1.

Samples of flyers (ninth grade).

OPEN IN VIEWER

Figure 2.

Samples of emails (kindergarten).

OPEN IN VIEWER

Figure 3.

Samples of text messages (all grades).

Sample

Table 2 shows how many students the district attempted to contact for the study (by the various modes of communication) and how many students ultimately had a OneApp Main Round application. Across all grades, we attempted to contact the families of 9,829 students. Of those students, 7,265 appeared in the Main Round. Those who did not appear might have enrolled in private school without submitting a OneApp, waited until after the Main Round to attempt to obtain a seat, or moved out of New Orleans, among other possibilities. Of the students who did appear, 7,067 had usable application records (e.g., applied for the grade level we anticipated), meaning that about 72% of the students initially contacted ultimately appeared in our analytic sample. ¹⁴ The share was larger for high school applicants (85%) than kindergarten and pre-K applicants (70% and 58%, respectively).

OPEN IN VIEWER

Table 2

Sample Sizes by Grade

Variable	Total	Pre-kindergarten	Kindergarten	Grade 9
Contacted for study	9,829	2,795	3,598	3,436
Submitted OneApp in Main Round	7,265	1,819	2,531	2,915
Had usable application data	7,067	1,635	2,521	2,911
District sent flyer	7,067	1,635	2,521	2,911
District sent flyer and text message	6,986	1,621	2,495	2,870
District sent flyer, text message, and email	5,626	1,599	2,461	1,566
Not contacted for study but applied	2,897	809	1,231	857

Note. Sample sizes reported in number of students. Students with and without guaranteed seats included in sample. Analytic sample consists of those in “Had usable application data” row.

We used mailing addresses as the basis of our randomization, so the district sent a flyer to all of these families. It sent a text message to 99% of them, and the full set of correspondences—flyer, text message, and email—to about 80% (email records were more limited, especially for potential high school applicants). As we cleaned the mailing addresses, phone numbers, and email addresses before attempting contact, only a small number of attempted communications bounced back to the district. Our records indicate that fewer than 1% of the flyers and 5% of the emails failed to send. ¹⁵ However, we cannot know what percentage of the intended recipients actually received and read the materials.

Another 2,897 students appeared in the OneApp data without us initiating contact (e.g., because the applicants were new to New Orleans public schools). ¹⁶ These numbers reflect the challenges of identifying, in advance, which families will apply for an upcoming year and having usable contact information for those families. This is a particular challenge for families with young children who have not previously applied for, or enrolled in, a public school or ECE program. Still, even for pre-K and kindergarten, over two thirds of all applicants in the OneApp Main Round had been contacted for the study.

While the differences between which families the district contacted and which families actually applied do not threaten the RCT’s internal validity, it is worth considering their implications for external validity. It could be, for example, that families that recently moved to New Orleans are underrepresented in our analytic sample, or that families in the sample are unusually familiar with the OneApp process and New Orleans schools (having applied or enrolled before). If the study’s informational materials have stronger effects on families less familiar with their options, we might expect larger effects from the broader population of applicants than from this study sample. In addition, a disproportionate share of the OneApp’s early childhood seats had been available only to families living below the federal poverty line (typically those applying to Head Start programs). As a result, the district’s contact information for potential pre-K and kindergarten applicants likely consisted of a disproportionate number of families in poverty.

Analysis

As an RCT with two distinct treatment conditions and a control condition, the empirical models for this analysis are straightforward. We obtain intent-to-treat (ITT) estimates of the effects of disseminating information about schools. Although we have some data related to how many messages were returned to sender (very few), we cannot observe the consumption of these materials in a way that would allow us to estimate treatment-on-the-treated (TOT) effects.

We examine an assortment of outcome variables related to the structure of applicants’ ranked requests and subsequent placements. For each treatment, we are especially interested in whether applicants requested and received placements in the schools presented to them (high-growth schools for the growth treatment; in-geographic-zone schools for the distance treatment). When we analyze the effects of the growth treatment (on growth-related outcomes), we compare outcomes for the treatment and control groups, omitting the distance group from those analyses. When we analyze the effects of the distance treatment (on distance-related outcomes), we compare outcomes for the distance and control groups, omitting the growth group from those analyses. In other words, for the growth group analyses, we code the treatment variable as 1 for the growth group, 0 for the control group, and missing for the distance group. ¹⁷

We examine both the average number of highlighted schools that applicants requested and the proportion of applicants that requested at least one highlighted school. While we might see effects on the former outcome from a small number of applicants reacting strongly to the materials provided (i.e., requesting several additional highlighted schools), we likely would not see effects on the latter outcome unless a large number of applicants acted on the information provided. We also test whether the information led applicants to change their first-choice request to one of the listed schools. First-choice requests were especially important, since the OneApp’s strategy-proof algorithm considered applicants for their first-choice requests before moving to their second choice and subsequent requests. In our analytic sample, about 82% of pre-K and kindergarten applicants were assigned to their first-choice school, so affecting placements would be difficult without affecting first choices. For ninth-grade applicants, only about 62% were assigned to their first choice, leaving more room for impact via effects on lower ranked choices. In addition, we test whether the intervention led applicants to request more schools overall—regardless of whether they came from the list of highlighted schools—to assess whether additional requests for highlighted schools seem to supplement or replace other school requests. Finally, we examine applicants’ Main Round school placements.

Our model for estimating ITT effects is as follows:

Y i j = α + β ( T r e a t j ) + θ ( B l o c k j ) + X i j δ + ε i j ,

(1)

where Y i j represents the outcome for student i in family j (with family defined as students living at the same address), and T r e a t j indicates whether the family was in the treatment group (the growth group for growth-control comparisons; the distance group for distance-control comparisons). We include randomization-block fixed-effects, B l o c k j , to account for the fact that we conducted randomization separately for each of the seven geographic zones. We run models with and without a vector of covariates, represented by X i j . These covariates consist of students’ race/ethnicity, family income (based on eligibility for FRPL), special education status, gender, and grade level. In models with covariates, we code missing data as a distinct category to keep from dropping students with missing background information. We cluster the standard errors by family (home address) to account for some students in the study being members of the same family and therefore sharing the same treatment assignment.

In addition to presenting estimated effects for the full sample, we present estimates for notable subsamples. We show results for applicants who did not have a guaranteed seat in any school. Typically, students currently enrolled in a school—for example, students in a school-based pre-K that offered kindergarten—would have a guaranteed seat in that school in the following year. We disaggregate results for this group because applicants without a guaranteed seat might be less certain about their plans or familiar with their options, and therefore more likely to use the study’s informational materials. We also test for heterogeneity in treatment effects across students of different backgrounds. We do this in two ways. First, we test for significant interactions between student background characteristics and treatment status for each of the demographic variables, treatments, and outcomes. (These results appear in the “Results” section text but not tables.) Second, we report treatment effects disaggregated by students’ family income, special education status, race/ethnicity, and gender.

For ease of interpretation, we describe results from ordinary least squares (OLS) regression models even for dependent variables with binary outcomes (and report results from logistic regression models in an appendix table). The OLS and logit models produce similar results in sign and significance.

Results

We examine how the growth treatment affected growth-related outcomes and the distance treatment affected distance-related outcomes. ¹⁸ Specifically, we assess: whether applicants requested any of the (high-growth or in-zone) schools listed in the informational materials they received, how many of the listed schools they requested, whether their first-choice school was listed, and whether they were assigned to a listed school. In general, we find the strongest effects on whether applicants requested listed schools and how many of those schools they requested.

Interpreting these findings would have been more complicated if the treatments affected whether families submitted a OneApp. That was not the case. We observed no significant differences in the probabilities that applicants assigned to the growth, distance, and control conditions submitted a Main Round application. ¹⁹ If the growth and distance materials affected the probability that families would submit an application in the first place (e.g., by reminding parents of the upcoming deadline), it appears that the control materials succeeded in neutralizing that effect.

Full Sample (and by Grade Level)

Table 3 provides the estimated effects of the growth treatment (comparing growth with control). Results are presented in separate columns for models with and without covariates, and separate panels for the full sample of applicants (Panel A) and the subset that did not have a guaranteed seat in any school (Panel B). Although one might expect more precise estimates from models with covariates and stronger effects for students without a guaranteed seat (who must choose a new school and might be more drawn to informational materials), the results are similar. Given that, this section presents results from the most straightforward analyses: full-sample results from models without covariates.

OPEN IN VIEWER

Table 3

Effects of Growth Treatment

	Control				Growth minus control (no covariates)				Growth minus control (covariates)
Variable	All	Pre-kindergarten	Kindergarten	Grade 9	All	Pre-kindergarten	Kindergarten	Grade 9	All	Pre-kindergarten	Kindergarten	Grade 9
Panel A: All students
Any high-growth school request(s)	0.579	0.216	0.555	0.807	0.027*	−0.029	0.018	0.038**	0.019	−0.028	0.021	0.039**
					(0.015)	(0.024)	(0.025)	(0.017)	(0.013)	(0.024)	(0.024)	(0.017)
High-growth schools requested (#)	1.06	0.29	1.23	1.34	0.09**	−0.02	0.10	0.12**	0.09**	−0.02	0.12	0.12**
	[1.33]	[0.63]	[1.72]	[1.01]	(0.04)	(0.04)	(0.09)	(0.05)	(0.04)	(0.04)	(0.08)	(0.05)
First choice is high-growth school	0.299	0.055	0.394	0.355	0.008	−0.002	−0.010	0.022	0.007	0.000	−0.009	0.020
					(0.014)	(0.014)	(0.024)	(0.021)	(0.013)	(0.014)	(0.024)	(0.021)
Assigned to high-growth school	0.233	0.070	0.319	0.248	0.014	0.012	−0.022	0.041**	0.015	0.014	−0.022	0.040**
					(0.012)	(0.016)	(0.022)	(0.020)	(0.012)	(0.016)	(0.022)	(0.020)
Total schools requested (#)	4.09	3.27	3.19	5.37	0.20*	−0.22	0.28	0.19	0.15	−0.26	0.33**	0.20
	[3.42]	[2.68]	[3.40]	[3.42]	(0.10)	(0.17)	(0.17)	(0.16)	(0.10)	(0.17)	(0.16)	(0.16)
Observations (maximum)	2,346	541	860	945	4,733	1,082	1,664	1,987	4,733	1,082	1,664	1,987
Panel B: Students without guaranteed seat
Any high-growth school request(s)	0.745	0.370	0.828	0.840	0.022	−0.052	0.015	0.030*	0.013	−0.047	0.017	0.029*
					(0.015)	(0.038)	(0.027)	(0.016)	(0.014)	(0.038)	(0.028)	(0.016)
High-growth schools requested (#)	1.43	0.50	2.14	1.42	0.09*	−0.04	0.12	0.11**	0.09*	−0.03	0.14	0.10**
	[1.42]	[0.76]	[2.01]	[0.99]	(0.05)	(0.06)	(0.14)	(0.05)	(0.05)	(0.06)	(0.14)	(0.05)
First choice is high-growth school	0.342	0.093	0.497	0.359	0.016	−0.007	0.006	0.021	0.012	−0.003	0.008	0.018
					(0.017)	(0.024)	(0.036)	(0.022)	(0.016)	(0.025)	(0.035)	(0.022)
Assigned to high-growth school	0.245	0.118	0.334	0.247	0.029*	0.020	−0.007	0.044**	0.028*	0.024	−0.009	0.044**
					(0.015)	(0.027)	(0.032)	(0.021)	(0.015)	(0.028)	(0.032)	(0.021)
Total schools requested (#)	5.44	4.87	5.35	5.69	0.12	−0.31	0.28	0.14	0.10	−0.30	0.30	0.13
	[3.29]	[2.45]	[3.68]	[3.33]	(0.12)	(0.21)	(0.26)	(0.16)	(0.12)	(0.21)	(0.26)	(0.16)
Observations (maximum)	1,623	314	428	881	3,335	618	851	1,866	3,335	618	851	1,866

Note. Standard deviations appear in brackets. Standard errors appear in parentheses. Table shows results of OLS regressions conducted at student level with standard errors clustered by home address. All values reported as proportions unless otherwise indicated with number sign. Covariates consist of race/ethnicity, family income (FRPL status), special education status, gender, and grade level. OLS = ordinary least squares; FRPL = free or reduced-price lunch.

*

p < .10. **p < .05. ***p < .01.

Across all grade levels, the growth treatment led to a 2.7 percentage point (4.7%) increase in the probability of requesting at least one high-growth school (p < .1). It led applicants to request 0.09 (8.5%) more high-growth schools on average (p < .05), while requesting about 0.2 (4.9%) more schools overall (p < .1). These effects came largely from high school applicants. High school applicants in the growth group were 3.8 percentage points (4.7%) more likely to request at least one high-growth school, requested 0.12 (9.0%) more high-growth schools on average, and were 4.1 percentage points (16.5%) more likely to be assigned to a high-growth school (p < .05 in all cases). We do not observe statistically significant effects, for the full sample or any particular grade, on the probability of requesting a high-growth school as the applicant’s first choice. We also do not observe any significant effects on the applications for pre-K or kindergarten seats.

Table 4 presents a parallel set of estimates for the distance treatment (comparing distance with control). Here, too, the results are similar regardless of whether the models include covariates and the sample is restricted to students without a guaranteed seat. We see little impact from the distance treatment. The only significant effect is on the probability that treated students requested at least one in-zone school. For the full sample, the distance treatment led to a 2.8 percentage point (4.2%) increase in the probability of requesting at least one in-zone school (p < .05). By grade level, this was statistically significant only for kindergarten applicants (p < .1), although we observe positive coefficients for pre-K and ninth grade as well.

OPEN IN VIEWER

Table 4

Effects of Distance Treatment

	Control				Distance minus control (no covariates)				Distance minus control (covariates)
	All	Prekindergarten	Kindergarten	Grade 9	All	Prekindergarten	Kindergarten	Grade 9	All	Prekindergarten	Kindergarten	Grade 9
Panel A: All students
Any in-zone school request(s)	0.661	0.637	0.635	0.697	0.028**	0.013	0.041*	0.027	0.028**	0.016	0.040*	0.026
					(0.014)	(0.030)	(0.023)	(0.020)	(0.014)	(0.030)	(0.023)	(0.020)
In-zone schools requested (#)	1.21	1.34	1.21	1.13	0.06	0.01	0.08	0.06	0.06	0.02	0.08	0.06
	[1.41]	[1.64]	[1.60]	[1.02]	(0.04)	(0.10)	(0.08)	(0.04)	(0.04)	(0.10)	(0.08)	(0.04)
First choice is in-zone school	0.444	0.498	0.532	0.336	0.006	0.014	0.012	−0.006	0.003	0.015	0.008	−0.007
					(0.015)	(0.032)	(0.025)	(0.020)	(0.014)	(0.032)	(0.025)	(0.020)
Assigned to in-zone school	0.411	0.451	0.472	0.333	0.003	−0.014	0.023	−0.009	0.000	−0.015	0.018	−0.009
					(0.014)	(0.030)	(0.024)	(0.021)	(0.014)	(0.030)	(0.024)	(0.021)
Total schools requested (#)	4.09	3.27	3.19	5.37	0.03	0.07	−0.01	0.07	0.06	0.11	0.04	0.06
	[3.42]	[2.68]	[3.40]	[3.42]	(0.10)	(0.17)	(0.16)	(0.16)	(0.10)	(0.17)	(0.16)	(0.16)
Observations (maximum)	2,346	541	860	945	4,680	1,094	1,717	1,869	4,680	1,094	1,717	1,869
Panel B: Students without guaranteed seat
Any in-zone school request(s)	0.730	0.774	0.719	0.719	0.021	−0.017	0.043	0.019	0.020	−0.014	0.036	0.018
					(0.015)	(0.033)	(0.030)	(0.020)	(0.015)	(0.033)	(0.030)	(0.020)
In-zone schools requested (#)	1.51	1.96	1.84	1.19	0.06	−0.09	0.10	0.05	0.05	−0.07	0.06	0.05
	[1.54]	[1.82]	[1.99]	[1.02]	(0.05)	(0.14)	(0.13)	(0.05)	(0.05)	(0.14)	(0.13)	(0.05)
First choice is in-zone school	0.414	0.537	0.499	0.335	−0.003	−0.008	0.002	−0.007	−0.005	−0.005	0.001	−0.007
					(0.017)	(0.040)	(0.035)	(0.020)	(0.017)	(0.040)	(0.036)	(0.020)
Assigned to in-zone school	0.373	0.462	0.383	0.336	−0.004	−0.054	0.031	−0.010	−0.006	−0.054	0.023	−0.010
					(0.017)	(0.038)	(0.033)	(0.022)	(0.017)	(0.038)	(0.033)	(0.022)
Total schools requested (#)	5.44	4.87	5.35	5.69	−0.04	−0.02	−0.11	0.01	−0.03	−0.01	−0.12	−0.01
	[3.29]	[2.45]	[3.68]	[3.33]	(0.12)	(0.20)	(0.25)	(0.16)	(0.12)	(0.20)	(0.25)	(0.16)
Observations (maximum)	1,623	314	428	881	3,272	651	865	1,756	3,272	651	865	1,756

Note. Standard deviations appear in brackets. Standard errors appear in parentheses. Table shows results of OLS regressions conducted at student level with standard errors clustered by home address. All values reported as proportions unless otherwise indicated with number sign. Covariates consist of race/ethnicity, family income (FRPL status), special education status, gender, and grade level. OLS = ordinary least squares; FRPL = free or reduced-price lunch.

*

p < .10. **p < .05. ***p < .01.

Appendix B shows results from logistic regression models for the binary outcome variables (requested high-growth/in-zone school, first choice was high-growth/in-zone school, and placed in high-growth/in-zone school). These estimates, reported in odds ratios, are similar in sign and significance to the results from Tables 3 and 4. In addition, we explored whether the kindergarten or ninth-grade materials, which did not include private schools, might have led applicants to request fewer private schools via the OneApp. We find no significant differences across groups in the percent requesting a private school as a first choice (3.3% of the growth group, 3.2% of the distance group, 4.0% of the control group), nor in the percent requesting at least one private school (5.0% growth, 4.5% distance, 5.4% control). Moreover, with fewer than 10% of applicants filling their applications with 12 schools, it is unlikely that the information led to a crowding out at the bottom of many families’ rank-ordered request lists.

Heterogeneity of Effects

Next, we assess heterogeneity in treatment effects across student subgroups. We focus on applicants to kindergarten and ninth grade, since our demographic information for pre-K applicants is limited. The specific subgroups we examine—by family income, special education status, race/ethnicity, and gender—correspond to the groups reported in Table 1. In general, we find reasonably consistent effects across subgroups with one notable exception: the growth treatment (and perhaps distance treatment) had strong effects on the application behaviors of families of SWDs.

To assess heterogeneity, we first tested for statistically significant interactions between students’ background characteristics and treatment status, looking separately at each treatment and outcome. We found six interaction coefficients significant at p <.05. Four of these involve comparisons of treatment effects for applicants with and without disabilities. In each of these cases, effects were stronger for families of SWDs. The growth treatment elicited a stronger response for SWDs (than other applicants) on: the probability of requesting at least one high-growth school; the number of high-growth schools requested; and the total number of schools requested. The distance treatment elicited a stronger response for SWDs (than other applicants) on the total number of schools requested. The only other significant interactions suggest that the distance treatment had a stronger effect on the total number of schools requested for males than females, and that the distance treatment was more likely to lead families eligible for FRPL (than families ineligible for FRPL) to rank in-zone schools first.

Table 5 shows the effects of the growth treatment (Panel A) and distance treatment (Panel B) disaggregated by subgroups. In this table, unlike the analyses described in the preceding paragraph, asterisks indicate that the treatment estimate for a particular subgroup is statistically significant from zero. Here, too, the most striking effects come from families of SWDs. For example, the growth treatment produced a 15.5 percentage point increase in the probability that families of SWDs requested at least one high-growth school, led these applicants to request an additional 0.5 high-growth schools overall, and increased their total number of schools requested by 1.6. Each of these estimates is significant at p < .01. Evidence of heterogeneity for other groups is sparse. We observe more statistically significant effects for Black students than White or Hispanic students, but this might just reflect differences in sample size. Tests of the interactions between student race and treatment status do not produce significant effects on any outcome. Table 5 also contains more significant estimates for male students than female students from the growth treatment. However, without statistically significant interactions—or corresponding findings from the distance treatment—we are reluctant to draw strong conclusions from this.

OPEN IN VIEWER

Table 5

Heterogeneity of Effects (Kindergarten and Grade 9 Applicants)

	Free/reduced-price lunch		Special education		Race/ethnicity			Gender
Variable	Yes	No	Yes	No	Black	White	Hispanic	Female	Male
Panel A: Growth treatment and outcomes
Any high-growth school request(s)	0.033*	0.039	0.155***	0.025	0.051**	0.008	0.040	0.025	0.068***
	(0.020)	(0.032)	(0.051)	(0.018)	(0.020)	(0.064)	(0.069)	(0.025)	(0.026)
High-growth schools requested (#)	0.089	0.062	0.498***	0.055	0.159***	0.102	−0.126	0.079	0.182***
	(0.058)	(0.109)	(0.152)	(0.054)	(0.052)	(0.189)	(0.164)	(0.065)	(0.068)
First choice is high-growth school	0.013	−0.013	−0.012	0.011	0.004	0.001	0.065	0.020	0.004
	(0.020)	(0.034)	(0.051)	(0.018)	(0.020)	(0.065)	(0.071)	(0.025)	(0.027)
Assigned to high-growth school	0.008	0.026	0.068	0.010	0.015	−0.007	0.035	0.006	0.032
	(0.019)	(0.032)	(0.046)	(0.018)	(0.019)	(0.062)	(0.073)	(0.024)	(0.026)
Total schools requested (#)	0.291*	0.192	1.637***	0.129	0.411***	0.506	−0.336	0.157	0.536***
	(0.159)	(0.246)	(0.402)	(0.141)	(0.158)	(0.423)	(0.495)	(0.197)	(0.201)
Observations (maximum)	2,221	830	346	2,700	2,153	221	197	1,361	1,290
Panel B: Distance treatment and outcomes
Any in-zone school request(s)	0.023	0.054*	0.039	0.033*	0.042**	0.001	−0.015	0.062**	0.002
	(0.019)	(0.033)	(0.055)	(0.017)	(0.020)	(0.062)	(0.064)	(0.025)	(0.025)
In-zone schools requested (#)	0.050	0.184*	0.268**	0.069	0.126**	0.193	−0.108	0.125**	0.106
	(0.053)	(0.102)	(0.119)	(0.051)	(0.050)	(0.157)	(0.142)	(0.059)	(0.066)
First choice is in-zone school	−0.024	0.058*	0.002	0.002	0.002	−0.041	−0.008	0.039	−0.031
	(0.020)	(0.034)	(0.055)	(0.018)	(0.020)	(0.067)	(0.067)	(0.025)	(0.027)
Assigned to in-zone school	−0.003	0.026	0.001	0.009	0.007	−0.031	−0.048	0.034	−0.030
	(0.020)	(0.034)	(0.055)	(0.018)	(0.020)	(0.065)	(0.068)	(0.026)	(0.027)
Total schools requested (#)	−0.066	0.375	1.092**	−0.056	0.194	0.626	−0.542	−0.067	0.341*
	(0.157)	(0.249)	(0.424)	(0.139)	(0.157)	(0.480)	(0.492)	(0.197)	(0.201)
Observations (maximum)	2,208	813	308	2,765	2,108	221	208	1,329	1,263

Note. Standard errors appear in parentheses. Table shows results of OLS regressions conducted at student level with standard errors clustered by home address. All values reported as proportions unless otherwise indicated with number sign. Students with and without guaranteed seats included in sample. Models do not include covariates. OLS = ordinary least squares.

*

p < .10. **p < .05. ***p < .01.

Appendices C and D further disaggregate the results in Table 5 for applicants to kindergarten and applicants to ninth grade, respectively. Below, we consider possible explanations for why these informational materials appear to have produce stronger responses in applications for SWDs than other applications.

Conclusion

Market-based theories of school choice assume that families have and use high-quality information. If families are uninformed or misinformed about their options (or how the choice process works), they might not request and receive seats in the schools they would prefer. This could help to explain differences researchers have found between applicants’ stated preferences (which suggest an intense focus on academic quality) and revealed preferences (which do not; Stein et al., 2010). Information limitations could have negative consequences for students and schools alike.

For this study, we partnered with school district leaders in New Orleans to conduct a school choice information RCT. We attempted to design and administer the treatments in ways that address the various challenges related to informing families and measuring the effects. We sent materials directly to families using multiple modes of communication—U.S. mail, email, and text messages. The information was simple in format but, in the case of the growth treatment, more reflective of program quality than most other information available. Due to fortuitous timing, the growth information was essentially new to all families, with potential to lead many parents to update their impressions of schools. Moreover, our data allow us to observe the full structure of applicants’ preferences, from their top choice through their 12th choice, and disaggregate by students’ background characteristics.

We found significant ITT effects from the growth treatment, with the growth group more inclined than the control group toward high-growth schools. However, those effects were concentrated among certain subgroups and on certain outcomes. Effects were stronger for high school than pre-K/kindergarten applicants. For example, high school applicants in the growth group were 3.8 percentage points (4.7%) more likely to request a high-growth school, requested 0.12 (9%) more high-growth schools on average, and were 4.1 percentage points (16.5%) more likely to be assigned to a high-growth school, compared with control group applicants. Comparing these effects to those arising from other school choice information RCTs is challenging because of differences in the settings, information conveyed, modes of communication, and outcomes studied. Broadly speaking, however, the findings from this RCT appear consistent with findings from RCTs in Charlotte-Mecklenburg (Hastings & Weinstein, 2008) and New York City (Corcoran et al., 2018).

One finding from this study that has not arisen from other information RCTs is that we observe stronger responses from families of SWDs. (If anything, Corcoran et al. (2018) seem to find stronger effects for families of students without disabilities.) Our finding seems compatible with qualitative research showing that many families of SWDs struggle to find useful school information, ultimately submitting similar school requests to other families despite reporting particular needs (McKittrick et al., 2020). Although this is speculative, it seems notable that this RCT, unlike many others, involved information about student growth. Perhaps growth data are particularly useful and relevant to families of SWDs, since growth data suggest how much students will learn regardless of where they begin, even if their starting point is well below grade level. Another possibility is that parents of SWDs are especially attuned to their children’s educational needs and responsive to information about it. This suggests a more general pattern, however, that has not yet been evident in other school choice information studies.

The distance treatment had minimal effects, although it produced a 2.8 percentage point (4.2%) increase in the probability that applicants requested at least one school in their geographic zone. We anticipated more modest effects from the distance treatment than the growth treatment, although information about school location can be informative in an evolving New Orleans school setting. In our view, the contrast between the growth and the distance treatment effects serves as a useful reminder that families do not respond to just any information about schools. They want information that addresses their needs and priorities.

Notably, we do not to find strong effects on applicants’ first-choice requests. This seems consistent with results from Ainsworth et al.’s (2020) school choice information RCT in Romania. Perhaps most applicants seriously consider only a few schools before stopping their searches, and they end up much clearer and better informed about their top choices than other schools. If information interventions are more likely to change applicants’ low-ranked schools than top-ranked schools, then the extent to which these interventions affect placements will depend on factors such as the degree of oversubscription. All else equal, we suspect that information will have stronger effects where lower proportions of applicants are placed in their first-choice programs. First-choice match rates differ across locales, with, for example, roughly half of ninth-grade applicants receiving a top-choice offer in Chicago (Barrow & Sartain, 2019), while over 80% received a top-choice offer in Denver (Denver Public Schools, 2019).

Another intriguing finding is that the effects were so concentrated among high school applicants. Prior research suggests that high school applicants respond in unpredictable ways to school choice information, perhaps because of the role that children play in choosing their own high schools (Ajayi et al., 2017; Corcoran et al., 2018). Future research could help to reveal the mechanisms by which this information operates. Researchers tend to describe school choice information as filling gaps in families’ knowledge about schools and the choice process. However, this is not the only way that information can affect choices. For example, presenting information that emphasizes academic quality could lead applicants to weigh academic quality more heavily (by signaling what they should value). This intervention’s note that some parents value academic quality might have produced a recalibration of families’ decision-making criteria. An additional possibility is that directing school choice materials to parents could have shifted the balance of some intra-family decision-making from children to their parents. If parents have different preferences than their children (e.g., greater concern about school academic performance), then a change in their roles in the decision-making process could yield different choices even without changes to any individual’s knowledge or criteria.

Although we cannot be certain of why the effects on pre-K and kindergarten were modest, some explanations seem plausible. Parents selecting (school-based) pre-K programs in New Orleans are often selecting their child’s school through eighth grade, which could limit the importance of a pre-K-specific rating. Many programs that scored highly on the state’s pre-K rating are not located in popular elementary schools, and parents may not have been willing to risk an undesirable elementary school placement for a more favorable pre-K placement. Also, certain practical considerations—such as commute time—could weigh heavier for young children, possibly limiting the role of school performance in decision-making (Harris & Larsen, 2019).

With so much variation in school choice policies and settings, it is important to consider the generalizability of any information intervention. New Orleans has a choice-based system without attendance boundaries that essentially requires all families to choose schools. On one hand, this suggests the families in this study might be especially adept and experienced with school choice, and perhaps unusually well informed about their options. On the other hand, the universality of school choice in New Orleans means that this intervention is not speaking to only the most engaged, savvy, or disillusioned families in the city—a subset of active choosers we might expect to find in other cities. Rather, the results come from a large proportion of the city’s public school population. Other considerations for generalizability include whether a city has a unified enrollment system and the extent to which some schools are oversubscribed.

Stepping back, we see this RCT’s results as further demonstration that providing information about academic quality can affect applicant behaviors, especially for certain groups of families. At the same time, we see reason to be realistic about how much influence these types of low-cost information interventions will have, and how likely they are to have a major impact on school choice markets. This holds even truer for more passive efforts to disseminate information, such as placing content on websites that many applicants will never visit. The reality is that school-choosing families—and particularly families in poverty—confront barriers that extend well beyond information issues. These include transportation challenges, burdensome application processes, and an inadequate supply of seats in desirable schools. Information can only go so far if that information serves to make families aware of the schools in which they cannot, or would not wish to, enroll their children.