Abstract
Prior research suggests that families prefer schools with higher test scores, shorter distances from home, and certain student demographics. We build on this using data from New Orleans, a context well suited to identification of parent preferences because of its deferred acceptance algorithm and extensive, standardized, and broadly accessible school information. This allows us to study revealed preferences for a richer set of characteristics. We find that families prefer schools with higher school value-added, more extracurricular activities, and after-school childcare. We also find heterogeneity by family income that is more consistent with income constraints than preference heterogeneity. Finally, we show how methodology and data shape the results.
Keywords
T
The way in which choice and competition affect the market for education depends on the characteristics of schools that families prefer. If families value school effectiveness with respect to academics, then there is the potential for schooling choices and competition to lead to improved school quality and better student outcomes (Beuermann et al., 2021; Campos & Kearns, 2022). However, if families prefer characteristics that are unrelated (or negatively related) to school effectiveness, there is a possibility of reducing school quality and social welfare (Barseghyan et al., 2019). Thus, understanding family preferences is crucial in understanding the potential effectiveness of school choice policies.
Measuring preferences can be difficult, however. Several prior studies have used revealed preferences from coordinated school enrollment systems (Abdulkadiroğlu et al., 2017, 2020; Ainsworth et al., 2022; Beuermann et al., 2021; Burgess et al., 2015; Campos & Kearns, 2022; Glazerman & Dotter, 2017; Hastings et al., 2010). These systems, which often rely on deferred acceptance algorithms (Roth, 1982; Roth & Xing, 1997), use the rankings of schools that families submit as the basis for determining which schools their children attend. Although not all mechanisms in use are strategy-proof, 1 they all provide a strong basis for identification of preferences.
The findings across these prior studies of schooling preferences are broadly consistent. Each finds that parents prefer schools that are closer to home and have high test score levels. Importantly, however, there is disagreement surrounding preferences for school effectiveness, as measured by school value-added. While several papers find that parents do not prefer higher value-added schools (Abdulkadiroğlu et al., 2020; Glazerman & Dotter, 2017), some more recent research suggests that parents do have some preference for higher value-added schools (Beuermann et al., 2021; Campos & Kearns, 2022). This is noteworthy given that value-added conveys more valid information about quality than test score levels (Harris, 2011; Kane & Staiger, 2002). However, value-added is also less readily available than test scores, and there is some evidence that shows when value-added is available it is utilized by families (Ainsworth et al., 2022; Valant & Weixler, 2020).
Many of these prior studies have been limited, however, in the range of characteristics they could measure and study. Prior literature suggests that families value a wide variety of school characteristics, including not only academics and distance to school but also safety and student demographics (Buckley & Schneider, 2002; Glazerman & Dotter, 2017; Hailey, 2022). The existence of after-school options, sports, school year length, and a variety of other school characteristics can also play important roles in a family’s choice of school. For example, does a family still rank a school with high test performance but few sports or after-school programs higher than a school with lower test performance but several sports and after-school programs? Do families who are more resource-constrained favor schools with extended days or aftercare options that reduce their household costs? If so, does the addition of these additional factors into the analysis change the results regarding the more commonly studied elements of test scores and distance? Prior studies have not been able to consider these questions due to a combination of inadequate data and school contexts that involve limited variation in schooling options.
We study school preferences in one of the most competitive school markets ever developed in the United States: New Orleans. The city’s school system departed from traditional school districts with two major policy changes. First, beginning in 2012, most families in the city who wished to have their children attend a public school had to participate in a centralized enrollment system called “OneApp” that uses a deferred acceptance algorithm. 2 Second, almost all the schools in New Orleans are now charter schools that have autonomy to operate more independently of government rules and union contracts, and this has led to horizontal product differentiation (Harris, 2020). The vast majority of these schools were required to provide transportation from anywhere in the city, and none were allowed to charge tuition. Not only do parents have more freedom to choose, but they apparently have a fairly wide variety of options to choose from, and detailed data regarding these options are publicly available (Arce-Trigatti et al., 2015). These traits make the New Orleans context well suited for identifying schooling preferences. 3
We also propose a new way of categorizing the factors affecting schooling demand that aligns more closely with traditional consumer theory from economics. Prior studies have focused on preferences for distance and academic outcomes. We instead place the wider set of characteristics in our study into two broad categories: (a) school characteristics from which families receive direct value (e.g., effectiveness in increasing academic outcomes and extracurricular activities); and (b) school characteristics that are valued because they reduce families’ other schooling or other household constraints. The latter set includes not only distance to school but also school hours, extended days, and after-school care. 4
While not all school characteristics fit neatly into these categories, this framework is useful, and this distinction between directly and indirectly valued factors becomes especially important when we test for heterogeneity in preferences between low- and middle-income families, which has been a focus of prior studies of family preferences for schools (Glazerman & Dotter, 2017; Hastings et al., 2010). For low-income families, these indirectly valued factors (e.g., free after-school care) are more influential over schooling demand, perhaps because their lower incomes force them to trade off their preferred schooling characteristics to satisfy basic, nonschooling needs (i.e., food and rent).
Taking into account such a wide variety of factors also has the potential to reduce omitted variables bias and thereby change the results for the more commonly studied factors. Although the size and direction of the omitted variables bias might be different in New Orleans and other choice-oriented settings, 5 we find in our preferred estimates that families do prefer higher school value-added, reinforcing some other recent evidence about the importance of school value-added from a survey experiment (Haderlein, 2022). At the elementary level, measured preference for value-added generally increases as more attributes are added to the model, while the high school results exhibit a less clear pattern.
An additional contribution of the present study is to highlight the advantages and disadvantages of various methods for understanding preferences. Revealed preference studies such as this one have the advantage of avoiding bias in self-reports and subjective interpretation that come with studies of stated preferences (e.g., surveys and vignettes), but a disadvantage is that it can be difficult to measure and isolate the characteristics of schools that make them preferred (e.g., because of omitted variables). Our study reduces that disadvantage, but we also show why it is a difficult problem to eliminate, what steps can mitigate these methodological problems, and how our own analysis, while intended to address these concerns, may do so imperfectly.
The next section outlines our discrete choice model and estimation procedures (mainly rank-ordered and conditional logit, in some cases, addressing school-level unobservable factors). After explaining our many data sources, we present results regarding average and heterogeneous preferences. We conclude with discussion about how our findings may help better understand and reinterpret the findings in the prior literature.
Model and Methods
The choice of schools can be analyzed through a discrete choice random utility model (McFadden, 1984) in which the alternative schools to be ranked are mutually exclusive and finite. Below, we provide an abbreviated model and refer the reader to prior and more complete discussions by Berry et al. (1995, 2004) and Train (2009).
Families (indexed by i) rank schools to maximize utility, which depends on all of the observed fixed attributes of the school (X j ), the attributes of schools that depend on family characteristics, such as distance to each school (X ij ), unobserved school characteristics (ξ j ), and a random individual utility component (ɛ ij ) such that the expected utility from the jth school is
The random component ɛ
ij
follows an iid extreme value distribution so that
We assume that the probability of observing the rank ordering r ≡ (r1, . . ., r j ) for a given individual i is the following product of probabilities:
The probability of observing school j = 1 as the top-ranked school is represented in the first term. The next term is the probability of observing school j = 2 as the second-ranked school, conditional on j = 1 being top-ranked, and so on ∀j ∈ J until the last term where there are only two options left. (The order of the subscript numbers here is arbitrary because any school can be top-ranked, not just j = 1.) The log-likelihood takes the general form:
Equation 2 shows a complete ranking of schools. However, this method also allows for incomplete rankings under the assumption that all unranked options are less preferred than all ranked options. In other words, unranked schools are still considered in the choice set under the assumption that they are less preferred than the lowest ranked school. We do not assume that families are indifferent among unranked schools.
In our setting, as in other recent studies (e.g., Glazerman & Dotter, 2017), the feasible objective is to estimate the influence of each school characteristic on families’ preference rankings. This focus on preference rankings allows us to incorporate both the factors entering directly into the family utility function and the indirect costs of schooling choices, for example, distance and after-school care. The relevant thought experiment involves the question, how would families’ rankings change if there were exogenous shocks in school characteristics?
At least four challenges arise in identifying preference rankings. First, families may sort themselves into neighborhoods. If families choose to live closer to their preferred schools, as we might expect, then controlling for distance in the estimation of (2) and (3) may be insufficient to account for the role of this factor. Identifying and controlling for feasible choice sets is one potential solution to this, as it may be reasonable to assume that families with the same choice sets are more similar to one another than those who have different choice sets (Burgess et al., 2015).
This sorting issue is less of concern in our setting, however. First, only 63% of families get their first choice of schools when their children first entered the system, so it would have been risky to move household locations based on schooling preferences. Second, every school in the city was required to provide transportation to all enrolled students regardless of their home address, making it feasible for families to apply to and attend any school in the centralized enrollment system. In addition, during this time, the selective admissions schools (i.e., those that were allowed to have academic admissions requirements) were not part of the centralized enrollment system (called the “OneApp”) and are therefore already excluded from the choice set. Thus, we believe that most families in the city face a similar choice set.
There are also potential endogeneity issues on the school side. Schools may make decisions about what products to provide, based on the information they have about demand. In that case, the fact that each school characteristic requires some production cost makes the ceteris paribus change in any given school characteristic seem unrealistic; with an essentially fixed spending level, schools cannot add one characteristic without subtracting something else. This makes it difficult to say that an individual school characteristic (e.g., school quality) affects school rankings, which has been the primary focus of prior research. This implies that (endogenous) decisions by both families and schools complicate the identification of preferences.
A third threat to identification, and one we partially address in the present study, is omitted school characteristics that are correlated with included ones and with the rankings. In other discrete choice settings, researchers have addressed this problem using the Berry et al. (1995, 2004) method. 6 However, this method still rests on the restrictive assumption that omitted variables are orthogonal to the included ones (Ackerberg et al., 2007). The same problem appears with other related methods (Blundell & Powell, 2001; Lewbel, 2004; Wojcik, 2000). If there are omitted characteristics, then the coefficients of the included ones will not reflect family preferences. Moreover, there is no way to sign the bias as the omitted variables have unknown relationships with the included variables.
We address some potential omitted school characteristics using an unusually extensive set of school characteristic indicators from a publicly available guide to schools produced each year by a nonprofit organizations in New Orleans since 2008. 7 This Parents’ Guide includes more than one hundred pieces of information about each school, including test scores, demographics, extracurricular activities, and other available programs. The data are initially self-reported by schools, but those in charge of the guide also interview school leaders to corroborate and edit some of the data elements. Having this much standardized information reduces the chances of omitted variables bias. But this is also insufficient and we propose an additional identification check later, leveraging the differences in information that families have when their first child enters schools versus when a second child enters.
Fourth, and finally, the extent of information available to families affects the results. We would certainly expect that families are better informed about school characteristics, such as extracurricular activities, that are discussed explicitly on school websites, compared with constructs such as school value-added, for which there is little, if any, direct information available. While research suggests that families gather information from word of mouth and that they can form expectations about value-added, this knowledge is likely subject to measurement error, which would tend to attenuate the preference coefficients toward a finding of weaker preferences. This may be why some prior studies have not found a preference for school value-added (Abdulkadiroğlu et al., 2020; Glazerman & Dotter, 2017). More generally, we expect bias toward finding that families prefer characteristics for which they have good information, other things equal.
Data
School Rankings and Revealed Preferences
Our expansive data allow us to build on prior work in several ways. We are able to study revealed preferences from a strategy-proof mechanism with a rich set of school characteristics in a highly differentiated market. Unlike most other studies, we are also able to do so for both elementary/middle and high schools and for almost all publicly funded schools in the city. We specifically study the demand for schooling among New Orleans’s families who are considering publicly funded schools and are able to study preferences for both academic and nonacademic factors, as well as the indirect costs families incur in the schooling process. The main outcomes are families’ OneApp school rankings and assignments in 2013, the second year the ranking system was used. 8 The covariates are measures of school characteristics timed to reflect the information families had available at the time they were making the given school decision. As school usually starts in the fall, this means we use the information available the prior spring of the same calendar year.
The school rankings come from the city’s OneApp school application system, in which families rank schools, and this information is used to assign students to schools (Abdulkadiroglu, 2009; Abdulkadiroglu et al., 2005). Any families interested in having their children attend a school in the OneApp had an incentive to fill out the application. 9 In assigning students to schools, the OneApp gives priority to students currently enrolled in the school, students with siblings in the same school, and those living within the schools’ (large) geographic catchment areas. 10
With rankings data, it is common to have explicit rankings for the top choices while other options are left unranked. This arises in our data as well for several reasons. First, the OneApp system only allows eight schools to be ranked. 11 Second, some families may have been willing to accept only a few public schools because they considered schools outside the OneApp system as their main alternatives. Third, among those families who already had a child in a OneApp school, those who wanted their children to return to that school the following year filled out a different section of the OneApp that did not involve ranking schools; these students show up in our data as having the current school ranked first and all others unranked. The rank-ordered logit incorporates unranked options in the likelihood function as potential options for each individual. This includes the cases where families only indicate they want to return to their original school, in which case that school is listed as top-ranked and J − 1 schools are unranked.
Of these three scenarios, we view the second as most likely to introduce bias, but the size and direction of such bias are unclear because we do not have information about where the unrankable options fall in the underlying preference ordering or about the characteristics of most of those schools because they are not in the Parents’ Guide. For example, if an omitted public school had a higher state-assigned School Performance Score (SPS) and was ranked higher by families, and was otherwise similar to the top-ranked school that participated in the OneApp, then we would understate the role of SPS. 12 We can only speculate on the actual biases involved in our analysis. Strictly speaking, our estimates are only unbiased if families are indifferent between the top-ranked OneApp school, and the (unranked) non-OneApp schools have the same characteristics as the top-ranked school. This assumption is unlikely to hold, and this remains as a limitation of this type of analysis, as all ranking systems have unrankable options (e.g., Abdulkadiroğlu et al., 2020; Glazerman & Dotter, 2017).
School Characteristics
Most of our school characteristics come from the Parents’ Guide. This guide initially started with the efforts of a single community activist and grew into a partnership among many community organizations. The guide was also part of a larger effort to organize parents in the pursuit of excellent public schools in the city. The Parents’ Guide was available online to anyone with internet access and in print in schools, libraries, post offices, and other public locations throughout the city. The leader of the guide, whom we interviewed, explained that the set of school characteristics included for each school was developed based on what participating parents and community members believed was most important. Over time, the guide came to include more than a hundred unique elements for each school. We obtained the underlying electronic files to turn these into analyzable data.
For purposes of this analysis, we had to narrow the set of school characteristics and we did so based on (a) the prior literature; (b) the contents of the Parents’ Guide, which themselves were the product of extensive vetting with parents; (c) our own discussions with education stakeholders in the early phases of this research; and (d) survey evidence about parent preferences provided by another local organization. We did not include student demographics as school characteristics because the city’s publicly funded schools had so little variation on these measures. Another important limitation is that safety measures have been shown to be important (B. A. Friedman et al., 2007), but data on this factor were not available for our study. As a result, to the degree that school safety is related to any of our observed characteristics, those characteristics are subject to omitted variable bias.
Our first measure of academic outcomes is the SPS assigned by the state of Louisiana to the vast majority of publicly funded schools. 13 The SPS is based on the average test score levels of students; in high schools, the test scores are combined with data about high school graduation rates. Since 2012, these have been translated into letter grades A–F. As both the numeric score and the star/letter grade were publicly available, we rescale the SPS so that a 1-unit increase is approximately equal to a one letter-grade increase. 14 This allows us to maintain the less coarse measure of SPS while still being easily interpretable.
As the publicly reported SPS index during the time period of analysis did not include a measure of school value-added, we calculated this ourselves using now-standard methods.
Following Kane and Staiger (2008), Chetty et al. (2014), and others, we estimate the following simple model:
where A ijt is achievement of student i in school j at time t, while X ijt represents one or more student- or school-level covariates using a two-step procedure recommended by Ehlert et al. (2014). The term θ j represents the school effect or value-added. There is a large and growing literature on the various methods for value-added estimation. Angrist et al. (2017) provide evidence that school-level value-added measures (VAM) like those estimated from Equation 3 have limited bias. Ehlert et al. (2014) show that the inclusion of an extensive array of additional covariates, beyond the lagged test scores, yields very similar value-added estimates (correlation of +0.9 and above). Note that the high school measures use the end-of-course exams taken in ninth or tenth grade as the dependent variable, and the lagged test scores are from the eighth grade standardized test. In many cases, this means the lagged test score was from a prior school, although some high schools include eighth grade (and prior grades).
We used various versions of the above VAM. The reported estimates do not involve shrinkage, but the results are very similar when applying a post-estimation shrinkage adjustment following Herrmann et al. (2016). The reported results also focus on math and English Language Arts (ELA) so that the academic content of the VAM aligned with the SPS, but we re-estimated using all four subjects. The results are also robust to this change.
Value-added measures can differ significantly from test levels because of differences in initial test levels when students enter a school (Harris, 2011; Kane & Staiger, 2002). While value-added is a better measure of school quality, there are various reasons why it might not be related to family rankings of schools in practice, as has been found in some prior studies that have examined them (Abdulkadiroğlu et al., 2017, 2020; Glazerman & Dotter, 2017). In addition to the fact that families do not observe them in Louisiana, school VAM tend to include considerable random error (Kane & Staiger, 2002) and they are correlated with test levels, leading to collinearity. The measured relationship between value-added and school rankings therefore will likely be attenuated.
Using home and school addresses, we calculated the linear distance between each home and every school that serves the relevant school grade. The role of distance may be nonlinear, however. In particular, families might think of the nearest school as the “default” no matter how close it is (Thaler & Benartzi, 2004). We also provide results using “nearest” school as well as a quadratic in school distance as additional checks. (Another potential nonlinearity arises because bus transportation may be available only beyond a certain distance, for example, 1.5 miles from school. Families on the margin of these zones might prefer schools that are slightly further away to gain access to bus transportation. However, these thresholds are not uniform across schools and therefore cannot be easily studied.)
One contribution of this study is making a clearer distinction between the factors that are directly valued and those that affect schooling and other household costs. The unusually rich information in the Parents’ Guide allows us to capture the availability of football and other sports, band and other extracurricular activities, school schedules, and other measures. For elementary schools, we also indicate whether the schools offer extended school days or after-school care (free or paid).
We also use the Parents’ Guide to identify whether school locations are “in flux” in the sense that they may be moving to a new location in the near future or if they have just recently moved locations. This is relevant for understanding the role of distance and also the general desire for certainty and stability. As the city’s school reforms were still developing at the time we carried out this study, changes in location are more common than it would be in most settings.
The quality of buildings varies considerably due to both the differences in flood damage and the fact that some buildings were much older before Hurricane Katrina. With substantial funding from the federal government, the city embarked on a large school construction and renovation initiative. We therefore created a variable equal to 0 for schools that were only modestly renovated, equal to 1 for schools with large renovations, and equal to 2 for each of the 11 brand new buildings. It is difficult to distinguish the role of building quality from being “in flux,” however. The meaning of the in flux variable is clearly different when the school will be moving to a new building versus an older one that received less post-Katrina building funds.
Based on input from a group of community stakeholders, we added an indicator for “legacy” school, which denotes whether the school carried a name that was used prior to Katrina. In discussions with local leaders and practitioners, they believed this would be an important factor, especially in high school, where many families may want their children (or grandchildren) to attend school at the same high school that they themselves attended and to potentially play on the same sports team that they themselves played on. While not part of the Parents’ Guide, we worked with local officials to identify schools that would meet this criterion.
An additional measure that we have included is an indicator for whether or not the applicant indicated on the OneApp that they had a sibling in the school for which they were applying. Many families would prefer siblings to be enrolled in the same school and the OneApp gave priority to families with siblings already enrolled in the school. As a result, we expect this factor to be strongly related to family rankings. As families only marked the sibling indicator if they wished to receive priority in the lottery, interpretation of this factor is complicated. We discuss this later in the article.
We include a measure of school size for several reasons: (a) Some families may prefer the more personal environment of small schools and others may prefer large schools because of the possibility of finding a group of friends to fit in with; (b) some extracurricular activities, such as band and football, require large numbers of student participants; and (c) there are some economies of scale to school size that affect the quality and quantity of services provided. To the degree that (a) and (b) hold, the omission of school size would bias estimates for the other variables correlated with it. However, if school size matters because of (c), then omitting it would yield less biased estimates of programs subject to economies of scale because the school size coefficient would absorb some of the preference for the quality of these activities. As we cannot distinguish among these roles for school size, we run the models with and without this measure. Schools vary in the number of grades they offer; therefore, we rely mostly on a measure of the average size of each grade.
As our student-level data do not contain individual student or family demographic data, we substitute block group characteristics from the Census, especially average median household income. These are from the 2007–2011 American Community Survey, respectively, and are used in the analyses of preference heterogeneity. We considered additional variables, such as safety and student demographics, but were limited by data and other factors. 15
Our sample includes roughly 31,000 students in 2013, compared with the total enrollment of 44,791 (Cowen Institute, 2014). Nineteen schools (mostly run by the Orleans Parish School Board [OPSB]) did not participate in the OneApp in 2013, explaining the lower sample size in that case. 16 As all the selective admissions schools in the city were also in OPSB, this means the average academic ability of students in this analysis is below the city average. We have nearly complete data on all schools in both the OneApp and enrollment samples.
Table 1 provides descriptive statistics for the school characteristics. Here, and in all subsequent tables, elementary and middle school students are combined together because there are essentially no schools with traditional middle school grade structures in the city. For schools whose grades cut across elementary/middle and high school grades, we split the school based on the specific grades that students are applying to (e.g., in a school with Grades 7–12, Grades 7–8 are coded as an elementary analysis and Grades 9–12 are included as a separate high school).
Elementary School Characteristics
Note. The 2013 edition of the New Orleans Parents’ Guide to Public Schools; Louisiana Department of Education; Recovery School District. The value-added measures (VAM) are based on authors’ calculations. SPS = School Performance Score.
The first part of Table 1 provides information about the school characteristics in elementary school. The average elementary school offered about three different sports and six extracurricular programs, and were given an SPS score of 78.7. Nearly 70% of them had an extended school day, with 24% offering free aftercare and 20% offering paid aftercare. The second part of the table shows similar characteristics for high schools. Nearly 90% of high schools offered some combination of band and football, and two thirds of the 2013 high schools had names similar to schools that existed pre-Katrina, or “legacy schools.”
Preference parameters cannot be identified under perfect collinearity, which in this case constitutes economic bundling, that is, Service A is offered if and only if Service B is offered. In this case, football and band are complementary in consumption and come closest to perfect bundling. There are no schools that have football but not band. Narrowly speaking, we can identify the role of band because there is one school that has band and no football, but this one band program could be atypical (e.g., they might offer band but not marching band, or they may be very small programs). 17 In the analysis, our preferred specification has a variable equal to 1 when either band or football is offered. This approach requires the fewest assumptions but also provides the least separation in preferences for the two programs.
Results
Our objective is to estimate family preference rankings for schools. We begin by reporting average preferences for elementary and high schools (separately), 18 and then consider the potential role for omitted variables as a threat to identification in our study, as well as the prior literature. This is followed by analysis of effect heterogeneity by family income.
Average Revealed Preferences From School Rankings
Tables 2 and 3 report estimation from a variety of specifications. The first two columns represent our baseline model with the rank-ordered logit, using all the rankings available. The first is presented in distance units (found by dividing the logit coefficients on each characteristic by the coefficient on distance), which provides an intuitive interpretation of the coefficients. For example, the coefficient of 0.765 on SPS score suggests that families are willing to face an extra 0.77 miles distance to the school to gain an extra letter grade in SPS score. The second (and all remaining results) are presented in the more common odds ratios with robust standard errors. 19 Column 3 controls for school-level unobservable factors following methods similar to those of Berry et al. (2004) and Abdulkadiroğlu et al. (2020). Specifically, we calculate school-level mean utility by estimating a rank-ordered logit model with a full set of school fixed effects, distance to school, and the sibling preference indicator. In the second stage, we regress the mean estimated school effects on school characteristics. 20 Column 4 examines preferences related to the top-ranked school (conditional logit). As the switch to conditional logit and accounting for school-level unobservables yield very similar results to the rank-ordered logit without these controls, we proceed without them. 21
Estimates of School Choice Parameters: Elementary/Middle Schools
Note. All columns except the column labeled “Conditional logit” are from rank-ordered logit regressions. The first column contains estimates in terms of “miles” and are calculated as the rank-ordered logit coefficient on each characteristics divided by the coefficient on distance. Exponentiated coefficients are displayed, and robust standard errors are in parentheses. School effect standard errors are calculated using a 25% bootstrap sample. SPS = School Performance Score; VAM = value-added measures.
Significant at 10%. **Significant at 5%. ***Significant at 1%.
Estimates of School Choice Parameters: High School
Note. All columns except the column labeled “Conditional logit” are from rank-ordered logit regressions. The first column contains estimates in terms of “miles” and are calculated as the rank-ordered logit coefficient on each characteristics divided by the coefficient on distance. T-statistic is estimated by pooling the elementary and high school students into a single rank-ordered logit regression and interacting each variable with a high school indicator variable. The reported t-statistic is the t-statistic on this multiplicative interaction term. Exponentiated coefficients are displayed, and robust standard errors are in parentheses. School effect standard errors are calculated using a 25% bootstrap sample. VAM = value-added measures; SPS = School Performance Score.
Significant at 10%. **Significant at 5%. ***Significant at 1%.
The next four columns alter the set of school characteristics. 22 We add a “nearest school” indicator (Column 5) and squared terms for distance and SPS (Column 6). 23 These first five columns include a measure of school size (the number of students per grade), while Column 7 drops this measure. Finally, Column 8 excludes the VAM score. Given the above issues with identification, we draw conclusions about preferences only when the results are generally robust across specifications.
As predicted from prior studies, families have strong preferences for measured school performance and distance. Given that the SPS is publicly reported and has limited random error, the positive coefficient on that measure is unsurprising. A one letter-grade increase in the SPS is equivalent to reducing distance to school by 0.765 miles.
Even after controlling for SPS, school value-added is also positively related to school rankings. 24 For elementary/middle schools, increasing school value-added by one (school-level) standard deviation is equivalent to reducing distance by almost 2 miles. The fact that value-added is related to preferences even after controlling for SPS conflicts with some of the prior research on the topic (Abdulkadiroğlu et al., 2020; Glazerman & Dotter, 2017; Hastings et al., 2010). We later explore whether omitted variable bias can explain these differences.
We also find that families are more likely to send their children to the nearest school.
This might suggest that some families view the nearest school as the default choice, even when there is another viable school option only slightly farther away. Small changes in SPS matter less in the lower range (e.g., where F letter grades are given) compared with higher levels. (Nonlinear and interaction terms are difficult to interpret in a logit, but we carried out additional tests and find that they do generally reflect the basic structure of the nonlinearity in this case.)
We are especially interested in the role of extracurriculars and indirect cost factors, which prior studies of revealed preferences have not been able to address. Football and band are particularly popular in New Orleans, 25 so it is not surprising that families prefer high schools with these programs. Having either band or football increases the odds of making a high school top-ranked by about 20%, about the same as 2 miles of driving distance.
The patterns are similar across elementary and high schools, but there are also some predictable differences. In high school, students often get intensively involved in one or two sports or other extracurricular programs (Brown, 1992) This may partly explain why preferences for band/football, as well as other sports, are all stronger at the high school level. 26 The coefficients on SPS and value-added at the high school level are 2 to 3 times larger, and the coefficient on band/football is 10 times larger, compared with the elementary level. We expected to see the same pattern with other extracurriculars as well, but families seem to pay little attention to nonathletic extracurricular programs in high school.
Cost factors also play a role in the rankings. In addition to transportation costs, parents are responsible for caring for their children. 27 They can meet this responsibility by supervising children themselves or by having older siblings do so—both of which impose costs on families— or by sending their younger children to schools that provide after-school care (paid or unpaid).
The results reinforce this theory as aftercare (paid and unpaid) is consistently important to elementary families. 28 Paid aftercare is equivalent to reducing distance by about 0.7 miles. We cannot attribute all of the preferences for these factors as through a cost reduction. Families may value other attributes of aftercare and may prefer schools closer to home, not because of distance but because students can attend schools with their neighbors. However, we find this characterization useful in highlighting how family constraints may affect family demand for certain school characteristics. We explore the issue of cost considerations more in section “Additional Methodology Issues” below.
Families appear to value high schools with a long tradition or “legacy” in the city, dating to the pre-Katrina years. This could be because families want to continue traditions, sending children to the schools that parents or other family members attended. Alternatively, this could reflect established reputations; although the schools now have new operators in the post-Katrina period, they may perceive that having the same name means that it has programs and qualities similar to prior years. The fact that legacy status seems especially important in high school might be because adults in New Orleans tend to identify themselves by the high school they attended. 29
Elementary school rankings are lower when schools are “in flux” (i.e., have recently changed locations and/or have plans to do so in the near future), although the role of this factor seems small in magnitude compared with others. The estimates of the role of new and refurbished school buildings are erratic in both elementary and high school, due in part to the fact that some of the schools recently moved to new buildings, creating collinearity between the new building and in flux variables. When we drop the in flux variable, the preference for new buildings becomes stronger in high school.
In addition to distance, families give extremely high rankings to schools for one child when a sibling already attends these schools. Having two children in two different schools, for example, would be costly to parents who would then have to go to two different schools for parent–teacher conferences and keep track of two sets of rules, policies, and schedules. The sibling schools seem to matter more at the elementary level, perhaps because younger children have less specialized interests. Younger students may also feel safer with an older sibling walking them to the bus stop 30 and watching out for them at school.
The extreme magnitudes of the sibling coefficients, however, largely reflect the structure of the data rather than actual preferences. First, the OneApp only allows families to indicate which schools siblings attend for the top eight schools. For all the other possible schools, it is implicitly assumed that there are no siblings, which automatically makes the sibling variable a strong predictor of rankings, independent of actual preferences. On the other hand, families only need to request the sibling option if they wish to obtain priority for school assignment. In this way, families that wish to remain in their current school would not need to indicate sibling status because they would already have priority. Finally, the sibling coefficients likely capture school unobservables. That is, if there is an unobserved factor that led families to select a school for one child, that same characteristic likely affects the ranking for the other sibling. While this leads to an upward bias in the sibling coefficient (as an estimate of the actual desire to have children in the same schools), it also provides some evidence on bias for other variables.
Overall, these results suggest that schooling demand is influenced by a wide variety of school factors. Although they prefer schools with strong academic performance (as measured by both SPS and value-added), New Orleans public school families with high school students have strong preferences for athletics. Also, the costs of reaching schools, as well as the need for after-school care, seem to create trade-odds with respect to other school characteristics, such as academics.
Omitted Variables
Omitted variable bias is one of the main threats to identification of preferences in this and other studies of revealed preferences. Schools that place greater focus on academics, for example, may have fewer resources to devote to extracurricular activities. We have worked to address this by using a richer set of covariates. The additional covariates do not guarantee lower bias, but they could explain some of the differences with regard to prior studies that have only a few school characteristics. We note, in particular, that the Parents’ Guide was designed to capture the factors most important to families, so it is less likely that important factors are left out.
We might also gauge the role of omitted variables by examining families with multiple siblings. Specifically, we compare the coefficients on the easy-to-observe (Parents’ Guide) characteristics between the sibling and nonsibling samples. If there is little bias, we would expect small differences in coefficients between the two samples; the sibling group should be more knowledgeable about the unobserved factors that are most likely to create bias, and therefore their preference rankings might look different than the nonsibling sample.
Our results are generally consistent with the idea that our results have limited bias. Table 4, Column 1 redisplays the same column from Tables 2 and 3, whereas Column 2 drops the sibling variable. Dropping the sibling variable has relatively little influence on the results. Also, comparing the sibling (Column 3) and nonsibling (Column 4) subgroups, the relative sizes of the key coefficients, and therefore the broad conclusions of the study, are similar. In particular, we note that the coefficient on school value-added is similarly positive and statistically significant for both subgroups at both grade levels. The coefficients are larger for the sibling sample, which may reflect that this is the only factor in the model that is not in the Parents’ Guide, so nonsibling families may be less aware of it until they develop their school-related social networks and hear more word-of-mouth school recommendations.
Preferences for Students With and Without Siblings
Note. Estimates from rank-ordered logit regressions. Exponentiated coefficients are displayed, and robust standard errors are in parentheses. P values on the difference between sibling and nonsibling preferences come from the interaction of being a member of the sibling subsample with each school characteristic. SPS = School Performance Score; VAM = value-added measures.
Significant at 10%. **Significant at 5%. ***Significant at 1%.
The above analysis is not definitive, in part, because the sibling and nonsibling samples could be different in several ways aside from their previous experience with schools. For example, their household income, age, location, and number of schools ranked can also differ across these groups. The fact that the results are similar across both the sibling and nonsibling groups suggests this is likely not the case, but we cannot rule out a scenario where different biases across groups lead to similar estimates.
The positive valuation in our article for school value-added differs from several prior studies. Many factors could explain that, for example, New Orleans families could have different preferences from those in other locations. Another possibility involves the omitted variables in prior studies that we have included. To better understand the differences in results across studies and role of data, we re-estimated our models limiting the set of school characteristics that have been included in prior studies. Both of the above explanations seem relevant. As shown in Table 5, New Orleans families seem to prefer higher value-added elementary schools even when omitting the richer set of covariates. However, the results are meaningfully different when the additional school characteristics are added. At both the elementary and high school levels, preference for test scores levels is somewhat overstated when including minimal characteristics, while estimates of distance are relatively static.
Estimation of Preferences With Varying School Characteristics
Note. Each column represents estimates from separate rank-ordered logit regressions. Exponentiated coefficients are displayed, and robust standard errors are in parentheses. SPS = School Performance Score; VAM = value-added measures.
Significant at 10%. **Significant at 5%. ***Significant at 1%.
At the high school level, the VAM estimate is a bit more erratic. The negative preference estimates are reduced after the inclusion of other school characteristics, but the estimates do not change to positive until the final set of attributes is included, specifically the “legacy” attribute. In Supplementary Appendix Section A2 in the online version of the journal, we provide correlation matrices for the school attributes (see Supplementary Tables A2 and A3 in the online version of the journal). These correlations suggest that the sensitivity of high school results to the legacy variable is likely a result of the strong correlation between those attributes for high schools.
Additional Methodology Issues
The issue of omitted school characteristics is not the only methodological issue. Below, we revisit some issues that were first raised in the methods discussion—the issue of unranked options and the Independence of Irrelevant Alternatives (IIA) assumption—and provide additional robustness checks.
We noted earlier that there are limits on the number of schools that can be ranked and that many families do not rank all the options they could. As a result, a little over 80% of families only rank one school. As many families rank one option because they want to continue in the current school the following year, we can partially address this issue by focusing on entry grades where continuation is not an option. Fewer students in kindergarten and ninth grade rank only one choice (approximately 60% and 50%, respectively). Results estimated separately by grade can be found in Supplementary Table A1 and Figure 1A through 1H in the online version of the journal. The results are generally robust in these cases, but also display noteworthy patterns across grades.
The results reported so far also use the rank-ordered logit, which requires the IIA assumption. To determine whether this assumption is biasing our results, we also estimated a mixed logit model, similar to Hastings et al. (2010) and Train (2009). Estimates from this model are presented in Supplementary Appendix A5 the online version of the journal and show that our results are generally robust to the IIA assumption. They also provide evidence about the variation in preferences across families. Additional results (available upon request) compare results using ranked preferences versus anther common method of student assignments (Buddin et al., 1998; Reback, 2008). 31
Heterogeneity by Income
Prior studies have tested for heterogeneity in preferences and demand by income (Glazerman & Dotter, 2017; Hastings et al., 2010). These analyses suggest that low-income families have relatively low demand for academic quality. However, given the above evidence that the estimation strategy and data influence the results, especially with regard to preferences for school value-added, additional analysis is warranted. As we argue below, some commonly omitted school characteristics may be particularly important to low-income families because of the tighter constraints they face.
Several theories have been put forth regarding why low-income families might have different preference rankings and take a deficit perspective on the topic. Hastings and Weinstein (2008) focus on a lack of information among low-income groups. Second, families may perceive that their children will be more comfortable and/or successful if their classmates are more similar academically (Hoxby & Weingarth, 2005), and groups with lower test scores might therefore prefer schools where academic performance is similarly low.
We propose an alternative explanation. Even among families with the same schooling preferences, there are reasons to expect low-income families to have weaker demand for academics—specifically because of income constraints. Diminishing marginal utility/happiness from income means that any indirect financial expenditures involved in schooling choices (e.g., childcare and transportation) yield greater losses in personal well-being for low-income families. Compounding this effect, some of the family resources that are necessary for education are also important for other household purposes. In particular, low-income families are less likely to own automobiles that are used for many purposes and the absence of a car increases the marginal cost to families of sending their children to schools further away. 32
To better understand how these income-related constraints play out in practice, we estimate the baseline model from Tables 2 and 3 separately by Census block group income terciles where the bottom tercile has the lowest neighborhood average median income. The simple average of the median incomes is US$16,174 in the first tercile, US$28,461 in the second tercile, and US$48,337 in the third tercile, that is, income in the third tercile is roughly 3 times what it is in the first tercile. While none of these are what might be considered “high-income” areas, further subdivision of the results in the third tercile would reduce statistical power, especially in the highest income neighborhoods where very few families send their children to the publicly funded schools we are studying.
The results, presented in Table 6, are generally consistent with our predictions, especially at the elementary level. The t-tests in the last column show that coefficients on our academic measures are indistinguishable across income levels in three of the four cases (elementary test levels/SPS being the exception). The lowest income families with elementary-age children have weaker demand for SPS. (The point estimates on value-added yield a similar pattern, but they are statistically significant only between the medium- and high-income groups.) The indirect costs also seem to affect their choices more: Low-income families rank schools with free after-school care, extended days, and weekend classes higher than high-income families. The lowest income families also have weaker preferences than higher income groups for paid after-school care, presumably for the same reason. These differences are striking, especially considering the differences across income groups are almost certainly attenuated with the use of block group–level demographics.
Heterogeneity by Income
Note. All columns are from rank-ordered logit regressions. Income is defined as the median block group income based on data from the 2007–2011 American Community Survey. Terciles are based on the population in the sample. T-statistic is from a pooled regression across all income terciles where all characteristics are interacted with indicator variables for middle and top tercile indicator variables. The reported t-statistic is from the multiplicative interaction with the top tercile indicator variable. Exponentiated coefficients are displayed, and robust standard errors are in parentheses. VAM = value-added measures.
Significant at 10%. **Significant at 5%. ***Significant at 1%.
The patterns differ somewhat in high school. Band/football and other sports still seem more important to the lowest income families (only the latter is significant between lowest and highest income groups, although band/football is significant between middle- and high-income families). There are no differences in either SPS or value-added preference, and the pattern of coefficients runs in the opposite direction on value-added compared with elementary school. We also generally see lower preference for sports and music for the highest income group. One potential explanation for this is that wealthier families may be able to afford these experiences outside of the school through other paid organizations, for example, there are many dance programs around the city that are not school-based.
Our results generally align with those of prior studies that have found weaker preferences for academics among low-income families (Glazerman & Dotter, 2017; Hastings et al., 2010). Academic measures are relatively less important than extracurriculars and indirect cost considerations in the lowest tercile. However, our analysis points to a different explanation—one that is related to income itself and the way in which schooling choices intersect with household budgets. This role for cost factors is important and reinforces the importance of considering a wide range of school factors when studying preference heterogeneity.
Conclusion
Identifying how families view and rank school characteristics is a difficult task. We rarely have information on how families rank schools in real choice settings, and even when we do, the information about schools is often limited. In this article, we were able to match families’ school rankings to a wide range of school characteristics to estimate the relative preferences families have for these schools. While prior research has shown that academic quality and distance matter greatly, we have also shown that factors such as aftercare, extracurriculars, and sports are also valued.
Our results also conflict somewhat with the prior research that suggests families, especially those with low incomes, are uninterested in higher school value-added. While it is possible that our analyses, too, involve omitted variable bias, we note that the measures we used were created by an organization whose purpose was to provide information to families about the factors they care about most. Still, we cannot rule out all other explanations for the discrepancies between our results and these prior studies. For example, the correlation between school characteristics, and therefore the nature of omitted variable bias, might differ in the other contexts that researchers have studied. What we do observe in our setting is that the preference for value-added is stronger when we include our full set of school characteristics compared with a more restricted set of variables. At the high school level, this finding is sensitive to the inclusion of legacy status, which is likely a variable that is more relevant in the New Orleans context compared with many other cities.
Our richer set of school characteristics allows us to show that, in addition to academic factors, the costs are also important to families—especially those from low-income neighborhoods. While parents are not required to pay tuition and fees in traditional public schools and charter schools, other costs vary indirectly, depending on which schools they choose. In addition to distance and transportation costs, the availability of after-school care is another important indirect cost. Predictably, these costs seem to affect schooling demand for the lowest income families more than others.
Our study highlights not only the strengths and weaknesses of specific methodological choice and data sets, but also those of entirely different broad approaches. We have focused on quantitative methods of revealed preferences that require researchers to be able to distinguish preferences from enrollment, identify and measure all school characteristics that might be relevant, and for there to be sufficient variation across schools to identify their properties, without locational sorting of housing. These limitations also apply to other studies of this type (Abdulkadiroğlu et al., 2020; Burgess et al., 2015; Glazerman & Dotter, 2017), and one contribution of the present study is clarifying these issues.
But the analysis of revealed preferences is not the only method at our disposal. Survey experiments are also useful ways to identify preferences (Haderlein, 2022; Hailey, 2022). As noted earlier, we used survey evidence to help identify the school characteristics that are most likely to be important in our revealed preference analysis. Still another approach involves open-ended responses from interviews (e.g., Goldring & Phillips, 2008), which do not require the researcher to identify and measure every characteristic that might matter. On the other hand, these open-ended responses can be difficult to interpret and/or place in conceptually clear categories.
Our findings also have important implications for the influence of increasingly popular school choice policies. The effects of these policies, such as charter schools and vouchers, depend on family preferences for schools. In particular, the fact that parents incur indirect costs and have preferences for a wide variety of factors means that even when schools do compete, it is based not only on academics. Instead, school leaders may have to reallocate resources away from academics to pay for after-school care and other nonacademic services to attract families. This would help explain why about half the studies on school choice find no effect of competition on student test scores, and the effects tend to be small in magnitude (e.g., Gill & Booker, 2007), and why charter schools (Angrist et al., 2011; Center for Research on Education Outcomes, 2013) as well as vouchers (Abdulkadiroğlu et al., 2018) have mixed effects on test scores. Family preferences may not create sufficient pressure to increase these observable metrics.
The results in this analysis could also help explain the growing achievement gap between low- and high-income students (Reardon, 2011). While government funding has become more equalized over time by income (Jackson et al., 2016), our estimates suggest that low-income families do seem to weigh academic measures less heavily than their high-income counterparts. This is a significant issue given the arguments, going back to M. Friedman (1962) and more recently Howell and Peterson (2006), that school choice and competitive markets have the potential reduction in the achievement gaps.
As the share of students attending charter schools and publicly funded private schools rises, the potential to understand preferences, and the impact of those preferences on the schooling market, will only continue to grow. With this research, we have tried to better understand the empirical challenges and the means of addressing them.
Supplemental Material
sj-pdf-1-epa-10.3102_01623737221134528 – Supplemental material for What Schools Do Families Want (and Why)? Evidence on Revealed Preferences From New Orleans
Supplemental material, sj-pdf-1-epa-10.3102_01623737221134528 for What Schools Do Families Want (and Why)? Evidence on Revealed Preferences From New Orleans by Douglas N. Harris and Matthew F. Larsen in Educational Evaluation and Policy Analysis
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The authors carried out this work within the Education Research Alliance for New Orleans, based in the Department of Economics at Tulane University. The center is funded by generous grants from the Arnold Foundation, William T. Grant Foundation, and the Spencer Foundation. In addition, the authors wish to thank the Cowen Institute for Public Education Initiatives, Orleans Parish School Board, and Recovery School District for providing data. Paula Arce-Trigatti, Lynn Dupont, Alica Gerry, Gregory C. Rigamer & Associates (GCR), Robert Santillano, Debra Vaughan, and especially Jill Zmmerman made important contributions. For their useful comments, we also thank Robert Bifulco, David Figlio, Bill Jackson, Leslie Jacobs, Neerav Kingsland, Susanna Loeb, Parag Pathak, Rose Peterson, Grant Schreiner, Kenneth Train, Jon Valant, Patrick Wolf, and participants in sessions at Louisiana State University, the Association for Public Policy and Management (APPAM), the Association for Education Finance and Policy (AEFP), and the Great Schools conference in San Francisco, CA.
Supplemental Material
Supplemental material for this article is available online.
Notes
Authors
DOUGLAS N. HARRIS, PhD, is professor and chair of the Department of Economics, the Schlieder Foundation Chair in Public Education, founding director of the Education Research Alliance for New Orleans and the National Center for Research on Education Access and Choice (REACH) at Tulane University. His research focuses on a range of K–16 education policies, including school choice, charter schools, accountability, teacher labor markets, and college access.
MATTHEW F. LARSEN, PhD, is an assistant professor of economics at Lafayette College. He is a nonresident research associate at the Education Research Alliance for New Orleans. His research focuses on education policy and school choice.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
