Abstract
As parents are increasingly given flexibility to enroll their children in a school of their choice, understanding parents’ preferences for school qualities is essential. Using a randomized survey experiment, this study adds to the existing literature by assessing parents’ preferences in a controlled environment, where they can be isolated from information asymmetries and constraints. Results suggest that achievement matters to parents but status matters more when evaluating quality and growth matters more when choosing between schools. Additionally, student demographics affect both parents’ perception of school quality and their likelihood of selecting into a school. This article has important implications for the theory and practice of accountability as it offers new insights on parents’ latent preferences for school qualities.
Many recent education policies give parents more autonomy over their children’s education. Open enrollment districts, for example, allow parents to send their children to schools that better reflect their preferred qualities (Corcoran et al., 2018). Additionally, many states have revised their accountability portals to make them more accessible for parents as a condition of the Every Student Succeeds Act (Burnette, 2017). Yet while parents are increasingly given choice and information about their children’s schools, much of what we know about parents’ preferences for school qualities is inferred from their revealed preferences in existing markets (Abdulkadiroglu et al., 2017; Harris & Larsen, 2017; Hastings et al., 2008). Revealed preferences are useful for predicting real-world behaviors; however, they are susceptible to constraints and misconceptions about schools and the choices available.
Better understanding parents’ preferences for school qualities has important consequences for the broader policy objectives underlying these initiatives—specifically, improving the equity and efficiency of public schooling (e.g., Deming, 2011; Hoxby & Rockoff, 2004). For example, if parents select schools based on certain undesirable qualities like the racial/ethnic composition of the student body, existing literature cannot differentiate whether these behaviors reflect parents’ true preferences or whether parents are using these qualities as proxies for other factors, like academic achievement, which is highly correlated with student demographics in some markets. Moreover, the revealed preference literature may struggle to disentangle the unique contribution of academic achievement status (i.e., a point-in-time measure that often evaluates how well students perform against a standard) and achievement growth (i.e., students’ progress over time) as parents may not have access to quality growth information or understand the difference between these two metrics.
To address these limitations, I assess parents’ preferences for school qualities using a randomized survey experiment administered on Amazon’s Mechanical Turk (MTurk) platform, a controlled environment where information and constraints can be held constant. This offers several advantages, including the ability to control the information to which parents are exposed and to construct choice sets unconstrained by geography. Specifically, I explore three questions:
How do parents rate the importance of various school quality indicators?
How does a school’s performance on various school quality indicators affect parents’ evaluation of school quality?
How does a school’s performance on various school quality indicators affect parents’ likelihood of sending their children to a school?
Additionally, I assess areas of alignment and misalignment across the research questions and differences by race/ethnicity.
Results suggest that achievement status matters most to parents when evaluating schools but that achievement growth is more important when choosing schools. These preferences are consistent across race/ethnicity. Additionally, the racial/ethnic composition of the school affects both parents’ evaluation and their selection of schools, with both White and Black/Hispanic parents preferring more diverse schools, followed by schools composed primarily of students of the same race (i.e., White parents prefer schools with mostly White students, and Black and Hispanic parents prefer schools with mostly students of color).
By estimating preferences for school qualities in an experimental environment, this study offers new insights about parents’ preferences for school qualities in the absence of misinformation and constraints. The results have important implications for the theory and practice of public accountability as they shed light on what qualities matter most to parents in an ideal market, thus providing evidence for how states and districts might design choice policies that account for these preferences.
Theoretical Framework: A Market for Schools
Advocates of school choice have long drawn on the free market metaphor to justify how school choice policies can improve the quality of public education (e.g., Friedman, 1962; Henig, 1996; Loeb et al., 2011). Free market theorists argue that allowing parents greater choice over where they send their children to school can infuse healthy competition into the market, thereby motivating schools to improve in response to market demand (e.g., Adnett & Davies, 2003; Chubb & Moe, 1990; Friedman, 1962; Hoxby, 2003; Loeb et al., 2011).
Based on the literature, Figure 1 displays a simplified theory of action for how market reforms are intended to improve education. The diagram is outlined as follows:
If parents have more choice over schools, they will enroll their children in the highest-quality schools available, conditional on preferences, information, and constraints (e.g., tuition, transportation).
In the aggregate, allowing parents to enroll their children in the highest-quality schools will create competition by rewarding high-quality schools and punishing low-quality schools with more or less student enrollment (and associated per-pupil funding dollars), respectively.
The cumulative effect will improve the overall quality of public schools by allowing higher-quality schools to enter the market, forcing lower-quality schools to innovate to attract students, and purging schools that fail to improve from the market (e.g., Henig, 1996; Hoxby, 2003).
Theory of action for how a market for schools can improve public education.
This model relies on several theoretical assumptions. Fundamentally, it assumes that parents will naturally seek to maximize their self-interest by enrolling their children in the highest-quality schools, classically defined as a school that will maximize human capital gains (Becker, 1962, 1994), for example, by making the greatest contribution to student achievement.
There are many reasons, however, to believe that this core assumption may be violated. First, parents may not have the necessary information to meaningfully evaluate schools, resulting in choices made effectively at random (for more information on parents’ interpretation of school accountability information, see Glazerman et al., 2018; J. Schneider et al., 2018; Valant, 2014; Valant & Loeb, 2015). More so, parents may choose schools based on observable differences unrelated to school quality (e.g., the racial/ethnic composition of the student body; Abdulkadiroglu et al., 2017; Ball & Vincent, 1998; Holme, 2002), which could lead to severe consequences, such as increasing segregation (Bifulco et al., 2009; Fong & Faude, 2018) or disincentivizing schools from working with marginalized populations (e.g., Davidson et al., 2015).
Additionally, parents may fail to select into their preferred schools due to constraints—for example, because they are forced to choose from a limited number of schools that are geographically proximate (Andre-Bechely, 2007; Gross et al., 2015; Hastings et al., 2005; He & Giuliano, 2018) or from a pool where their “ideal” school does not exist (e.g., a school with high academic achievement scores and where the students reflect their racial identity; see Bayer et al., 2007). Moreover, constraints may disproportionately affect parents of low socioeconomic status and parents of color (Andre-Bechely, 2007; Bayer et al., 2007; Hastings et al., 2005), a stated target population for market-based reforms (e.g., Deming, 2011; Hoxby & Rockoff, 2004).
The focus of this study is to examine one pathway of the proposed theoretical model—the contribution of parents’ preferences to their enrollment decisions (bolded in Figure 1). This, in combination with understandings about parents’ decision-making processes, can shed light on the extent to which parent choices are aligned with theoretical assumptions and policy goals.
Literature Review
The existing literature on parents’ preferences for schools falls into three primary categories: stated preferences, revealed preferences, and experimental methods. Here, I review these three literatures, ultimately arguing that each method suffers from bias, either due to self-report (stated preferences) or due to the contribution of misinformation and constraints (revealed preferences). This motivates the current study, as more experimental work is needed to understand parents’ latent preferences for school qualities, disentangled from context.
Stated Preferences
Most of what we know about how parents say they evaluate schools is inferred from their responses to survey questions. The studies reviewed in this section are relatively outdated as access to new data has allowed researchers to employ more sophisticated techniques. Notably, many of these studies were conducted before No Child Left Behind and the era of accountability and thus perhaps miscalculate the importance of test scores for today’s parents. Nonetheless, they offer a useful starting place into parents’ perspectives about how they evaluate schools.
In the aggregate, the stated preference literature suggests that parents are most interested in a school’s academic quality (e.g., test scores, teacher quality), although the type and magnitude of these preferences may vary by both survey question and parent demographic characteristics. M. Schneider and colleagues (1998), for example, found that when parents were presented with an extensive list of school characteristics (e.g., teacher quality, class size, high test scores, discipline), the largest share of parents reported teacher quality as an important factor (77% of parents), followed by high test scores (55% of parents). Similarly, Van Dunk and colleagues (1998) found that parents were most interested in curriculum and teacher quality, and Kleitz and colleagues (2000) found that parents focused on educational quality, class size, and safety when exercising school choice.
There has also long been interest in whether preferences for school qualities are consistent across subgroups of parents. While heterogeneous preferences are not inherently problematic from a theoretical perspective—indeed, some scholars argue that one benefit of market-based reforms is that they can facilitate greater school-to-student match (Friedman, 1962; Henig, 1996; Loeb et al., 2011)—systematic differences across subgroups could result in inequitable opportunity access. There is some evidence of heterogeneous preferences from the stated preferences literature. M. Schneider and colleagues (1998), for example, found that preferences for academic qualities were magnified for low-income and less educated parents, although Kleitz and colleagues (2000) found evidence of homogeneous preferences.
While the stated preferences literature offers an important first step to understanding parents’ preferences for school qualities, it suffers from several important limitations. Fundamentally, stated preference techniques are prone to bias, as respondents may answer falsely (e.g., due to social desirability bias) or be unable to predict behavior in hypothetical circumstances (Tourangeau et al., 2000).
Several qualitative studies provide evidence to this effect by underscoring the discrepancies between how parents say they evaluate schools and their actual choices. Roda and Wells (2013), for example, found that while White, upper-middle-class parents expressed preferences for schools with racially and socioeconomically diverse student bodies, they ultimately sent their children to predominantly White schools due to a perceived lack of racially diverse alternatives. Holme (2002) also found that White, middle- to upper-income parents sent their children to Whiter and more affluent schools, adding that parents primarily gathered school quality information from biased sources, including social networks or socioeconomic status symbols.
Several additional qualitative studies illuminate other factors that parents consider when choosing schools and examine how these factors are evaluated in context. Bell (2007), for example, explored parents’ geographical preferences and how they were shaped by the meaning people ascribed to locations and neighborhoods. In two other studies, Hamlin (2020) explored how parents used factors like school location and parental involvement to infer measures of school safety, and Cucchiara and Horvat (2014) explored how identity influenced choice decisions—for example, liberal urbanite parents selecting diverse, urban schools. While the purpose of this study is to isolate preferences from such contextual factors, these studies underscore confounding factors that complicate parents’ enactment of preferences in the real world.
Revealed Preferences
Revealed preferences are thought to mitigate response bias by extracting preferences from observed behavior (Richter, 1966; Samuelson, 1948; Wong, 2006). In this context, the revealed preferences literature infers parents’ preferences for school qualities by estimating the effect of school attributes on their enrollment decisions.
Before robust school choice policies, affluent parents could choose schools by purchasing homes in desirable attendance zones. Early pioneers in the revealed preferences literature for school choice exploited these behaviors to extract demand for school quality from discontinuous differences in housing prices across school attendance zone thresholds. Black (1999), for example, found that Massachusetts parents were willing to pay 2.5% more for a school with a 5% increase in test scores. Bayer and colleagues (2007) used a similar approach with Census data and found qualitatively similar results.
A limitation of these studies is that they assume that parents are paying more for test scores rather than features highly correlated with test scores (e.g., mean socioeconomic status of the student population). To determine the unique contribution of school performance data to market demand, Figlio and Lucas (2004) exploited midyear changes in school grades in Florida’s public accountability system to estimate parents’ willingness to pay for higher school ratings. While they found that discontinuous differences in school grades across attendance zones affected housing prices in the short term, the treatment effect diminished over time.
As choice policies allowed parents to more flexibly select schools, researchers increasingly gained access to rich data that could be used to estimate revealed preferences for a wider pool of attributes. For example, Hastings and colleagues (2008) estimated demand using parents’ ranking of schools in the school choice lottery in Charlotte, North Carolina. They found that parents considered both academic and nonacademic factors when selecting schools, and non-White parents were more concerned with test scores relative to White parents of similar socioeconomic status and student achievement levels. They also found that parents preferred to send their children to schools that were closer to home and where they were racially represented.
Three other studies estimated demand for school attributes using similar approaches. Using data from a centralized matching system after the elimination of school attendance zones in New Orleans post–Hurricane Katrina, Harris and Larsen (2017) found that parents preferred to send their children to schools with both higher test scores and higher school value-added scores but parents also considered nonacademic factors such as after-school care, extracurricular activities, whether the child had a sibling in the school, and proximity to home. They also found that low-income parents expressed higher preferences for nonacademic factors, presumably because they were more vulnerable to indirect school costs. Glazerman and Dotter (2017) employed a similar approach in Washington, D.C., and also found that parents selected schools based on academic factors, the demographic composition of the student body, and proximity to home, and that the magnitude of these preferences varied by parent demographics.
In a third study, Abdulkadiroglu and colleagues (2017) explored parents’ preferences for school effectiveness in New York City. They also found that parents preferred to send their children to schools that were close to home; however, they found no evidence that effectiveness predicted how parents selected schools.
In sum, findings from the revealed preference data are mixed. While there is consistent evidence that parents consider nonacademic factors when selecting schools (e.g., proximity to home, racial demographics of the student body, and after-school offerings; Abdulkadiroglu et al., 2017; Glazerman & Dotter, 2017; Harris & Larsen, 2017; Hastings et al., 2008), there is less evidence on the extent to which these factors matter relative to academic factors or vary between parents (Abdulkadiroglu et al., 2017; Glazerman & Dotter, 2017; Harris & Larsen, 2017). Furthermore, there is evidence that parents select into schools based on the demographic composition of the student body (Abdulkadiroglu et al., 2017; Glazerman & Dotter, 2017; Hastings et al., 2008), which is both unrelated to school quality and harmful for equity.
Findings about preferences for demographic characteristics are echoed by sociological literature on residential segregation. Using a nationally representative data panel on household income combined with Census data, Quillian (2002) estimated the probability of entering or exiting a given neighborhood based on neighborhood and individual characteristics. He found that on average, White people avoided mostly Black and racially diverse neighborhoods, a behavior that meaningfully contributed to residential segregation. The role of White parents in maintaining segregated schools is an important policy consideration, especially given the relationship between school segregation and low educational attainment for historically marginalized students (Owens, 2018; Quillian, 2014).
Additionally, there may be confounding factors that influence how parents choose schools. Using a survey administered to 1,699 parents in Indiana and district administrative data, Hamlin and Cheng (2020) estimated the role of parental empowerment, involvement, and satisfaction (measured by survey responses) on parents’ likelihood of selecting into charter, traditional public, and parochial schools (administrative data). They found that parents who selected into public schools reported lower feelings of empowerment, involvement, and satisfaction, underscoring the complex factors that may affect choice.
While revealed preferences can mitigate concerns of response bias, they are nonetheless susceptible to misinformation, particularly about the quality of schools and the pool of available options. For example, if parents express preferences for student demographic characteristics, this may represent their true preference, or they may be using student demographics to infer information about other qualities that they care about, like academics. Furthermore, historically marginalized groups may be less able to express their preferences given that constraints disproportionately affect these parents (Gross et al., 2015).
Experimental Survey Methods
To address these issues, a limited number of studies examined parent choice in an experimental condition. While experiments do not offer perfect predictions for parents’ real-world behavior, they are useful for estimating preferences in the absence of confounding factors, including information asymmetries and constraints. Billingham and Hunt (2016), for example, administered a survey to approximately 1,000 White respondents to estimate parents’ preferences for school racial composition, isolated from factors that may be inferred from race, like safety or academic performance. They presented respondents with information about two hypothetical schools—including the share of Black students and vignettes describing other school qualities that might be inferred from racial/ethnic demographics—and asked parents to select the school where they would most likely send their child. They found that race significantly predicted enrollment, suggesting that race was a factor, rather than a proxy, that influenced school choice.
In a second study, Houston and Henig (2019) administered a survey to approximately 5,000 respondents on Amazon’s MTurk platform to explore the effect of school achievement status and growth information on the respondents’ hypothetical decision to move into a district. This study is one of several recent studies that used a survey experiment design to estimate the effect of information on respondents’ perceptions of schools (e.g., Barrows et al., 2016; Clinton & Grissom, 2015; J. Schneider et al., 2018). The authors found that participants who received achievement growth data were less likely to move to more White and affluent districts relative to participants who received achievement status data alone.
Contribution
The existing literature on how parents choose schools suffers from several important limitations. Fundamentally, the more methodologically rigorous studies (i.e., revealed preferences studies) are unable to distinguish preferences from confounding factors in the real world, like misinformation and constraints. While several recent studies rely on sophisticated survey methods to address this limitation, they are limited in scope. I add to this literature by conducting a survey experiment designed to test parents’ latent preferences for school qualities in a controlled environment where they have ample information and unlimited schooling options.
This approach offers several advantages. First, it allows the isolation of preferences so as to explore certain contradictions in the literature, like parents’ stated preference for valuing diverse schools (Billingham & Hunt, 2016) and revealed preference for schools that are racially/ethnically and socioeconomically homogeneous (Abdulkadiroglu et al., 2017; Holme, 2002). By assessing parents’ behavior in an idealized market, I can shed light on whether parents prefer diverse schools but are constrained by the market (e.g., because high-performing diverse schools are not geographically proximate) or whether they actually prefer more homogeneous schools. I am also able to control for the information that participants receive—which is important given evidence that parents in the real world rely heavily on information from social groups (Holme, 2002)—and to remove certain constraints (e.g., cost, transportation) that may disproportionately affect some groups (Gross et al., 2015). Finally, I am able to explore school quality dimensions—like chronic absenteeism and graduation rates—that are included in many public accountability portals (Education Week, 2017) and yet thinly explored in the literature.
While surveys are useful for assessing parents’ preferences in a vacuum, they may be less useful for predicting parents’ actual behaviors. Given the complexity of school choice decisions in the real world and the time devoted to making these decisions, there may be substantive differences between how parents behave in a real versus hypothetical market. Nonetheless, exploring parents’ preferences in a controlled environment offers new insights that can help explain the observed contradictions in revealed and stated preferences, through which we can better assess the foundational assumptions undergirding school choice.
Data and Methods
This study used a randomized survey experiment design where I strategically manipulated school attributes on a hypothetical school report card to explore how parents evaluated and selected schools from an array of school quality indicators. Specifically, I explored parent preferences with regard to three research questions:
Research Question 1: How do parents rank the importance of various school quality indicators?
Research Question 2: How does a school’s performance on various school quality indicators affect parents’ evaluation of school quality?
Research Question 3: How does a school’s performance on various school quality indicators affect parents’ likelihood of sending their child to a school?
I additionally tested for differences by subpopulation (i.e., parents’ race/ethnicity) and examined alignment across the three questions, for example, whether parents weighted school characteristics differently when evaluating (Research Question 2) and selecting (Research Question 3) schools. While exploratory, misalignment between questions might suggest that parents were less aware or less reliable reporters of their preferences (i.e., if Research Question 1 differs from Research Question 2 or Research Question 3), or had different criteria for rating schools and selecting schools to enroll their children (i.e., if Research Question 2 differs from Research Question 3).
Survey Recruitment Platform
I recruited participants from MTurk, an online platform that connects employers with workers who will complete human intelligence tasks in exchange for small payments (Amazon MTurk, 2018). Two potential concerns about using MTurk for experimental research are that (1) the sample is not nationally representative and (2) the typical MTurk worker is fundamentally different from the average individual, given their willingness to complete tasks for a relatively small financial incentive (Berinsky et al., 2012; Follmer et al., 2017). Despite these concerns, MTurk is considered a relatively simple and cost-effective tool for obtaining large samples that are at least of similarly high quality as samples obtained from more traditional sampling techniques, especially when quality control metrics are used (Berinsky et al., 2012; Chmielewski & Kucker, 2020; Follmer et al., 2017; Kennedy et al., 2020; Litman & Robinson, 2020; Peer et al., 2014; Smith et al., 2016). Importantly, survey experiment studies tend to find similar effect sizes in MTurk samples compared with nationally representative samples (Coppock, 2019; Mullinix et al., 2015).
There are several methods for filtering MTurk samples for certain characteristics. Using the MTurk platform alone, a requester (i.e., the person requesting the task) is able to refine a sample to workers who meet a prespecified set of qualifications using data collected and maintained by MTurk. While MTurk’s list of qualifications is expansive, it does not contain certain important characteristics such as race/ethnicity, which are of interest to many social science researchers. To address this and other issues, CloudResearch was founded to cater the MTurk experience for academics (CloudResearch, 2019). CloudResearch (2019) maintains its own list of screening qualifications, allowing researchers to filter samples by additional criteria.
One advantage of MTurk is the ability to obtain large samples quickly and at low cost. Given budget constraints, I obtained samples specifically powered for subgroup analysis by race/ethnicity; however, comparisons of additional subgroups are ripe areas for future research.
For this study, I administered my survey to 1,227 MTurk workers in two waves. First, I administered the survey to 859 respondents on the MTurk platform, limiting my sample to MTurk workers who resided in the United States, who were parents, and who had various high-quality ratings. Second, I recruited 368 respondents through the CloudResearch platform who met all the criteria from the first wave and who identified as Black or Hispanic.
Survey Development
Because it would be infeasible in terms of both time and cognitive demand to expose parents to an expansive list of school quality indicators, I administered a preliminary survey to 200 parents on the MTurk platform with the aim of narrowing the pool of school attributes to a manageable number for my final survey instrument. In my presurvey, I asked the respondents to rate the importance of 33 school attributes on a 5-point Likerttype scale and used exploratory factor analysis to determine the underlying data patterns that guided my decisions about which qualities to retain (DeVellis, 2012). I retained one to two attributes per factor to minimize overlap and included attributes based on the criteria that they (1) had high factor-loading patterns, (2) answered policy-relevant questions (e.g., including both achievement status and achievement growth for comparison, despite loading onto the same factor), or (3) featured prominently in existing public accountability systems (e.g., graduation rates, chronic absenteeism). I describe the presurvey methodology and exploratory factor analysis in the Supplemental Appendices A to G in the online version of the journal. Ultimately, I retained 6 school attributes from my initial list of 33: academic achievement status, academic achievement growth, quality of school leadership, graduation rates, chronic absenteeism, and racial demographics of the student body.
Exploratory factor analysis is a useful tool for trimming attributes as it both provides a systematic way to identify important qualities and minimizes overlap between attributes on my final survey by grouping items with similar response patterns. Minimizing overlap is particularly important for discrete-choice experiments (DCEs), which seek to extract the greatest amount of information from the least number of choice sets (Lancsar & Louviere, 2008). Nonetheless, there are other methods that could have been used to trim the list of attributes, such as confirmatory factor analysis or a theory-driven approach, which may have resulted in a more conceptually orthogonal set of factors to investigate. An unavoidable limitation of the survey’s design is that it cannot provide information on attributes that are not included in the survey, and there are numerous other factors, such as class size, extracurricular activities, or socio-emotional curricula, that would also be apt to include in this type of experiment. While the inclusion of alternative attributes would have, by definition, yielded different results, their omission does not preclude the validity or importance of the study results. Furthermore, both input (e.g., school leadership) and output (e.g., achievement status and growth, graduation rates) attributes are included, perhaps complicating parents’ evaluation of such measures. Exploring additional school quality dimensions using similar experimental techniques is a ripe area for future research.
Final Sample and Survey Instrument
My final sample was more educated and female relative to the general population; however, many of the other variables were approximately consistent with population shares (see Table 1 for key variables of interest and the online supplementary materials for a complete list). While MTurk allowed me to limit respondents by parental status, it did not allow me to limit them based on child age, such that while all respondents included were parents, approximately 75% had children under 18 years of age. To mitigate concerns that parents of children over 18 years might systematically vary in their preferences, I ran all analyses excluding these participants and did not find major differences. The vast majority of the parents (60%) had at least one child enrolled in a public school. Given that many parents had multiple children, I pooled responses among all parents with children at all school levels (e.g., elementary and secondary).
Descriptive Statistics for the Survey Sample
Note that while all the respondents in the sample are parents, they are not all parents of school-age children. Accordingly, the share of parents with children less than 18 years old is provided. For the complete list of descriptive statistics, see the supplemental appendix in the online version of the journal.
An overview of the final survey instrument is as follows. The survey respondents experienced three survey blocks, as well as a series of demographic questions. The first block asked the respondents to rate the importance of various school attributes and was intended to gather data to address Research Question 1. The second and third blocks exposed the participants to performance report cards for hypothetical schools and asked them to rate school quality (Block 2) and to choose between schools to hypothetically enroll their children (Block 3). Blocks 2 and 3 were intended to facilitate the analysis of Research Questions 2 and 3, respectively. I randomized both the order of the blocks and the order of the questions within blocks between respondents to prevent bias stemming from response effects (Groves et al., 2009; Tourangeau et al., 2000).
Block 1
The first block was intended to capture parents’ stated preferences for school attributes using more typical survey approaches. I presented the respondents with the list of indicators selected by my presurvey and asked them to rank the factors in order of importance when choosing where to send their children to school.
Block 2
The second block was intended to capture how school performance indicators affect parents’ perception of school quality. This type of survey item represented an improvement from the questions in Block 1, as the process of evaluating schools from a series of attributes required respondents to rely on heuristics, thus revealing their subconscious preferences (Tversky & Kahneman, 1973). In the second block, I exposed the participants to a report card for a hypothetical school and asked them to rate the quality of the school on a 7-point Likert-type scale. On each indicator, the school could receive one of three possible ratings, corresponding to its relative performance in the distribution of schools. A sample question from Block 2 is shown in Figure 2.
Sample question from Block 2.
For the majority of the included indicators, the schools could receive ratings of below average, average, or above average. When these labels were not applicable (e.g., school demographics) or when they were potentially confusing (e.g., chronic absenteeism, where the indicator is reverse coded), I modified the rating levels accordingly. The universe of possible levels for each indicator is shown in Table 2. I randomly distributed all 729 possible scenarios into 35 blocks of 20–21 schools and randomly assigned respondents to one block.
Universe of Possible Ratings on School Indicators
Block 3
The third block was intended to extract parents’ preferences for school qualities from their behavior in a hypothetical market using a DCE. This block differed from Block 2 as it required parents to make trade-offs between school attributes, similar to the selection process in a real market. In the third block, I exposed respondents to accountability report cards for two hypothetical schools and asked them to pick the school where they would be most likely to enroll their child. This format is typical for DCEs, which are widely used to assess preferences in marketing, environmental, transportation, and health economics (e.g., de Bekker-Grob et al., 2012; Lancsar & Louviere, 2008).
While there are no design restrictions on the number of product attributes or levels that may be included in a DCE, options that are too complex may be so cognitively burdensome that respondents are unable to accurately assess the utility of each attribute bundle (Mangham et al., 2009). I included six school attributes, each with three possible levels, a design that is aligned to best practices from the health literature (DeShazo & Fermo, 2002; Mangham et al., 2009).
Feasibility concerns are even more pronounced in a DCE, as a full factorial design (i.e., a choice set that includes all possible combinations of attribute levels; see Mangham et al., 2009; Reed Johnson et al., 2013) would result in 265,356 total combinations of options under the current survey conditions (Huber & Zwerina, 1996; Kuhfeld, 2005; Mangham et al., 2009; Marshall et al., 2010; Ryan et al., 2012). To address feasibility issues, I used a blocked factorial design, which strategically generates choice sets to extract the maximum preference information with the fewest possible items (Mangham et al., 2009; Reed Johnson et al., 2013), quantified by the D-efficiency measure (Burgess & Street, 2005; Carlsson & Martinsson, 2003; Kuhfeld, 2005; Lancsar & Louviere, 2008; Mangham et al., 2009; Street et al., 2005). 1
I constructed the choice sets and blocks to maximize D-efficiency using the modified Fedorov algorithm (Arne, 2015; Carlsson & Martinsson, 2003; Cook & Nachtsheim, 1980; Zwerina et al., 2000). I reduced the total number of combinations to 504, which I divided among 28 blocks of 18 side-by-side school comparisons. A sample question is shown in Figure 3.
Sample question from Block 3.
Empirical Strategy
To answer Research Question 1—How do parents rank the importance of various school quality indicators?—I calculated means and frequencies of item rankings across each school attribute. Additionally, I descriptively analyzed the differences in item means between White parents and Black/Hispanic parents using t statistics, a standard measure of group difference.
For Research Question 2—How does a school’s performance on various school quality indicators affect parents’ evaluation of school quality?—and Research Question 3—How does a school’s performance on various school quality indicators affect parents’ likelihood of sending their child to a school?—I analyzed my data using a regression framework. I estimated the “effect” of each performance indicator on parents’ school quality rating (Research Question 2, using data from Block 2) and likelihood of selecting into a school (Research Question 3, using data from Block 3) using Model 1:
where
For Block 2, I fitted the model as a two-level random-intercept model (questions within people) to account for serial correlation between questions within respondents. For Block 3, I fitted the model as a conditional logit, grouping school profiles by choice sets and clustering standard errors at the person level, as recommended in the literature (e.g., Lancsar & Louviere, 2008; Ryan et al., 2012). To test whether the effect differed by subpopulation, I fitted the model separately for White parents and for Black and Hispanic parents and compared coefficients between models using the Wald chi-square test (Jann, 2008).
Two concerns with using a conditional logit model with discrete-choice data are (1) violation of the independence of irrelevant alternatives assumption, or that the relative probability of choosing between two alternatives is independent of the addition of a third alternative, and (2) failure to account for multiple observations per individual. I addressed the first concern by using binary response models in Block 3, which deliberately constrains each choice set to two options and ideally bolsters the defensibility of the independence of irrelevant alternatives assumption (Ryan et al., 2001). The person-level clustered standard errors addressed the second concern. As a robustness check, I fitted the Block 3 models using a random-parameter logit model specification, which accounts for both potential threats (Revelt & Train, 1998; Ryan et al., 2001), and did not find major differences.
Results
Given that DCEs are relatively new to the parent choice literature, it is worth outlining what these methods do and do not offer. Fundamentally, DCEs are designed to assess latent preferences, or in this context the underlying preferences on which parents select and evaluate schools. They are considered both more reliable than traditional stated preference techniques (e.g., M. Schneider et al., 1998) and less vulnerable to constraints and misunderstandings than revealed preference techniques (e.g., Abdulkadiroglu et al., 2017; Bergman et al., 2020; Harris & Larsen, 2017). DCEs thus offer a new way to examine school choice preferences that avoids some of the limitations of other popular methodologies.
Still, DCEs bring their own set of limitations, which affects the interpretation of results. Fundamentally, DCEs are likely worse predictors of real-world behavior relative to revealed preferences studies as numerous other factors—including residential location, constraints, and misinformation—influence school choice. Furthermore, these methods provide no evidence about what defines a quality school; rather, they offer new insights about what attributes parents might value.
There are two additional limitations that are useful for interpreting results. First, DCEs only allow the inclusion of a limited pool of attributes, such that other important attributes are by definition missing from the school profiles. While we might ideally interpret results as the unique contribution of each included attribute, we cannot guarantee that respondents are reacting to the survey wording itself and not some association that they have with the survey wording. This is a concern with any survey experiment. For example, if parents associate student achievement scores with the socioeconomic characteristics of the school, we cannot ensure that they are not conflating the academic indicators with some theorized socioeconomic characteristics. While the inclusion of racial/ethnic indicators likely mitigates this specific concern to some degree, the association between attributes included in and those absent from the survey may still be a factor.
A second concern is social desirability bias, or that respondents will hesitate to respond accurately for fear of expressing unfavorable attitudes. The design of Blocks 2 and 3 is thought to mitigate these concerns as respondents are asked to make a series of evaluations in rapid succession. Because such questions require respondents to rely on heuristics (i.e., subconscious strategies to simplify their decision making) to complete the task (Katz, 2006; Tversky & Kahneman, 1973), they may be less aware of the direct impact of the demographic attribute on their choice pattern. Moreover, the survey employs strategies thought to limit social desirability bias, including assuring confidentiality, using an anonymous internet survey (i.e., without an interviewer), and asking about respondent demographics only after they had completed the survey blocks (Tourangeau et al., 2000).
Research Question 1: How Parents Rank School Quality Indicators
Table 3 shows the results of the Block 1 analysis, including item means and standard deviations on each school attribute, as well as t-test results comparing White parents with Black and Hispanic parents. Recall that the respondents were asked to rank the attributes in order of importance (with 1 being the most important), such that items with the lowest means should be interpreted as the most important attributes. To examine the distribution of rankings across each attribute, I additionally provide histograms of the rank-order importance of each attribute in Figure 4.
Mean Importance Rank for School Attributes
Note. Survey question: Please rank the following school attributes in order of importance for deciding where to send your child to school, where 1 is the most important attribute and 6 is the least important attribute.
p < .5. **p < .01. ***p < .001.
Frequency of rank-order importance frequencies by school attribute.
On average, respondents cited academic achievement status as the most important attribute when deciding between schools
In the final column of Table 3, t tests comparing response patterns across attributes between White parents and Black/Hispanic parents are shown. While White and Black/Hispanic parents ranked achievement status, achievement growth, and school leadership similarly, I found significant differences in their chronic absenteeism, graduation rate, and student demographics rankings. White parents on average ranked chronic absenteeism and graduation rate as slightly more important, and Black and Hispanic parents ranked school demographics as more important.
Research Question 2: How Parents Evaluate Schools From School Quality Indicators
I present the results from my random-intercept models predicting school quality ratings in Table 4. All included attributes were significant predictors of school quality ratings, with higher scores on achievement status, achievement growth, school leadership, chronic absenteeism, and graduation rates predicting higher school quality scores. Additionally, parents rated schools with mostly students of color about 0.05 points lower on average than mostly White schools, holding all other variables constant
Multilevel Model Predicting School Quality Score With School Attributes
Note. Standard errors are in parentheses and at the person level. Question: Based on the school report card below, how would you rate the quality of School X on a scale from “Extremely Low Quality” to “Extremely High Quality”?
p < .1. **p < .05. ***p < .01. ****p < .001.
When I fitted separate models for White parents and Black and Hispanic parents, I found that responses from the two groups were statistically equivalent across most indicators (i.e., achievement growth, school leadership, chronic absenteeism, graduation rates). However, there were differences across the achievement status indicator, with achievement status mattering more for White parents
Research Question 3: How Parents Select Schools From School Quality Indicators
I present the results of the conditional logit models predicting school choice in Table 5. The predictors of school choice were similar to those of school quality ratings (Block 2), as parents expressed preferences for higher scores on achievement status, achievement growth, school leadership, and graduation rates. There were, however, some noteworthy differences. Mainly, these models suggested that achievement growth was the most important factor to parents compared with achievement status, which was more important in Block 2. For example, moving from the lowest to the middle rating on achievement status increased the odds that a parent enrolled their child in a given school by approximately 77%
Conditional Logit Predicting School Choice With School Attributes
Note. Exponentiated coefficients; standard errors are in parentheses and at the person level. Question: Based on the school report cards for School A and School B listed below, in which of these two schools would you be most likely to send your child?
p < .1. **p < .05. ***p < .01. ****p < .001.
To show the relationship between achievement status and growth another way, Figure 5 displays the predicted probabilities of school selection by achievement status for high- and low-growth schools, holding all other indicators at their means. Parents were more likely to select into schools with both higher achievement status and higher achievement growth, and the difference between the predicted probabilities for high- and low-growth schools was greatest for low–achievement status schools.
School selection by achievement status and growth.
Another difference in preferences between Block 2 and Block 3 was in chronic absenteeism. While the coefficient on chronic absenteeism remains significant in these models, it is of less importance relative to the other indicators. For example, moving from a worse than average to average rating increased the odds of enrollment by 68%
The subgroup analysis revealed several differences in response patterns between White parents and Black/Hispanic parents. On school leadership, the impact of an average or above-average school leader on the probability of school attendance was slightly higher for White parents (
To illustrate these differences, Figure 6 shows the probability of school selection by student demographics for Black/Hispanic parents and White parents. Black and Hispanic parents showed a stronger preference for diversity, and the biggest gap in preferences by parent race/ethnicity was for schools with mostly students of color, as White parents expressed much lower preferences for these schools relative to Black and Hispanic parents.
School selection by student demographics for Black/Hispanic and White parents.
Notably, I did not find significant differences between racial/ethnic subgroups in their preferences for academic qualities. This contradicts some of the findings on heterogeneous preferences in the revealed preference literature (e.g., Harris & Larsen, 2017; Hastings et al., 2008) and potentially highlights the advantages of assessing preferences in a controlled environment that alleviates potential constraints.
Limitations
This study suffers from several important limitations. Fundamentally, respondents’ behavior in a controlled environment may differ from their behavior in the real world. While I justify the survey environment precisely because I can control for real-world constraints, these methods may be less appropriate for predicting behaviors in existing markets. Furthermore, the estimated coefficients on each attribute may be confounded by associations the respondent has with the indicator, for example, with academic achievement and socioeconomic characteristics of the student body. While respondents’ interpretation of questions is always an issue in survey studies, we might cautiously interpret the estimated coefficients as parents’ preferences for their conception of each indicator rather than their preferences for each indicator in isolation.
Additionally, I deliberately constrained the number and levels of my school attributes to reduce cognitive demand. While this is a necessary feature of these types of experiments, it by definition omits important attributes, for example, socio-emotional curriculum or after-school activities. Exploring parents’ preferences around these attributes using similar methods is an important area for future research. Also, future research might consider using other methods (e.g., confirmatory factor analysis or theoretically driven approaches) to select a more conceptually orthogonal pool of attributes to explore.
Due to resource limitations, I also limited the subgroup analysis to comparisons between White and Black/Hispanic parents. Surveys targeting parents of other racial/ethnic identities, including Asian and mixed-race parents, are also critical for study. Furthermore, future research might consider how preferences differ between other subgroups, like parents who select into different school types or who have children with special education needs, and how parent preferences for school qualities have changed during and post the COVID-19 pandemic.
Discussion
In this article, I used a randomized survey experiment to explore three questions about how parents evaluate and select into schools. Returning to the path diagram outlined in Figure 1, the purpose of this article is to estimate parents’ preferences for school qualities and whether these preferences are consistent with theoretical assumptions (i.e., that parents choose schools based on their contributions to student achievement). Additionally, I explored whether preferences differ by parent race/ethnicity, which could result in undesirable school-sorting outcomes (e.g., increased racial isolation; disproportionate access to high-quality schooling). Comparison of preferences across racial/ethnic groups is also a strength of this methodology, as revealed preferences may be vulnerable to misinformation and disproportionate constraints.
Nonetheless, choice in the survey environment may differ from choice in the real world, where constraints and misinformation abound. Survey methods offer valuable insights on how parents might choose in an ideal market, yet they are less useful predictors of actual behaviors, which occur in environments far from ideal and where school choices are often closely linked to neighborhood choices. While understanding parents’ preferences in the absence of real-world confounders can offer insight on the mechanisms underlying observed behaviors, these preferences may function quite differently across choice contexts, for example, districts where attendance is and is not determined by residence. Additionally, because certain school qualities are omitted to support the DCE design (e.g., class size, extracurricular activities, or socio-emotional curricula), I cannot offer insight into parents’ relative evaluation of such dimensions or how parents make sense of school quality among more complex school quality profiles (e.g., including information from accountability portals, school visits, and social circles).
My results suggest that academic factors are important to parents, particularly achievement status and achievement growth. Interestingly, achievement status is more important to parents when evaluating school quality (Block 2) and ranking the importance of school attributes to their school choice decisions (Block 1); however, achievement growth is more important to parents when faced with a choice between schools (Block 3). This is promising because it suggests that with sufficient information, parents could be nudged into choosing schools based on achievement growth, undoubtedly a better measure of a school’s contribution to student learning than achievement status (Koedel et al., 2015).
Parents also express relatively lower preferences for chronic absenteeism when choosing schools (Block 3), despite its being of approximately equal importance in their school ratings (Block 2). Again, this highlights differences between parents’ perceptions about what makes a quality school and the factors that are important to them when selecting schools for their children. Additionally, chronic absenteeism is the nonacademic indicator chosen by many states to be tracked and featured in their accountability portals as a condition of the Every Student Succeeds Act (Education Week, 2017). While chronic absenteeism is both easily measured and likely important for school accountability systems, my findings suggest that it is less influential to parent choice decisions.
As discussed, I found multiple areas of misalignment between parents’ responses across survey blocks. While I cannot be sure of the mechanism causing these differences, it is worth speculating briefly about some potential sources. First, misalignment between how parents select and evaluate schools could echo existing qualitative work suggesting that parents choose schools based on fit, or schools with qualities that match the qualities they observe in their children (Bell, 2005; Haderlein, 2020). Examples include choosing schools with a strong socio-emotional focus for a shy child, or average-performing schools for children who are not the top performers in their classes. In this context, parents might logically believe that a high-quality school is one that enrolls high-performing students (achievement status), but they may be more sympathetic to achievement growth when thinking about the experience they want for their children. Similarly, parents may select schools that offer complementary supports to those that they provide; for example they may be less concerned with chronic absenteeism because they believe that they have more control over their children’s attendance patterns. These are important insights, both for helping parents select into academically rigorous schools and for evaluating whether market signals are consistent with accountability ideals (e.g., by rewarding contributions to student learning). It is also possible that the variations in tasks across blocks might exacerbate or mitigate certain biases, and more work is needed to better understand the observed differences.
I also find that school demographics are an important factor to parents in both their assessment and their choice of schools. This finding is problematic in and of itself, as the types of students enrolled in the school should have no effect on the school’s quality, except insomuch as segregated schools serving poor and historically marginalized populations typically have fewer resources and less access to advanced coursework relative to predominantly White schools (Owens, 2018). Furthermore, while it is both reasonable and known that parents seek schools where their children are racially represented, this finding could disincentivize schools from working with historically marginalized populations and exacerbate segregation and resource disparities. While the coefficients on demographic indicators are small compared with those on the other indicators (e.g., achievement), the patterns are noteworthy given the potential implications for equity.
Moreover, I found varying preferences for demographics across racial/ethnic groups. For White parents, perceptions of school quality decrease as the share of students of color increases. Somewhat surprisingly then, when choosing between schools, White parents are marginally more likely to seek diverse schools (i.e., a mix of White students and students of color) but much less likely to seek schools with mostly students of color, relative to mostly White schools. Black and Hispanic parents express meaningfully greater preferences for diverse schools in Block 3 but ultimately prefer schools with mostly students of color relative to mostly White schools. These findings that Black and Hispanic parents have stronger preferences for racial diversity and that White parents tend to avoid schools with large shares of minoritized students are confirmed by the existing literature (Billingham & Hunt, 2016; Ellison & Aloe, 2019; Holme, 2002; Quillian, 2014) and raise concerns about the segregation effects that could result from unfettered choice.
In contrast, I found similar preference patterns for academic indicators between White and Black/Hispanic parents. This conflicts with some of the results from the revealed preferences literature, which finds evidence of more heterogeneous preferences (Harris & Larsen, 2017; Hastings et al., 2008). While speculative, these discrepancies might reinforce the advantages of assessing preferences using experimental methods, as they are less vulnerable to aspects of existing markets including misinformation and constraints.
Conclusion
School choice reforms are often justified by an efficiency argument, as allowing parents to choose schools can infuse healthy competition into the market for schools. Yet the ability of school choice reforms to achieve their goals relies on the ability of parents to select schools that both reflect their preferences and are aligned to theoretical ideals. If parents lack the information or ability to select into high-quality schools, for example, it would be unlikely for school choice reforms to generate market mechanisms that would reward high- performing schools. Additionally, if parents hold undesirable preferences for school qualities, like preferences for the demographic composition of the student body, school choice policies may result in unintended consequences such as the segregation of schools.
From the existent literature we know that parents don’t always select into schools that are aligned to policy objectives. Namely, parents do not always select the highest-achieving school in their choice set (Abdulkadiroglu et al., 2017; Bell, 2005), and White and more affluent parents tend to select into Whiter and more affluent schools (Holme, 2002; Kimelberg & Billingham, 2013). We do not know, however, whether these choices reflect their true preferences or whether they are a symptom of the complicated process of selecting schools.
By isolating parent preferences from the conditions of their choice, this study offers evidence to this effect. Optimistically, I found that parents care about sending their children to academically rigorous schools conditional on the indicators presented. Parents express strong preferences for schools with high achievement status and growth, and growth (arguably the more accurate measure of school quality) is more important to parents when choosing between schools. At the same time, I found that parents choose schools based on demographic characteristics, with White parents avoiding schools with high shares of students of color, which echoes findings about the drivers of residential segregation (Quillian, 2002). This is particularly concerning in a DCE because we can reasonably assume that parents are expressing preferences for school demographics rather than using demographics as a proxy for other included attributes like academics.
This article has several implications for research and policy. First, given evidence that parents hold some desirable and undesirable preferences for school attributes, policymakers might think about how to design more thoughtful choice systems that take these preferences into account and that are appropriate for their local contexts. For example, given White parents’ preferences for Whiter schools, schools and districts might think about their ideal distribution of students across schools and create systems to achieve these aims. For example, if districts are concerned about creating diverse schools, they might implement policies such as quotas for historically marginalized groups or advantaging minoritized groups in choice algorithms. Moreover, districts and states might consider ways to facilitate selection of schools based on parents’ more desirable preferences, like those for achievement growth. This could include making these indicators widely available to parents and creating information campaigns to help parents better understand each distinct indicator.
Furthermore, while revealed preferences are likely a superior method for predicting choice behaviors, this article highlights potential limitations in assessing latent preferences. Notably, while parents express heterogeneous preferences for academic qualities in the real world (Bell, 2005; Hastings et al., 2008), I found evidence of more homogeneous preferences. Given what we know about constraints, this article might motivate the use of novel methods to study school choice preferences, particularly among historically marginalized groups.
This article presents evidence that complicates the choice model. At the very least, it suggests that while parents may choose schools based on academic factors they also choose based on qualities like student demographics. Accordingly, we must task researchers and policymakers to create systems more thoughtfully designed to achieve policy objectives.
Supplemental Material
sj-pdf-1-aer-10.3102_00028312211046360 – Supplemental material for How Do Parents Evaluate and Select Schools? Evidence From a Survey Experiment
Supplemental material, sj-pdf-1-aer-10.3102_00028312211046360 for How Do Parents Evaluate and Select Schools? Evidence From a Survey Experiment by Shira Alicia Korn Haderlein in American Educational Research Journal
Footnotes
Supplemental Material
Supplemental material for this article is available online.
Notes
S
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
