The Effect of the Community Eligibility Provision on the Ability of Free and Reduced-Price Meal Data to Identify Disadvantaged Students

Abstract

The Community Eligibility Provision (CEP) is a policy change to the federally administered National School Lunch Program that allows schools serving low-income populations to classify all students as eligible for free meals, regardless of individual circumstances. This has implications for the use of free and reduced-price meal (FRM) data to proxy for student disadvantage in education research and policy applications, which is a common practice. We document empirically how the CEP has affected the value of FRM eligibility as a proxy for student disadvantage. At the individual student level, we show that there is essentially no effect of the CEP. However, the CEP does meaningfully change the information conveyed by the share of FRM-eligible students in a school. It is this latter measure that is most relevant for policy uses of FRM data.

Keywords

accountability achievement gap at-risk students educational policy evaluation regression analyses quasi-experimental analysis

Introduction

The use of free and reduced-price meal (FRM) eligibility as a proxy for student disadvantage is ubiquitous in education research. Moreover, policymakers at the federal, state, and local levels have historically relied on FRM data in their efforts to monitor and regulate educational outcomes and allocate funding.¹ It is common knowledge that FRM eligibility is a noisy and coarse proxy for student poverty (Bass, 2010; Chingos, 2016; Harwell & LeBeau, 2010; Michelmore & Dynarski, 2017), but while imperfect, it has been shown to be an effective indicator of disadvantage nonetheless (Domina et al., 2018).

The Community Eligibility Provision (CEP), which is a recent policy change to the National School Lunch Program (NSLP) administered by the U.S. Department of Agriculture (USDA), gives cause for concern about the continued use of FRM data to identify disadvantaged students. The CEP allows all students in participating schools and districts to receive free meals (FMs), regardless of students’ individual circumstances. Setting aside the substantive impacts of the CEP on student outcomes, which have been studied elsewhere, our focus is on understanding its effects on data quality.² The extent to which FRM data can continue to be used to identify high-need students in the post-CEP era is a question of critical importance, as researchers and policymakers have become dependent on using these data in this capacity. Concerns about the data effects of the CEP have been raised in recent policy reports and in the popular press (Camera, 2019; Chingos, 2018; Greenberg, 2018), but to the best of our knowledge, we provide the first causal evidence documenting how the CEP has affected the ability of FRM data to identify disadvantaged students.

Our research design is based on empirical models that predict key student outcomes—test scores and attendance—using FRM data. This approach follows on recent, related work by Domina et al. (2018) and Michelmore and Dynarski (2017). Like in these previous studies, our interest is not in understanding how outcomes compare between FRM and other students in the models per se. Rather, it is in how these comparisons change when the CEP is adopted and what is implied by the changes. Evidence that students coded as FRM-eligible gain in the performance distribution relative to their more advantaged peers with the CEP in place, holding all else equal, would imply that the CEP has reduced the ability of FRM data to identify student disadvantage.

Our analysis is based on administrative microdata from Missouri. The CEP was first adopted by schools and districts in Missouri during the 2014 to 2015 school year and we construct a student data panel from 2011–2012 to 2016–2017, spanning three pre-CEP and three post-CEP years. The Missouri administrative data are inclusive of CEP recoding. This means that students who are not FRM-eligible based on individual circumstances but attend CEP schools cannot be separately identified in the post-CEP years; that is, they are coded as FRM students in the data.

We estimate models of student outcomes using data with and without imposing CEP data conditions. Although this is not possible after 2013 to 2014 in Missouri due to the CEP data overwrite, it is possible using data from years prior to the implementation of the CEP. For our analysis, we use the Missouri data to identify CEP-adopting schools in 2014 to 2015 and later, then use the pre-CEP data from 2011–2012 to 2013–2014 to compare the actual FRM data to a scenario where we recode the data as if the CEP were already in place during the prepolicy years. By comparing the results from models of student outcomes with and without CEP data coding in place from 2011–2012 to 2013–2014, holding all else equal, we identify the causal data effects of the CEP.

We focus on how the CEP data censoring affects two FRM-based variables. First, we examine individual student FRM designations, which are commonly used to control for student disadvantage in education research. Second, we examine the share of FRM-eligible students in a school. This variable is sometimes used by researchers to control for schooling context and has historically played an important role in education accountability and finance policies. As this work is exploratory, we did not set ex ante benchmarks to establish what would be meaningful minimum detectable effects (MDEs) of the CEP on the information contained by these variables. However, as a practical matter, this is of limited importance in our application because our use of the full state data set ensures a well-powered analysis.³

For students’ individual FRM designations, we find that the CEP has essentially no effect on their informational content. There are two factors that drive this null finding. First, students who experience a change in coding status due to the CEP are not a random sample—they are already a disadvantaged group, as evidenced by their attendance at high-poverty schools. While these students are “miscoded” in a technical sense because of the CEP, the substantive effect of the miscoding is modest. Second, and more importantly, we show that the number of students who experience an FRM status change due to the CEP—even in the extreme hypothetical scenario in which all eligible schools in Missouri adopt the CEP—is small. This result is not widely understood and may seem initially surprising. The explanation lies in the CEP rules, which are such that eligible schools and districts already have high shares of FRM-eligible students—about 80% on average. This means that relatively few students switch status when a school adopts the CEP. Note that this is not a Missouri-specific result, but rather, it is a product of the rules that govern CEP eligibility nationally, which we elaborate on below.

In contrast, the CEP meaningfully affects the information contained by the share of FRM-eligible students in a school. We show that the strong signal of student disadvantage conveyed by a very high FRM share in the pre-CEP period is obscured substantially with the CEP in place because a set of relatively better-off schools are coded with a 1.0 FRM share. This finding is notable because it is the school share of FRM-eligible students that is focal to finance and accountability policies targeted toward low-income students.⁴

The reason that the CEP affects the individual and school share FRM variables differently is that these variables embody different information influenced by the CEP. The main difference lies in the fact that the CEP is implemented unevenly across schools—that is, some schools shift to FRM shares of 1.0 while others stay the same. This feature of the data change is captured in the variance of the school FRM share variable, but not the individual FRM indicator. We elaborate on the mechanics in more detail below.

Our findings inform contemporary research and policy applications of FRM data. For researchers, the precise nature of the informational degradation in FRM data resulting from the CEP—that is, its effect on aggregate FRM measures but not individual measures—guides appropriate use of these data. From a policy perspective, the results increase the appeal of finding new measures of disadvantage to aid in the identification of high-need schools. In the “Discussion” section, we review available alternatives and efforts in some states to respond to the new data conditions brought on by the CEP. The alternatives that some states are using offer benefits relative to FRM data but also have limitations. Importantly, research has fallen behind policy in this area: States are reacting to the CEP by shifting away from FRM data—some more than others—but the alternatives they are shifting to have not been rigorously evaluated.

The CEP and Missouri Context

CEP Program Rules

The CEP allows high-poverty schools and districts to provide FMs (breakfast and lunch) to all students without collecting individual household applications.⁵ Eligibility for the CEP is based on the Identified Student Percentage (ISP), which must be at or above 40 to qualify. The ISP is calculated as the percentage of students who are directly certified for FM receipt via participation in other means-tested programs such as the Supplemental Nutrition Assistance Program (SNAP), Temporary Assistance for Needy Families (TANF), and the Food Distribution Program on Indian Reservations. Students can also be grouped with the directly certified population if they are classified as belonging to a particular disadvantaged group, such as foster, migrant, homeless, or runaway youth. District-collected FRM eligibility data do not factor into eligibility for the CEP.

For a given ISP value, the FRM-eligible percentage will be substantially higher for two reasons. First, while the income threshold for SNAP, a key program that leads to direct certification, is the same as that for FMs under the NSLP at 130% of the poverty line, some students from households with incomes above this threshold are eligible for reduced-price meals under the NSLP, for which the income threshold is higher (185% of the poverty line). In education research and policy applications, students eligible for FRMs are typically grouped together as “low income” students, resulting in a larger population of students identified as FRM-eligible relative to the directly certified population (Massachusetts Department of Elementary and Secondary Education [DESE], 2017). The second reason is that empirically, FRM eligibility is assigned to more students than income eligibility alone would dictate (Domina et al., 2018).

The fact that the ISP eligibility threshold corresponds to a much larger FRM percentage has implications for the number of CEP-induced changes to students’ FRM designations when the policy is adopted. For instance, in Missouri in 2013 to 2014, the year before CEP implementation—and, as a result, the last year in which the full informational value of FRM data is preserved—schools where at least 40% of students were directly certified had 79% of students coded as FRM-eligible, on average.⁶ This basic descriptive statistic previews the finding below that relatively few students change FRM status due to the CEP.

Conditional on being at or above the ISP threshold value of 40, schools and districts choose whether to participate in the CEP. Participants are reimbursed for the FMs by the USDA using a kinked formula. The meal reimbursement rate is 1.6 times the ISP, which means for a just-eligible school, the reimbursement rate is 64%. Once the ISP reaches 62.5, the reimbursement rate plateaus at 100%. After a school or district is accepted into the CEP, it can offer FMs and receive reimbursement for 4 years without the need to reapply. Our data panel covers the first 3 years of CEP implementation in Missouri—therefore, schools that we observe implementing the CEP remain covered throughout the timeframe we study.⁷

For portions of our analysis, we leverage CEP program rules to identify all CEP-eligible schools, regardless of actual participation. We define eligible schools as those with at least 40% of students who are directly certified. We use this approximation of the ISP based on direct certification data because not all schools and districts in Missouri report an ISP value.

Missouri Context

Prior to the introduction of the CEP, Missouri was a middle-ranked state (25th) in terms of the fraction of students eligible for FRMs via the NSLP (Snyder et al., 2019). To provide broader context, in the first panel of Figure 1, we use 2014 data from the Common Core of Data (CCD) to plot the distributions of the school FRM shares in Missouri and other states. The figure shows that Missouri’s distribution is not unique or anomalous. The 25th, 50th, and 75th percentiles of the Missouri distribution are 0.40, 0.56, and 0.70, compared with 0.32, 0.54, and 0.76 for the U.S. distribution as a whole.

Figure 1.

Distributions of the pre-CEP school FRM share in Missouri and other states in 2014 (left); distribution of the direct certification (DC) school share (our ISP proxy) in Missouri in 2014 (right).

Missouri is just below average in terms of CEP coverage (30th in state rankings), with about 13% of students in CEP schools.⁸ Based on data from 2016 to 2017, Missouri ranked 32nd and 36th among the 50 states in terms of the fraction of CEP-eligible schools and districts participating in the program.⁹ Thus, Missouri is slightly below average among states in terms of total participation and participation conditional on eligibility, but it is not an outlier (i.e., there are many states with similar participation patterns).¹⁰

It is also important to address the efficacy of districts’ direct certification processes in Missouri, given that direct certification data are used to determine CEP eligibility. One way to measure districts’ direct certification processes is simply to count how many districts have a process in place at all. As of the 2014 to 2015 academic year, the first year the CEP was available in Missouri, 96% of school districts were directly certifying students, which is slightly above the national average rate of 95% (Moore et al., 2016). Another way to measure this is to identify the fraction of school-aged SNAP participants statewide who are directly certified. As of the 2016 to 2017 academic year, 95% of these students were directly certified in Missouri, which is again above the national average of 92% (USDA, 2018). Our summary assessment of the direct certification processes in Missouri is that they are about average, or slightly above average, along measured dimensions. The second panel of Figure 1 plots the Missouri distribution of our proxies of schools’ ISPs based on their direct certification shares. We use the 2014 data so that the distribution is comparable to the FRM distribution in the first panel. Consistent with the preceding discussion, the (proxied) ISP distribution is clearly shifted to the left of the FRM distribution.

It is well documented nationally that, conditional on eligibility, schools with higher ISPs participate in the CEP at higher rates. This follows from the kinked incentive structure of meal replacement rates described in the previous section. In national data and again focusing on the 2016 to 2017 school year, schools with ISPs in the 40 to 49, 50 to 59, and 60+ ranges had CEP participation rates of 20.7%, 57.5%, and 74.2%, respectively (Food Research & Action Center, 2017). We find that the selection pattern is similar in Missouri, with participation rates within these same bands (using our ISP proxy) of 23.2%, 52.2%, and 80.0%, respectively.

The National Data Landscape

We are not aware of comprehensive documentation of how states are handling the collection of FRM data with the CEP in place. To gain some insight, we collected data on school FRM shares for all 50 states using the CCD in 2013 to 2014 and 2015 to 2016. This 2-year window spans CEP implementation. We identify four possible ways that CEP-induced changes to FRM data may manifest in the CCD in comparisons of states’ 2013 to 2014 and 2015 to 2016 data: (a) an increase in the share of schools listed as 100% FRM-eligible, (b) an increase in the share of schools listed with missing FRM eligibility data, (c) an increase in the share of schools listed with “not applicable” FRM eligibility data, and (d) an increase in the share of schools listed with “suppressed” FRM eligibility data.¹¹

An increase in the school share in the first category—100% FRM-eligible—between 2013 to 2014 and 2015 to 2016 for a state is the clearest indicator of a CEP-induced shift. In fact, although many schools in 2013 to 2014 had very high FRM school shares, none had a reported share of exactly 100%. In contrast, in 2015 to 2016, 11 states had at least 5% of schools listed as 100% FRM-eligible, and many more had a nonzero fraction of schools listed in this category.¹²

The CCD also indicates heterogeneity in how FRM data are reported with the CEP in place, revealed by increases in the shares of schools in the second, third, and fourth categories between 2013 to 2014 and 2015 to 2016. An extreme example is Massachusetts, where all schools reported missing FRM values in 2015 to 2016. Summing across all four categories, 16 states had more than a 5-percentage point increase in the fraction of schools identified between 2013 to 2014 and 2015 to 2016.

These data point to a clear shift in the reporting of FRM eligibility in many states after the CEP was introduced. It is noteworthy that, at least as of 2015 to 2016, CEP schools in many states still seemed to be collecting individual FRM eligibility information, although a reasonable hypothesis is that these efforts will erode over time because the data have no direct value for assigning meal status and are costly to collect. A final note of caution is that the accuracy of individual FRM data at CEP schools may be reduced because families may feel less obligated to respond, or respond accurately, to the FRM eligibility questionnaire that districts typically administer because no stakes (i.e., subsidized meals) are attached.

Data

We use student-level administrative microdata provided by the Missouri DESE for the analysis. As noted above, our data panel covers a period from 2011–2012 through 2016–2017, spanning 3 years in both directions from the first year of CEP adoptions in Missouri, 2014 to 2015 (hereafter, we refer to school years by the spring year; e.g., 2014 to 2015 as 2015). A separate school-level data file provided by the DESE indicates which schools are participating in the CEP in each year from 2015 onward. Data on student direct certification are also available from 2013 onward from the DESE.

The most critical data element is the FRM indicator variable, which is available throughout the data panel. When a school adopts the CEP, all students are coded in the data as eligible for FMs. We follow the standard practice in research and policy applications of combining free and reduced-price meal students into a single group of “FRM-eligible” students. We then assess the implications of the CEP with respect to student disadvantage as conveyed by belonging to this group. We also aggregate FRM data to the school level to assess the effect of the CEP on school-level FRM information. In addition, we briefly expand our framework to identify “free” and “reduced-price” meal students separately and assess the implications of the CEP for each data element. Finally, we use data on student race/ethnicity and gender, whether students are English language learners (ELLs), and whether students have individualized education programs (IEPs), for portions of our analysis.

We evaluate changes to the informational content of FRM data using predictive models of student attendance and achievement in math and English language arts (ELAs) in Grades 4 to 8.¹³ We define the student attendance rate as the total number of days attended divided by the total number of days enrolled, on a 0 to 1 scale.¹⁴ All test scores are standardized to have a mean of zero and a variance of one within subject-grade-year cells.

We also extend portions of our analysis to examine students in high school grades (9–12). We analyze high schools separately because of features at the high school level that imply potentially differential effects of the CEP. We expand on these features in detail below, but chief among them are (a) the concern that high school students are less likely to enroll in FRM programs voluntarily in the absence of the CEP, (b) differences in CEP eligibility rates between high schools and other schools due to lower levels of poverty among families of high school students, and (c) the fact that high school enrollment is less localized, which results in less cross-school variance in poverty. All of that said, the main themes of our results based on students in Grades 4 to 8 carry over to the high school results.

Figure 2 documents the rollout of the CEP in Missouri during our data panel for schools serving at least one grade in the 4 to 8 range. The changes over time in CEP implementation are cumulative and shown as (a) the count of schools, (b) the share of schools, and (c) the share of enrollment. The enrollment share is consistently below the share of schools, reflecting the fact that the average CEP-adopting school in Missouri is smaller than the average school statewide. This, in turn, reflects the fact that many eligible schools are in rural areas.

Figure 2.

CEP school counts and CEP coverage of schools and students in Missouri over time for schools with any combination of Grades 4 to 8.

Table 1 provides summary statistics for the student data. Our data include more than 1,700 schools with at least some coverage of tested grades and subjects (e.g., K–5, K–8, and 6–8) and more than 1.8 million student-year observations summed over the pre- and post-CEP years of the data panel. On average, during the post-CEP portion of the data panel, 11.6% of students in Grades 4 to 8 in Missouri attended a CEP school.

Table 1

Means and Standard Deviations (in Parentheses) of Key Data Elements

Variable	Pre-CEP years (2012–2014)	Post-CEP years (2015–2017)
Student outcomes
Standardized math score	0.016 (0.989)	0.011 (0.991)
Standardized reading score	−0.006 (0.986)	−0.017 (0.987)
Attendance rate	0.954 (0.046)	0.954 (0.044)
Student characteristics
Race/ethnicity: White	0.743 (0.437)	0.726 (0.446)
Race/ethnicity: Black	0.164 (0.370)	0.159 (0.366)
Race/ethnicity: Hispanic	0.050 (0.218)	0.059 (0.236)
Race/ethnicity: American Indian	0.004 (0.065)	0.004 (0.063)
Race/ethnicity: Asian/Pacific Islander	0.020 (0.139)	0.020 (0.142)
Race/ethnicity: Other	0.019 (0.137)	0.031 (0.173)
Female	0.488 (0.500)	0.488 (0.500)
English as second language (ESL)	0.030 (0.172)	0.040 (0.196)
Individualized education program (IEP)	0.123 (0.328)	0.130 (0.336)
Measures of disadvantage and CEP
FRM status (student level, a + b)	0.512 (0.500)	0.529 (0.499)
(a) Free meal status	0.434 (0.496)	0.460 (0.498)
(b) Reduced-price meal status	0.078 (0.269)	0.069 (0.253)
FRM school share (school weighted, a + b)	0.507 (0.227)	0.523 (0.256)
(a) Free meal school share	0.429 (0.219)	0.455 (0.260)
(b) Reduced-price meal school share	0.078 (0.040)	0.068 (0.043)
FRM school share distribution
FRM share ≥ 0.90	0.064 (0.245)	0.122 (0.328)
75 ≤ FRM share < 90	0.083 (0.276)	0.051 (0.221)
50 ≤ FRM share < 75	0.379 (0.485)	0.360 (0.480)
25 ≤ FRM share < 50	0.304 (0.460)	0.287 (0.452)
FRM share < 0.25	0.170 (0.376)	0.180 (0.384)
Attends CEP school	0	0.116 (0.321)
N (schools)	1,748	1,737
N (student years)	920,541	916,760

Note. The means of the standardized test scores differ slightly from zero because we standardize scores based on the full population of students but perform the analysis only for students in tested grades who are not held back. This data restriction is not substantively important for our analysis (only a small fraction of students are held back) but improves comparability of FRM and non-FRM students conceptually within grades. CEP = Community Eligibility Provision; FRM = free and reduced-price meal.***, **, and * indicate statistical significance at the 1%, 5%, and 10% level, respectively.

Method

Individual FRM Eligibility

We first aim to determine how much the CEP has degraded the proxy value of individual FRM eligibility as an indicator of student disadvantage. We focus on contemporaneous FRM information, which is the information typically used by researchers and policymakers.¹⁵

First, consider an initial regression of the following form:

Y_{i g s t} = δ_{0} + F R M_{i t} ψ_{1} + X_{it} ψ_{2} + ω_{g} + ξ_{t} + e_{i g s t} .

(1)

In Equation (1), $Y_{i g s t}$ is the outcome of interest—either a math test score, an ELA test score, or the attendance rate on a 0 to 1 scale—for student i in grade g at school s in year t. $F R M_{i t}$ is an indicator equal to one if student i is coded as FRM-eligible in year t, and $X_{it}$ is a vector of indicators for the other student characteristics shown in Table 1. Conceptually, the variables in the X-vector are no different than the FRM eligibility indicator, but we separate out $F R M_{i t}$ visually because its coefficient, $ψ_{1}$ , is focal to our analysis. $ω_{g}$ , $ξ_{t}$ , and $e_{i g s t}$ are grade fixed effects, year fixed effects, and idiosyncratic errors clustered at the school level, respectively.¹⁶

We use different data sets that reflect different CEP conditions in estimating the model in Equation (1), as discussed in Section “CEP Data Conditions and Estimation Issues” below. Our interest is in how the estimate of $ψ_{1}$ changes as CEP coverage increases. It is intuitive that the CEP will reduce the ability of $F R M_{i t}$ to identify individually disadvantaged students, and accordingly, the estimate of $ψ_{1}$ should be biased positively by the CEP. However, the magnitude of the effect is unclear. Moreover, inference from Equation (1) is complicated because $F R M_{i t}$ is a student-level indicator that categorizes students into one of the two exhaustive groups, resulting in an example of Simpson’s paradox in a direct comparison of FRM-eligible and FRM-ineligible students (Simpson, 1951).

The inference problem with Equation (1) caused by Simpson’s paradox is easiest to illustrate if we make two assumptions, both of which are reasonable. First, assume that students whose individual FRM designations change as a result of the CEP are, on average, (a) more advantaged than students who would be coded as FRM-eligible even without the CEP (i.e., always-eligible students) and (b) less advantaged than students who are never coded as FRM-eligible regardless of the CEP (i.e., never-eligible students). Second, assume that the characteristic of “advantage” referenced in the preceding sentence is positively related to student achievement and attendance. Under these two assumptions, when the CEP takes effect, the achievement and attendance outcomes for the groups of FRM and non-FRM designated students will both rise, on average. This is because the non-FRM group loses its least advantaged members and the FRM group gains new members who are relatively advantaged within the group. Depending on the shapes of the functions mapping “advantage” to outcomes at different places in the distributions and the weight of the data in each category, the outcome gaps between FRM-eligible and FRM-ineligible students—which are captured conditionally by $ψ_{1}$ in Equation (1)—could rise, fall, or remain the same when the CEP is implemented.

The fundamental reason for the ambiguity is that there is no reference group of students whose membership is unaffected by the CEP in Equation (1). When a student is recoded as FRM-eligible due to the CEP, the student necessarily leaves the FRM-ineligible group, resulting in changes in the composition of both groups. To avoid Simpson’s paradox and permit appropriate inference, we require a reference group of students unaffected by the CEP recoding that can be used to benchmark the change in performance of FRM-identified students after the CEP takes effect. We construct such a reference group using students who are themselves FRM-ineligible and who attend low-poverty schools, which we define as schools with FRM shares below 0.25 during the three prepolicy years 2012 to 2014. None of these schools should be close to eligible for the CEP based on their ISPs—recall that the average prepolicy FRM share of CEP-eligible schools in Missouri is 0.79—and indeed, empirically none are observed adopting the CEP during our data panel.

We build the reference group into Equation (1) with the following modification, as shown by Equation (1a):

\begin{array}{l} Y_{i g s t} = {\tilde{δ}}_{0} + F R M_{i t} {\tilde{ψ}}_{1} + N F R M_{i t}^{75} {\tilde{ψ}}_{2} \\ + X_{it} {\tilde{ψ}}_{3} + {\tilde{ω}}_{g} + {\tilde{ξ}}_{t} + {\tilde{e}}_{i g s t} . \end{array}

(1a)

Equation (1a) largely replicates Equation (1), and like terms and coefficients are similarly defined. The difference is that we divide students into three groups based on their individual FRM status and the FRM share of the school attended during the pre-CEP years 2012 to 2014: (a) $F R M_{i t}$ , which is equal to one if the student is individually coded as FRM-eligible and zero otherwise, (b) $N F R M_{i t}^{75}$ , which is equal to one if the student is not individually coded as FRM-eligible and the student attends a school with an FRM share between 0.25 and 1.0, and (c) $N F R M_{i t}^{25}$ , which is equal to one if the student is not individually coded as FRM-eligible and the student attends a school with an FRM share below 0.25. The latter group is the omitted reference group.

The technical value of subdividing non-FRM-eligible students by the school attended is that it breaks the flow of students from the omitted comparison group to the FRM category. That is, when the CEP takes effect, in Equation (1a), students flow from the $N F R M_{i t}^{75}$ category to the $F R M_{i t}$ category, but not from the omitted $N F R M_{i t}^{25}$ category. Thus, the composition of the omitted comparison group of non-FRM students at low-poverty schools is preserved, unlike in Equation (1). In turn, this preserves the substantive interpretation of the coefficient ${\tilde{ψ}}_{1}$ as the achievement gap between FRM-eligible students and the unperturbed (by the CEP) population of non-FRM students at low-poverty schools. If the CEP-induced coding results in a less disadvantaged pool of FRM-eligible students, ${\tilde{ψ}}_{1}$ should become less negative as more schools adopt the CEP.

The School FRM Share

Next, we estimate a version of Equation (1) where the right-hand-side variables are aggregated to the school level:

Y_{i g s t} = δ_{0} + {\bar{FRM}}_{st}^{b} δ_{1} + {\bar{X}}_{st} δ_{2} + λ_{g} + ϕ_{t} + ε_{i g s t} .

(2)

Equations (1) and (2) differ only in that Equation (2) regresses individual student outcomes on school-average student characteristics, rather than student-level characteristics. For ease of exposition, our primary model does not include the FRM share linearly but rather divides schools into bins based on the school-average FRM share. ${\bar{FRM}}_{st}^{b}$ is a vector of indicator variables that identifies the bin, b, to which school s belongs in year t. The bins are as shown in Table 1 and categorize schools with shares of FRM-eligible students of (a) 0.90 or more, (b) 0.75 to 0.89, (c) 0.50 to 0.74, (d) 0.25 to 0.49, and (e) less than 0.25. Table 1 shows summary statistics for each bin in the pre- and post-CEP periods. We omit Bin-5 schools in the model as the comparison group. We continue to cluster our standard errors at the school level.

To interpret changes in $δ_{1}$ due to the CEP, note that it increases the value of the continuous-variable ${\bar{F R M}}_{s t}$ for some schools with pre-CEP values below 1.0, which are then pushed into Bin-1. The schools that shift into Bin-1 due to the CEP are less impoverished than other Bin-1 schools. Thus, the CEP should attenuate the outcome gap between students who attend Bin-1 and other schools (i.e., make it less negative), although the magnitude of the effect is difficult to predict ex ante. A similar Simpson’s paradox phenomenon may apply to the comparisons of Bin-1 to Bins 2 and 3 in Equation (2), and to a lesser extent Bin-4 (because a small number of schools move between Bin-1 and Bin-4 under the various scenarios we consider below), but schools in Bin-5 do not change CEP status, and thus, comparisons that use Bin-5 as the anchor are straightforward to interpret.

We make three additional comments about Equation (2). First, the bin ranges we use to construct ${\bar{FRM}}_{st}^{b}$ are more differentiated at the high end of the FRM distribution to better illuminate the effects of the CEP, which primarily pushes schools from Bins 2 and 3 into Bin-1 based on program rules. A type of impact missed by the binned approach occurs for schools with FRM shares at or above 0.90 that adopt the CEP, which are coded as Bin-1 throughout. In an extension, we also estimate an analog to Equation (2) that enters the FRM school share as a linear, scalar variable to capture the effect of these moves, which we elaborate on below.

Second, we do not aggregate the outcome variables in Equation (2) in our preferred specification—that is, we use student-level outcomes as in Equations (1) and (1a). This most closely aligns Equation (2) with our objective of assessing the effect of the CEP on the ability of FRM data to identify disadvantaged students, which is inherently an individual-level prediction problem. That said, and given that the CEP treatment in Equation (2) is defined at the school level, running the same model using school-average achievement $({\bar{Y}}_{s t})$ as the dependent variable yields similar results. In fact, the results only differ because the student-level regression in Equation (2) is implicitly student-weighted, whereas the school-level analog is school weighted (with the implication that in the school-level model, small schools receive more weight in driving the parameter estimates).

Third, we also estimate variants of Equation (2) that include the individual student variables, $F R M_{i t}$ and $X_{it}$ , from above. The student-level variables improve the explanatory power of the models substantially because they are predictive of student outcomes. Moreover, because they are correlated with the school FRM share for individual students, they also reduce the predictive power of the FRM-bin indicators. However, their inclusion does not yield new substantive insights regarding the effects of the CEP on the information contained by either the student- or school-level FRM variables.

CEP Data Conditions and Estimation Issues

We estimate Equations (1a) and (2) on different data sets structured to reflect different CEP conditions. The first data set is the actual pre-CEP data set covering the years 2012 to 2014, the results from which we use to set the baseline for all our other comparisons. The other data sets are censored to implement CEP data changes prior to the actual policy, using the same years of data. We refer to our censored data sets as “pseudo-coding” the CEP.¹⁷

In the first pseudo-coded scenario, we modify the pre-CEP data for all schools that we observe adopting the CEP during the first year in Missouri (2015). Specifically, we overwrite the FRM data covering the 2012 to 2014 school years for these schools as if they had adopted the CEP during those years; that is, all students in these schools are recoded as FRM-eligible. Noting that no school had actually adopted the CEP during the pre-CEP years, we say that these schools are “pseudo-coded” to have adopted the CEP prior to actual adoption.

Using the 2012 to 2014 data, we reestimate Equations (1a) and (2) twice: once with the real pre-CEP data, where students in schools that adopted the CEP in 2015 are still distinguishable by their individual FRM status, and once with the pseudo-coded data, where all students in 2015 CEP schools are coded as FRM-eligible. By reestimating the models on the same exact data for the same exact schools in the same exact years, where the only difference is whether CEP coding rules are implemented, we can directly assess the data consequences of the CEP. Our approach holds all else constant, without ambiguity. This research design is well suited to support causal inference with regard to the data effects of the CEP and is superior—in the sense that the identifying assumptions are weaker—to a conceptually similar difference-in-differences research design.

We also extend the above-described exercise to two more pronounced scenarios. In the first of these, we pseudo-code all schools that adopted the CEP within the first 3 years in Missouri, rather than just the first year, in the 2012 to 2014 data. As would be predicted based on the slow growth in CEP adoptions after 2015 illustrated by Figure 2, this change does not meaningfully affect the findings. For the final scenario, we use the fraction of directly certified students to identify a sample of CEP-eligible schools, regardless of future adoption decisions, then pseudo-code all these schools as adopters in the pre-CEP period.¹⁸

The final scenario makes endogenous adoptions irrelevant because all eligible schools are coded as adopters. In contrast, in the first two pseudo-coded scenarios, the pseudo-coding is inclusive of endogenous uptake of the CEP. Both sets of results are informative. First, the scenarios based on real Missouri adoptions are of interest because selection into the national CEP program conditional on eligibility is negative (consistent with the incentive structure). Thus, results conditional on observed selection have real-world applicability.

The selection-free, full participation results are also of interest because they give a true upper bound on the data effect of the CEP. There are two reasons for the upper-bound interpretation. The first reason is obvious: When we pseudo-code all CEP-eligible schools, it allows for the highest level of school and student coverage based on program rules. The second, less-obvious reason is that the marginally included schools in the full-eligibility scenario have relatively fewer high-poverty students, reflecting the fact that conditional on eligibility, schools with lower direct certification rates are less likely to choose to participate. This means more students per school will experience an FRM status change at marginally added schools in the full-eligibility scenario. Moreover, the students whose coding status switches at these schools are less disadvantaged, on average, because the schools they attend are less disadvantaged.

We illustrate the importance of these two aspects of our upper-bound scenario in Figure 3. In the figure, we plot the fraction of non-FRM students against the within-school achievement gap between FRM and non-FRM students for CEP-eligible schools.¹⁹ All eligible schools are plotted, and data points for schools that we observe adopting the CEP through 2017 are overlaid with an x. Figure 3 also provides the regression line for the full sample (blue) and the regression line for the subsample of schools that adopted the CEP through 2017 (red).

Figure 3.

School non-FRM shares and within-school FRM achievement gaps (FRM minus non-FRM) in 2014, CEP-eligible schools.

The horizontal axis shows the range of the non-FRM student share at CEP-eligible schools in 2014, which is from near zero to almost 50%. It is clear that schools that ultimately adopt the CEP by 2017 (x’s), on average, have higher shares of students who are already FRM-eligible compared with schools that do not (circles). This illustrates that when we move to the upper-bound scenario that includes all eligible schools, the marginally added schools have higher internal rates of FRM status changes.

The vertical axis shows the achievement gap in math between FRM and non-FRM students in 2014 (standardized). High-poverty schools with values close to zero on the horizontal axis—that is, where the CEP has very little effect on the data because almost no students change status—have FRM and non-FRM populations with similar achievement, on average.²⁰ This is intuitive because at very high-poverty schools the small numbers of students who do not individually qualify for FRM are likely to be quite disadvantaged regardless.

As we move from left to right in the graph, the regression lines show that the within-school achievement gap between FRM and non-FRM students increases. This is also intuitive—as schools become less impoverished overall, the average incomes of non-FRM students are rising, but the FRM eligibility income thresholds are fixed, which suggests that FRM student incomes are rising more slowly. This implies that the true income gaps between FRM-eligible and FRM-ineligible students at wealthier schools are likely larger than at poorer schools. The implication is that the individual students who would potentially experience an FRM status switch at eligible but nonparticipating CEP schools are more advantaged than their counterparts at higher poverty schools that do participate.

In summary, Figure 3 shows that in the upper-bound scenario, more students are miscoded as FRM-eligible on a per-school basis because relatively fewer are individually FRM-eligible. The substantive importance of each individual miscoding is also larger at the marginally included schools because non-FRM students who attend lower poverty schools are less disadvantaged, on average.

Finally, recall from above that CEP adoptions can occur at the district or school level. For districts, each individual school does not need to be eligible for the CEP as long as the district is eligible collectively.²¹ The first two pseudo-coded scenarios capture real adoption decisions in Missouri and reflect the composition of district and school adoptions as it exists in practice. The third pseudo-coded scenario is based on identifying eligible schools to get the upper-bound effect; below we show that our results from this scenario do not differ substantively if we pseudo-code the data based on district-level eligibility instead.

Results

Individual FRM Eligibility

Table 2 shows results from the math achievement version of Equation (1a). The column headers indicate the different data sets used to estimate the equation, which reflect different CEP conditions. For each data set, models with and without the X-vector controls are estimated. The first set of results in Columns (1) and (2) use the real pre-CEP data and serve as the benchmark by which the effects of the CEP are assessed in later columns. The results for ELA achievement and attendance are substantively similar to the math results in terms of the implications of the CEP. For brevity and ease of presentation, we relegate them to the Online Appendix (Online Appendix Tables A1 and A2, available in the online version of the journal).

Table 2

Estimates of the Math Achievement Gap in Grades 4 to 8 by Individual FRM Coding Status, Various CEP Conditions

	Pre-CEPReal data		Pre-CEPPseudo-coding 1		Pre-CEPPseudo-coding 2		Pre-CEPPseudo-coding 3		Pre-CEP Pseudo-coding 3(random assignment)
Label	(1)	(2)	(3)	(4)	(5)	(6)	(7)	(8)	(9)	(10)
FRM $({\tilde{ψ}}_{1})$	−0.791	−0.604	−0.782	−0.596	−0.777	−0.592	−0.752	−0.572	−0.741	−0.561
	(0.021)***	(0.020)***	(0.021)***	(0.020)***	(0.021)***	(0.020)***	(0.021)***	(0.020)***	(0.023)***	(0.022)***
NFRM⁷⁵ $({\tilde{ψ}}_{2})$	−0.226	−0.215	−0.212	−0.211	−0.210	−0.209	−0.200	−0.203	−0.230	−0.219
	(0.020)***	(0.021)***	(0.020)***	(0.021)***	(0.020)***	(0.021)***	(0.020)***	(0.021)***	(0.022)***	(0.022)***
Other controls		Y		Y		Y		Y		Y
Share of students FRM	51.2%		52.9%		53.5%		56.5%		56.5%
Share of schools CEP	0		13.2%		16.4%		30.7%		30.7%
R ²	.104	.233	.104	.232	.103	.232	.097	.229	.086	.225
N (student years)	916,461	916,461	916,461	916,461	916,461	916,461	916,461	916,461	916,461	916,461

Note. All models include grade and year fixed effects. Standard errors clustered by school are reported in parentheses. The row labeled “FRM $({\tilde{ψ}}_{1})$ ” shows the estimated achievement gap between FRM students and non-FRM students attending schools where the FRM share was below 0.25 in 2014 (low-poverty schools). The row labeled “NFRM⁷⁵ $({\tilde{ψ}}_{2})$ ” shows the estimated achievement gap between non-FRM students attending schools where the FRM share was above 0.25 and non-FRM students attending schools where the FRM share was below 0.25. The “NFRM⁷⁵ $({\tilde{ψ}}_{2})$ ” comparison is not focal to our analysis but reported for completeness. The pseudo-coding scenarios overwrite the data during the pre-CEP period as if some schools adopted the CEP during that period: for pseudo-coding Scenarios 1, 2, and 3, the overwritten data are for schools that we observe adopting the CEP in 2015, schools that we observe adopting the CEP in 2017 or earlier, and all schools eligible for the CEP based on their ISPs, respectively. The higher numbered scenarios are inclusive of the lower numbered scenarios. The random assignment version of pseudo-coding Scenario 3 holds the CEP effect on the data fixed (in terms of the fractions of students and schools affected under that scenario) but randomly assigns schools to CEP status, regardless of the ISP. The share of students coded as FRM in each scenario is reported at the bottom of the table and calculated using data from the full sample shown in Table 1, regardless of test score availability. FRM = free and reduced-price meal; CEP = Community Eligibility Provision; NFRM, non-FRM students attending schools where the FRM share was above 0.25.

***

, **, and * indicate statistical significance at the 1%, 5%, and 10% level, respectively.

In addition to showing the regression results, Table 2 also shows how the FRM-eligible share of students in Missouri evolves under the different CEP conditions. The first pseudo-coded scenario in Columns (3) and (4), where we recode all schools that actually adopted the CEP in 2015 as if they had adopted it from 2012 to 2014, shows an increase in the FRM-eligible student share of just 1.7 percentage points, to 52.9%. This is despite the fact that 13.2% of schools are switched to CEP status. As noted above, there are two reasons for the small increase: (a) CEP-adopting schools typically have a small fraction of non-FRM-eligible students (those affected by the data change) owing to program rules and (b) the average CEP-adopting school is smaller than the average school in Missouri. The first issue is the most important driver of the small increase in FRM coverage attributable to the CEP.²²

The second scenario, in which we pseudo-code schools that adopted the CEP by the end of our data panel, only marginally increases the shares of CEP schools and FRM-coded students (to 16.4% and 53.5%, respectively), as predicted based on Figure 2. In Columns (7) and (8), we pseudo-code all CEP-eligible schools as CEP adopters. While we calculate that just more than 30% of Missouri schools are CEP-eligible, even at this upper bound, the hypothetical effect of the CEP on the share of FRM-coded students in Missouri is modest. It rises just 5.3 percentage points to 56.5%. Columns (9) and (10) explore a variant of the upper-bound scenario presented in Columns (7) and (8), which we will return to later.

Turning to the regression results in Columns (1–8), we report estimates from Equation (1a) of ${\tilde{ψ}}_{1}$ and ${\tilde{ψ}}_{2}$ , which convey average outcomes relative to the holdout group of non-FRM students who attend low-poverty schools, a group that is constant across all scenarios by construction. We focus our discussion on the parameter of interest, ${\tilde{ψ}}_{1}$ . The estimates of ${\tilde{ψ}}_{1}$ show that the CEP has essentially no effect on the ability of FRM eligibility to identify disadvantaged students as measured by test performance. For example, in the first pseudo-coded scenario, comparing the results in Column (1) with Column (3) shows that the value of ${\tilde{ψ}}_{1}$ hardly changes with the CEP coding in place from −0.791 to −0.782 student standard deviations. This implies a very small, inconsequential gain in performance for the FRM-coded population relative to the reference population of non-FRM students at low-poverty schools due to the CEP data effect. The estimates of ${\tilde{ψ}}_{1}$ decline substantially in Columns (2) and (4), relative to Columns (1) and (3), because the X-vector of other student controls removes the influence of these controls on the parameter of interest. However, comparing Columns (2) and (4) shows that the impact of the CEP remains negligible in the conditional model.

The results in Columns (5) and (6), using the second pseudo-coded scenario, are similar to the first. Moreover, even in Columns (7) and (8), where we pseudo-code the upper-bound CEP condition, there is only a very small effect of the CEP on the value of ${\tilde{ψ}}_{1}$ . It declines trivially by 0.03 to 0.04 student standard deviations on a base of 0.60 to 0.80 standard deviations, depending on whether we use the sparse or full model. All these results point toward the conclusion that the CEP does not meaningfully affect the ability of the FRM indicator to identify disadvantaged students at the individual level.

Given the limited effect of the CEP documented in Columns (1–8), the purpose of Columns (9) and (10) in Table 2 is to disentangle the two previously mentioned mechanisms that dull the CEP effect. The first mechanism is that the CEP changes FRM status for students who already attend high-poverty schools, reducing the substantive importance of miscoded FRM values. The second is that a relatively small number of students experience a status change as a result of the CEP.

The results in Columns (9) and (10) are from a modified version of the upper-bound scenario in Columns (7) and (8). The bottom rows of the table show that we hold the number of schools and students affected by CEP fixed at the same levels as in Columns (7) and (8; that is, 5.3% of students and 30.7% of schools). However, instead of pseudo-coding eligible CEP schools based on their direct certification shares as in Columns (7) and (8), in Columns (9) and (10), we randomly pseudo-code schools as CEP adopters. Thus, the number of miscoded students is held constant, but the students who experience an FRM status change in Columns (9) and (10) are no longer concentrated in high-poverty schools.

The estimates of ${\tilde{ψ}}_{1}$ in Columns (9) and (10) are somewhat less negative than those in Columns (7) and (8), as expected, but they change very little substantively. Put another way, holding the scope of the data change constant in terms of the number of students affected, even when we pseudo-code a much more advantaged student population as FRM-eligible, our estimates of ${\tilde{ψ}}_{1}$ are essentially unaffected. This indicates that the primary driver of our null results in Table 2 is the small number of students who experience an FRM status change due to the CEP.

The School FRM Share

Table 3 follows the structure of Table 2 but shows output from Equation (2). Again, we show results for math achievement in the main text and relegate the findings for ELA achievement and attendance to the Online Appendix because of their similarity (Online Appendix Tables A3 and A4). The data scenarios are the same as those in Table 2. Recall that schools are binned by the school-level FRM share in each year, with low-poverty schools (FRM share below 0.25) serving as the comparison group for the other groups of schools.

Table 3

Estimates of the Math Achievement Gaps in Grades 4 to 8 by FRM School Share Bins, Various CEP Conditions

	Pre-CEPReal data		Pre-CEPPseudo-coding 1		Pre-CEPPseudo-coding 2		Pre-CEPPseudo-coding 3		Pre-CEPPseudo-coding 3(random assignment)
Label	(1)	(2)	(3)	(4)	(5)	(6)	(7)	(8)	(9)	(10)
FRM school share Bin-1 (≥90%)	−1.140	−0.873	−0.904	−0.613	−0.859	−0.579	−0.725	−0.502	−0.656	−0.443
	(0.038)***	(0.041)***	(0.037)***	(0.037)***	(0.035)***	(0.033)***	(0.027)***	(0.024)***	(0.041)***	(0.032)***
FRM school share Bin-2	−0.713	−0.560	−0.685	−0.519	−0.679	−0.506	−0.640	−0.467	−0.693	−0.403
	(0.028)***	(0.029)***	(0.031)***	(0.033)***	(0.032)***	(0.035)***	(0.061)***	(0.055)***	(0.030)***	(0.031)***
FRM school share Bin-3	−0.435	−0.416	−0.427	−0.407	−0.424	−0.405	−0.413	−0.397	−0.439	−0.403
	(0.018)***	(0.019)***	(0.019)***	(0.019)***	(0.019)***	(0.019)***	(0.020)***	(0.020)***	(0.020)***	(0.021)***
FRM school share Bin-4	−0.224	−0.230	−0.227	−0.233	−0.227	−0.234	−0.226	−0.234	−0.228	−0.238
	(0.019)***	(0.020)***	(0.019)***	(0.020)***	(0.019)***	(0.020)***	(0.019)***	(0.020)***	(0.021)***	(0.023)***
Other controls		Y		Y		Y		Y		Y
Share of students FRM	51.2%		52.9%		53.5%		56.5%		56.5%
Share of schools CEP	0		13.2%		16.4%		30.7%		30.7%
R ²	.081	.088	.072	.083	.069	.083	.062	.082	.050	.078
N (student years)	916,461	916,461	916,461	916,461	916,461	916,461	916,461	916,461	916,461	916,461

Note. All models include grade and year fixed effects. Standard errors clustered by school are reported in parentheses. The bin categories are for FRM school shares of (a) ≥0.90, (b) 0.75 to 0.89, (c) 0.50 to 0.74, (d) 0.25 to 0.49, and (e) <0.25, as reported in the text. Bin-5 is the omitted group. The pseudo-coding scenarios overwrite the data during the pre-CEP period as if some schools adopted the CEP during that period: For pseudo-coding Scenarios 1, 2, and 3, the overwritten data are for schools that we observe adopting the CEP in 2015, schools that we observe adopting the CEP in 2017 or earlier, and all schools eligible for the CEP based on their ISPs, respectively. The higher numbered scenarios are inclusive of the lower numbered scenarios. The random assignment version of pseudo-coding Scenario 3 holds the CEP effect on the data fixed (in terms of the fractions of students and schools affected under that scenario) but randomly assigns schools to CEP status, regardless of the ISP. The share of students coded as FRM in each scenario is reported at the bottom of the table and calculated using data from the full sample shown in Table 1, regardless of test score availability. FRM = free and reduced-price meal; CEP = Community Eligibility Provision.

***

, **, and * indicate statistical significance at the 1%, 5%, and 10% level, respectively.

Unlike in the student-level models, there is a clear attenuating effect of the CEP in Table 3. In both the sparse models $(without {\bar{X}}_{st})$ and the full models, the coefficient on the Bin-1 indicator consistently declines as the influence of the CEP increases. Comparing Columns (1) and (2) with the upper-bound scenario in Columns (7) and (8) reveals a sharp change. The magnitude of the Bin-1 to Bin-5 gap—which compares schools coded with FRM shares at or above 0.90 with schools with FRM shares below 0.25—falls by about 0.40 student standard deviations. Thus, the informational content of a high school–level FRM share is clearly degraded by the CEP. We emphasize that the effects documented in Table 3 occur without any true changes in the world—they are driven entirely by whether we code FRM status using the CEP rules, holding everything else constant. This assures that the effects reflect the causal impacts of the CEP on the data.

Columns (9) and (10) again show results from the random assignment analog to Columns (7) and (8). Mirroring the generally greater impact of the CEP in the school-aggregated models in Table 3, the effect of randomly assigning CEP eligibility is also somewhat larger. Thus, a takeaway from Table 3, which is present but less visible in Table 2 due to the small overall changes in the effect sizes, is that the students who are miscoded as FRM-eligible due to the CEP are already disadvantaged in a meaningful way. Intuitively, this aspect of the CEP reduces the loss of information relative to the case where the miscoded students are from randomly selected schools.

As noted briefly in the introduction, the CEP has a larger effect on the information contained by the school FRM shares because the school shares embody CEP-induced changes in the concentration of FRM eligibility across schools. In contrast, there is no scope within the individual FRM indicator variable to capture this variation. This point can be illustrated with a counterexample. For instance, suppose that 5.3% of students changed their individual FRM status due to the CEP, but this was achieved by changing the status of exactly 5.3% of students at every school so that there was no cross-school variation in the CEP effect concentration. In this case, the ability of variation in the school FRM share to predict student outcomes would not change as a result of the CEP, even if there were a level effect that would be picked up by the individual FRM indicator. It is the effect of the CEP on cross-school variation in reported FRM eligibility, captured by the school share variable, that drives our more pronounced results in this section.²³

Finally, Online Appendix Table A5 shows results from an analog to Equation (2) where we change the dependent variable to the school-by-year average test score (averaged across all 4–8 grades in each school). As discussed above, the results are similar to what we show in Table 3 and only differ because the aggregation of the outcome implicitly reweights the data to the school rather than the student level. To confirm this, in results omitted for brevity, we find that if we estimate the school-level model as in the Online Appendix Table A5, but weight the school observations by the number of students in each school, our estimates match what we report in Table 3.

Sensitivity Analysis and Extensions

Sensitivity Analysis

We examine the sensitivity of our findings along two dimensions. First, in Equation (2), we replace the binned variable vector, ${\bar{FRM}}_{st}^{b}$ , with a continuous FRM school share variable, ${\bar{F R M}}_{s t}$ . An appealing feature of this extension is that the continuous variable captures some changes in schools’ FRM shares missed by the binned variables. Namely, if a school adopts the CEP and moves from a 0.90–0.99 FRM school share to a 1.0 FRM school share, this variation will not be captured by the binned variables but is captured by the continuous variable.

The results from the continuous-variable version of Equation (2) are shown in Table 4.²⁴ Like the results from the binned model, they point to a clear decline in the ability of the FRM school share to identify schools serving more disadvantaged student populations. The magnitudes of the coefficient changes are difficult to compare across models, but the changes between Columns (1)/(2) and (7)/(8) in Table 4 are large. For example, without the CEP in place in Column (2), a 50-percentage point increase in the FRM school share is associated with a lower math score of 0.479 student standard deviations, whereas in the upper-bound scenario in Column (8; pseudo-coding Scenario 3), this same change corresponds to a lower math score of just 0.306 student standard deviations. This result substantively mirrors the CEP data effects documented in Table 3 using the binned model.²⁵

Table 4

Estimates of the Math Achievement Gap in Grades 4 to 8 by the FRM School Share, Entered Linearly, Various CEP Conditions

	Pre-CEPReal data		Pre-CEPPseudo-coding 1		Pre-CEPPseudo-coding 2		Pre-CEPPseudo-coding 3
Label	(1)	(2)	(3)	(4)	(5)	(6)	(7)	(8)
FRM school share (linear)	−1.227	−0.958	−1.071	−0.811	−1.027	−0.763	−0.862	−0.611
FRM school share (linear)	(0.036)***	(0.034)***	(0.036)***	(0.035)***	(0.035)***	(0.034)***	(0.031)***	(0.028)***
Other controls		Y		Y		Y		Y
Share of students FRM	51.2%		52.9%		53.5%		56.5%
Share of schools CEP	0		13.2%		16.4%		30.7%
R ²	.079	.089	.074	.084	.072	.083	.064	.080
N (student years)	916,461	916,461	916,461	916,461	916,461	916,461	916,461	916,461

Note. All models include grade and year fixed effects. Standard errors clustered by school are reported in parentheses. The pseudo-coding scenarios overwrite the data during the pre-CEP period as if some schools adopted the CEP during that period: For pseudo-coding Scenarios 1, 2, and 3, the overwritten data are for schools that we observe adopting the CEP in 2015, schools that we observe adopting the CEP in 2017 or earlier, and all schools eligible for the CEP based on their ISPs, respectively. The higher numbered scenarios are inclusive of the lower numbered scenarios. The share of students coded as FRM in each scenario is reported at the bottom of the table and calculated using data from the full sample shown in Table 1, regardless of test score availability. FRM = free and reduced-price meal; CEP = Community Eligibility Provision.

***

, **, and * indicate statistical significance at the 1%, 5%, and 10% level, respectively.

Second, we estimate models that include the individual student and school-aggregated student characteristics simultaneously. We estimate two versions of a combined model: (a) a model where we add the student-level characteristics to Equation (2) as shown and (b) a model where we add them to the version of Equation (2) that enters the FRM school share linearly. The results from the former are shown in Table 5 and the results from the latter are relegated to the Online Appendix (Online Appendix Table A6). Although the simultaneous inclusion of student- and school-level FRM information reduces the predictive impact of each data element individually in all models, the effect patterns of the CEP are similar to what we show above and reveal no new substantive insights. That is, given that the CEP has such a small, inconsequential effect on the information contained by the individual FRM indicator, the combined models primarily reemphasize the point that the effect of the CEP is embodied in the FRM school share variable.²⁶

Table 5

Estimates of the Math Achievement Gap in Grades 4 to 8 by the Binned FRM School Share and Individual FRM Status Simultaneously, Various CEP Conditions

	Pre-CEPReal data		Pre-CEPPseudo-coding 1		Pre-CEPPseudo-coding 2		Pre-CEPPseudo-coding 3
Label	(1)	(2)	(3)	(4)	(5)	(6)	(7)	(8)
Individual FRM indicator	−0.477	−0.356	−0.482	−0.359	−0.482	−0.359	−0.483	−0.357
Individual FRM indicator	(0.006)***	(0.004)***	(0.006)***	(0.004)***	(0.006)***	(0.004)***	(0.007)***	(0.005)***
FRM school share Bin-1 (≥90%)	−0.774	−0.597	−0.512	−0.311	−0.466	−0.277	−0.330	−0.202
FRM school share Bin-1 (≥90%)	(0.037)***	(0.040)***	(0.036)***	(0.036)***	(0.034)***	(0.032)***	(0.027)***	(0.024)***
FRM school share Bin-2	−0.407	−0.332	−0.381	−0.296	−0.374	−0.284	−0.342	−0.244
FRM school share Bin-2	(0.027)***	(0.029)***	(0.030)***	(0.032)***	(0.031)***	(0.034)***	(0.059)***	(0.054)***
FRM school share Bin-3	−0.229	−0.258	−0.220	−0.248	−0.218	−0.246	−0.213	−0.245
FRM school share Bin-3	(0.018)***	(0.018)***	(0.018)***	(0.018)***	(0.018)***	(0.019)***	(0.019)***	(0.019)***
FRM school share Bin-4	−0.123	−0.149	−0.125	−0.151	−0.125	−0.152	−0.125	−0.153
FRM school share Bin-4	(0.018)***	(0.019)***	(0.018)***	(0.019)***	(0.018)***	(0.019)***	(0.018)***	(0.019)***
Other controls		Y		Y		Y		Y
Share of students FRM	51.2%		52.9%		53.5%		56.5%
Share of schools CEP	0		13.2%		16.4%		30.7%
R ²	.128	.242	.117	.237	.114	.236	.102	.233
N (student years)	916,461	916,461	916,461	916,461	916,461	916,461	916,461	916,461

Note. All models include grade and year fixed effects. Standard errors clustered by school are reported in parentheses. The bin categories are for FRM school shares of (a) ≥0.90, (b) 0.75 to 0.89, (c) 0.50 to 0.74, (d) 0.25 to 0.49, and (e) <0.25, as reported in the text. Bin-5 is the omitted group. The pseudo-coding scenarios overwrite the data during the pre-CEP period as if some schools adopted the CEP during that period: For pseudo-coding Scenarios 1, 2, and 3, the overwritten data are for schools that we observe adopting the CEP in 2015, schools that we observe adopting the CEP in 2017 or earlier, and all schools eligible for the CEP based on their ISPs, respectively. The higher numbered scenarios are inclusive of the lower numbered scenarios. The share of students coded as FRM in each scenario is reported at the bottom of the table and calculated using data from the full sample shown in Table 1, regardless of test score availability. FRM = free and reduced-price meal; CEP = Community Eligibility Provision.

***

, **, and * indicate statistical significance at the 1%, 5%, and 10% level, respectively.

Extensions

Free Versus Reduced-Price Meals

Next we model the data effects of the CEP on separate “free meal” (FM) and “reduced-price meal” (RM) variables. Thus far, we have used the combined FRM variable to capture membership in either group because the FRM variable is most policy relevant. The results in this section are meant to provide additional context.

The CEP converts all students in participating schools to FM-eligible. Thus, some students who were coded as RM-eligible at these schools are converted to FM-eligible (although the number of students affected by this change is relatively small—see Table 1) in addition to previously FM-ineligible and RM-ineligible students being reclassified as FM-eligible.²⁷ At the individual level, we assess the data effects of the CEP on these variables by entering them separately into a model that otherwise matches the structure of Equation (1a)—that is, we use Equation (1a) but disaggregate the FRM indicator into separate FM and RM indicators. These results are shown in Table 6. At the school level, we perform a similar disaggregation, shown in Table 7. We use the version of the school-aggregated model that enters the FM and RM school share variables linearly (as in Table 4), rather than as binned vectors, for presentational convenience.²⁸ In both tables, we report results from models of math achievement.

Table 6

Estimates of the Math Achievement Gaps in Grades 4 to 8, Separately for “Free” and “Reduced-Price” Meal Students, Various CEP Conditions

	Pre-CEPReal data		Pre-CEPPseudo-coding 1		Pre-CEPPseudo-coding 2		Pre-CEPPseudo-coding 3
Label	(1)	(2)	(3)	(4)	(5)	(6)	(7)	(8)
FMs $({\tilde{ψ}}_{1}^{F M})$	−0.841	−0.642	−0.827	−0.629	−0.820	−0.623	−0.785	−0.595
FMs $({\tilde{ψ}}_{1}^{F M})$	(0.022)***	(0.020)***	(0.022)***	(0.020)***	(0.022)***	(0.020)***	(0.022)***	(0.020)***
RMs $({\tilde{ψ}}_{1}^{R M})$	−0.512	−0.428	−0.495	−0.426	−0.492	−0.425	−0.478	−0.415
RMs $({\tilde{ψ}}_{1}^{R M})$	(0.019)***	(0.020)***	(0.019)***	(0.020)***	(0.019)***	(0.020)***	(0.020)***	(0.020)***
NFRM⁷⁵ $({\tilde{ψ}}_{2})$	−0.226	−0.216	−0.212	−0.211	−0.210	−0.209	−0.200	−0.203
NFRM⁷⁵ $({\tilde{ψ}}_{2})$	(0.020)***	(0.021)***	(0.020)***	(0.021)***	(0.020)***	(0.021)***	(0.020)***	(0.021)***
Other controls		Y		Y		Y		Y
Share of students FRM	51.2%		52.9%		53.5%		56.5%
Share of schools CEP	0		13.2%		16.4%		30.7%
R ²	.111	.236	.111	.235	.109	.234	.102	.231
N (student years)	916,461	916,461	916,461	916,461	916,461	916,461	916,461	916,461

Note. All models include grade and year fixed effects. Standard errors clustered by school are reported in parentheses. These results match the results in Table 2, except that the FRM indicator is split into separate free meal (FM) and reduced-price meal (RM) indicators. All comparative coefficients— ${\tilde{ψ}}_{1}^{F M}$ , ${\tilde{ψ}}_{1}^{R M}$ , and ${\tilde{ψ}}_{2}$ —are relative to non-FRM students attending schools where the FRM share was below 0.25 in 2014 (low-poverty schools). The pseudo-coding scenarios overwrite the data during the pre-CEP period as if some schools adopted the CEP during that period: For pseudo-coding Scenarios 1, 2, and 3, the overwritten data are for schools that we observe adopting the CEP in 2015, schools that we observe adopting the CEP in 2017 or earlier, and all schools eligible for the CEP based on their ISPs, respectively. The higher numbered scenarios are inclusive of the lower numbered scenarios. The share of students coded as FRM in each scenario is reported at the bottom of the table and calculated using data from the full sample shown in Table 1, regardless of test score availability. CEP = Community Eligibility Provision; FRM = free and reduced-price meal; NFRM, non-FRM students attending schools where the FRM share was above 0.25.

***

, **, and * indicate statistical significance at the 1%, 5%, and 10% level, respectively.

Table 7

Estimates of the Math Achievement Gaps in Grades 4 to 8, Separately by the School “Free” and “Reduced-Price” Meal Shares, Entered Linearly, Various CEP Conditions

	Pre-CEPReal data		Pre-CEPPseudo-coding 1		Pre-CEPPseudo-coding 2		Pre-CEPPseudo-coding 3
Label	(1)	(2)	(3)	(4)	(5)	(6)	(7)	(8)
Free meal school share (linear)	−1.308	−1.031	−1.070	−0.783	−1.023	−0.738	−0.875	−0.638
Free meal school share (linear)	(0.036)***	(0.039)***	(0.035)***	(0.037)***	(0.034)***	(0.034)***	(0.030)***	(0.027)***
Reduced-price meal school share (linear)	−0.353	−0.382	−0.411	−1.180	−0.598	−1.350	−1.003	−1.627
Reduced-price meal school share (linear)	(0.152)**	(0.144)***	(0.154)***	(0.149)***	(0.151)***	(0.145)***	(0.144)***	(0.146)***
Other controls		Y		Y		Y		Y
Share of students FRM	51.2%		52.9%		53.5%		56.5%
Share of schools CEP	0		13.2%		16.4%		30.7%
R ²	.083	.089	.074	.084	.071	.083	.064	.082
N (student years)	916,461	916,461	916,461	916,461	916,461	916,461	916,461	916,461

Note. All models include grade and year fixed effects. Standard errors clustered by school are reported in parentheses. The pseudo-coding scenarios overwrite the data during the pre-CEP period as if some schools adopted the CEP during that period: For pseudo-coding Scenarios 1, 2, and 3, the overwritten data are for schools that we observe adopting the CEP in 2015, schools that we observe adopting the CEP in 2017 or earlier, and all schools eligible for the CEP based on their ISPs, respectively. The higher numbered scenarios are inclusive of the lower numbered scenarios. The share of students coded as FRM in each scenario is reported at the bottom of the table and calculated using data from the full sample shown in Table 1, regardless of test score availability. CEP = Community Eligibility Provision; FRM = free and reduced-price meal.

***

, **, and * indicate statistical significance at the 1%, 5%, and 10% level, respectively.

Table 6 shows that RM students significantly outperform FM students in the pre-CEP period in math, by about 0.33 student standard deviations unconditionally (Column [1]) and 0.21 standard deviations conditionally (Column [2]).²⁹ In terms of the effect of the CEP, the finding from Table 2 that the CEP has a very limited effect on the individual data carries over to Table 6 for both the individual FM and RM controls. This is easiest to see by comparing the baseline pre-CEP results in Columns (1) and (2) with results from the upper-bound CEP adoption scenario in Columns (7) and (8). The changes in the estimates of ${\tilde{ψ}}_{1}^{F M}$ and ${\tilde{ψ}}_{1}^{R M}$ across these scenarios are modest—in the full specification, the coefficients decrease by 0.047 and 0.013, respectively.

The results in Table 7 are more difficult to interpret. The trend in the coefficient on the linear FM share is similar to what we show for the linear FRM share in Table 4. The coefficient on the RM share, in contrast, becomes more negative as the CEP takes stronger hold over the data. One reason is that the model is shifting explanatory weight that was falling on the FM share to the RM share as the information conveyed by the FM share becomes less informative. Interpreting the changes to the RM share coefficient also comes with two other caveats: It is estimated less precisely than the FM share coefficient, and its magnitude is misleading because a change from 0 to 1.0 in the RM share variable is a much larger move in the RM share distribution than the same change in the FM share variable in the FM share distribution (per Table 1).

We conclude that no new, substantive insights emerge about the data effects of the CEP from the models that split FM and reduced-meal students.

District-Level CEP Adoptions

The upper-bound condition in pseudo-coded Scenario 3 is based on the CEP eligibility of individual schools. In this section, we assess the sensitivity of our findings to reconstructing the upper-bound scenario to be based on district-level eligibility; that is, rather than coding all eligible schools as CEP adopters, we code all eligible districts as CEP adopters. If a district is eligible collectively, all schools in the district are coded as adopting the CEP (following CEP program rules). Allowing for district-level adoptions potentially increases the extent to which the CEP will degrade FRM information because within-district heterogeneity in income across schools could allow for some students who attend relatively wealthy schools (in generally, high-poverty districts) to change coded status.

We report the results from this exercise in Table 8, which are comparable to what we show under pseudo-coded Scenario 3 in Columns (7) and (8) of Tables 2 and 3. The comparison shows that our findings are similar, regardless of whether we use district- or school-level eligibility to construct the upper-bound scenario. A caveat is that Missouri has a high ratio of districts to schools (i.e., Missouri is a “small district” state), and the lack of sensitivity of our findings may not generalize to states with large districts (e.g., Florida, Maryland). That said, we note that the results in Columns (9) and (10) of Tables 2 and 3—where we randomly assign schools to CEP adoptions—will more than bound the effect of any additional heterogeneity among CEP schools owing to district-level adoptions, even in large-district states.

Table 8

Upper-Bound Effects of the CEP Based on Hypothetical District-Level, Rather Than School-Level, Adoptions

	Pseudo-coded adoptions are based on district eligibility
	Pre-CEPPseudo-coding 3 (comparable to Table 2)		Pre-CEPPseudo-coding 3 (comparable to Table 3)
Label	(1)	(2)	(3)	(4)
FRM $({\tilde{ψ}}_{2})$	−0.753	−0.570
	(0.021)***	(0.020)***
NFRM⁷⁵ $({\tilde{ψ}}_{2})$	−0.202	−0.211
	(0.020)***	(0.021)***
FRM school share Bin-1 (>90%)			−0.722	−0.446
			(0.029)***	(0.026)***
FRM school share Bin-2			−0.679	−0.544
			(0.041)***	(0.042)***
FRM school share Bin-3			−0.419	−0.414
			(0.019)***	(0.020)***
FRM school share Bin-4			−0.225	−0.239
			(0.019)***	(0.021)***
Other controls		Y		Y
Share of students FRM	56.3%		56.3%
Share of schools CEP	25.3%		25.3%
R ²	.097	.227	.062	.081
N (student years)	916,461	916,461	916,461	916,461

Note. Columns (1) and (2) are comparable to Columns (7) and (8) in Table 2, and Columns (3) and (4) are comparable to Columns (7) and (8) in Table 3. The notes to Tables 2 and 3 apply. CEP = Community Eligibility Provision; FRM = free and reduced-price meal; NFRM, non-FRM students attending schools where the FRM share was above 0.25.

***

, **, and * indicate statistical significance at the 1%, 5%, and 10% level, respectively.

High Schools

Next, we extend the analysis to high school students using two outcomes—attendance and the English II end-of-course (EOC) test score. The attendance models include students in Grades 9 to 12. The English II EOC models include students in the year they take the test, which for most students (about 90%) is Grade 10.³⁰

One reason that high schools merit separate attention is that high school students may be less likely to apply for free or reduced-price meals. The mechanism argued in the popular press is that high school students are more sensitive to the social stigma associated with participation (Pogash, 2008; Sweeney, 2018). The implication is that the CEP may generate larger changes in coded FRM eligibility among the high school population.

To explore this possibility, we use the direct certification data from the DESE to see if high school students are less likely to enroll in the NSLP conditional on the circumstances of their families. If they are, the translation between the direct certification share and the FRM share, prior to the CEP, should be weaker among high school students than among students in lower grades. But this is not the case. As noted previously, in schools covering Grades 4 to 8, we find that those with at least 40% directly certified students had an FRM share of 79%, on average, in 2014. Among Missouri high schools, the analogous FRM number is nearly the same—78%. Although social stigma has been shown to affect whether students actually receive their FMs when eligible (Schwartz & Rothbart, 2020), in terms of data on FRM eligibility, there is no indication of underreporting among high school students in Missouri when benchmarked against direct certification data.³¹

Noting this similarity across schooling levels, our investigation of high schools does uncover two notable contextual differences in the higher grades. First, a smaller fraction of high school students in total are FRM-eligible. Using data from the pre-CEP period, just 43.0% of students in Missouri high schools are FRM-eligible (see Online Appendix Table A9), compared with 51.2% of students in lower grades (per Table 2). A possible explanation for this result—conditional on the finding above that the mapping between direct certification and FRM status is similar in high school—is that families’ circumstances improve as their children age.

The second distinguishing feature of the high school sample, which is related to the first, is that many fewer high schools are eligible for and adopt the CEP. This suggests a smaller scope for the CEP to affect the data. We calculate that only 15.2% of Missouri high schools are CEP-eligible based on their direct certification shares, compared with 30.7% of schools covering Grades 4 to 8 (as in Table 2). This is because the distribution of the direct certification share among high schools has a lower mean, and a lower variance, than the distribution among schools serving lower grades. The lower mean reflects the point above that high school students’ families are not as impoverished; the lower variance is intuitive, given that high schools pool students from multiple lower grade schools, shrinking the building-level variance of student characteristics.

Findings from our analysis of high schools are reported in the Online Appendix Tables A7, A8, A9, and A10. Online Appendix Tables A7 and A8 show results using the English II EOC as the outcome, and Online Appendix Tables A9 and A10 show results for student attendance.³² The tables are structured following Tables 2 and 3 in the main text. The general insights from our analysis of Grades 4 to 8 carry over to the high school analysis. Specifically, the CEP has no substantive effect on the information contained by the individual FRM indicator, regardless of which outcome we assess (Online Appendix Tables A7 and A9). It also meaningfully reduces the informational content of the school FRM share as measured by test scores (Online Appendix Table A8) but not as measured by attendance (Online Appendix Table A10). In addition to the nonconforming finding for high school attendance, the pattern of results as CEP conditions strengthen is generally weaker in the high school analysis, suggesting more moderate CEP impacts on the information conveyed by FRM data. This is consistent with the scope for the effect of the CEP being smaller in the high school sample.³³

Discussion

What Have We Learned?

Our analysis makes three main contributions to inform our understanding of the data effects of the CEP. First, we show that the effect of the CEP on the number of students identified as FRM-eligible in Missouri is modest. The primary reason is that schools with an ISP above 40, which is the minimum level for CEP-eligibility, already have many FRM-eligible students. Specifically, we estimate that 79% of students in these schools are FRM-eligible in the absence of the CEP, on average. A 40% ISP corresponds to a much larger FRM-eligible share owing to the more stringent income threshold that primarily drives direct certification and the fact that meal subsidies are awarded more generously than income eligibility guidelines alone would imply (Domina et al., 2018). This result should generalize broadly because it is driven by the mapping between the ISP and FRM eligibility rate—it is not the product of anything anomalous about patterns of school poverty in Missouri.

The limited impact of the CEP on the number of FRM-eligible students is not widely understood. In some instances, the impact is directly misstated (e.g., Camera, 2019). A more common mistake is to imply that the entire student body at a CEP school gains access to FMs due to the CEP, without accounting for the substantial population of students who would receive free or reduced-price meals—but mostly FMs, per Table 1—even in its absence (e.g., see Food Research & Action Center, 2017, 2019; Neuberger et al., 2015)

Our second contribution, which follows from the first, is to show that the effect of the CEP on the informational content of individual student FRM status in state data is modest. We show that this is primarily driven by the small fraction of students who experience a status change because of the CEP. This result has direct implications for the use of individual FRM status to proxy for student disadvantage, which has been a widespread practice in research to date: If individual FRM status was a suitable proxy for disadvantage prior to the CEP (as suggested by Domina et al., 2018), there is no indication from our analysis that this has changed with the CEP in place.

We expect this result to generalize to other states with CEP take-up rates similar to Missouri. Moreover, the modest effect of the CEP in the upper-bound scenario in which all eligible schools—about 30% of schools in Missouri—hypothetically adopt the CEP further suggests this result will generalize to most states. A caveat is that in states with very high CEP eligibility and take-up rates, the number of students who experience a status shift could be larger than even our upper-bound scenario, and our results may not generalize in these cases (as of 2019, the Urban Institute reports that 8 of the 50 states had a CEP school participation rate above 30%: DE, IL, KY, LA, NM, NY, TN, and WV).³⁴

Our third contribution is to quantify the degree of informational degradation of the school FRM share as a proxy for disadvantaged circumstances caused by the CEP. The information loss in this variable has implications for both researchers and policymakers. For researchers, the concern is that the school FRM share is a less useful proxy for contextual disadvantage in the post-CEP era. This will be problematic in program evaluations where selection into treatment may occur along the dimension of student poverty. The use of the inferior, post-CEP school FRM variable directly as a control in a model to mitigate the influence of this type of selection, or indirectly to provide descriptive evidence of treatment-control balance outside of a model, increases the scope for undetected bias in the parameters of interest.³⁵ For policymakers, the concern is that with the CEP in place, resource and accountability policies based on FRM data—which have been ubiquitous in recent history—will not be as well targeted toward disadvantaged students.

For both researchers and policymakers working with CEP-affected data, the only comprehensive solution to recover the lost information about student disadvantage is to augment or replace the CEP-affected FRM data with alternative poverty metrics. In the next section, we describe other types of data that have been considered potential replacements for FRM data in the post-CEP era, although a current limitation with using any of the alternative measures is that we are not aware of any systematic research to vet their efficacy.

The Policy Challenge and Next Steps for Research

Recent articles, policy reports, and government reports document the variety of ways that policymakers are responding to the new data environment in the post-CEP era (Blagg, 2019; Chingos, 2016; Gindling et al., 2018; Greenberg, 2018; Greenberg et al., 2019; Grich, 2019; Massachusetts DESE, 2017). Some states continue to rely on FRM data with little or no change, while others continue to use FRM data but augment these data with other data sources. A growing number of states no longer use FRM data to identify student disadvantage at all, having entirely substituted into other metrics.

Unfortunately, states have little in the way of comprehensive research evidence to guide their responses to CEP data conditions. The most commonly advocated alternative source for identifying student disadvantage is direct certification data, which are already in use in some states (Greenberg et al., 2019). Direct certification data offer several advantages over post-CEP FRM data. Most notably, uncensored building-level values are accessible, and these data are cheaper and easier to collect because districts and states can plug into data already collected by other agencies (Grich, 2019).

However, direct certification data also have limitations. A basic concern is that the simple statistics used in state funding formulas, like the number of disadvantaged students, are affected by switching to direct certification data because direct certification rates are much lower than FRM eligibility rates. There are also more substantive issues with using direct certification to identify disadvantaged students, such as the systematic undercounting of student populations that are less likely to participate in the social safety net programs that lead to direct certification, namely, Hispanic students and undocumented immigrants (Massachusetts DESE, 2017; Zedlewski, Martinez-Schiferl, 2010). Schools and districts in states with large Hispanic and immigrant populations have the potential for measured poverty to shift markedly in a transition from FRM-based to direct certification–based metrics (Greenberg et al., 2019; Massachusetts DESE, 2017).

States and school districts are also considering other data sources to identify student disadvantage, sometimes in response to the limitations of direct certification data. One example is Medicaid data (Gindling et al., 2018; Greenberg et al., 2019). States are also using national surveys, like the American Community Survey, to construct measures of district-level poverty (Greenberg et al., 2019).

These different data sources all come with trade-offs, some that are obvious and well understood—like the undercounting of Hispanic students in direct certification data—and others that are less obvious and yet to be uncovered. There is an opportunity for researchers to contribute information to help policymakers during this time of uncertainty by rigorously evaluating alternative options for identifying high-need students. The changes states are currently pursuing, which we have summarized briefly above, all have conceptual merit. What is lacking is a comprehensive investigation of the costs and benefits of different approaches.

The next step for our work in Missouri is to adapt our analytic framework to compare the ability of different data sources, and combinations of data sources, to identify high-need students. Given the breadth of changes states are considering, and their potential implications for education finance and accountability policies nationwide, an expansive set of studies by a broad group of researchers to inform these changes would be desirable. The CEP has served as a shock to the long-standing data practices used in education to identify student disadvantage, breaking inertia in a way that would have been difficult to predict a decade ago. After the current period of change—the length of which is uncertain—it is likely that we will settle into a new inertial state in terms of how we use data to identify disadvantaged students. Research efforts that improve the next set of conditions into which policy settles can have far-reaching and long-lasting benefits.

Conclusion

Setting aside the substantive impacts of the CEP on student outcomes, there has been much consternation over how it affects the use of FRM data to identify student disadvantage in education research and policy applications. To the best of our knowledge, we present the first comprehensive, exploratory analysis designed to assess this issue empirically. Our findings are mixed. While the CEP has essentially no effect on the level of disadvantage conveyed by individual FRM eligibility, it does degrade the quality of information conveyed by the FRM-eligible share in a school. The implications of these results depend on the context in which FRM data are used.

We conclude with a brief note about the generalizability of our findings to other states. As indicated above, the first-order issues pertaining to generalizability are CEP eligibility and take-up rates. In states where eligibility and take-up rates are similar to those in Missouri, it seems likely that our substantive findings will generalize, given the structure of the CEP program. As of 2019, the Urban Institute reports that just eight states had more than 30% of schools participating in the CEP, which is the participation rate in our (hypothetical) upper-bound evaluation scenario in Missouri.

The other contextual factor that may influence the generalizability of our findings is the education governance structure in a state. In Missouri, we find no substantive differences in the upper-bound effect of the CEP, regardless of whether school- or district-level adoptions are considered. But this could be in part due to the “small district” structure in Missouri and may be less applicable to “large district” states. Noting this caveat, our analysis of the hypothetical case where schools are randomly assigned to CEP status, which surely generates more substantive miscodings in FRM eligibility than would occur even in large districts that are CEP-eligible overall, should bound the effect of any additional heterogeneity introduced by large-district adoptions. In states where the generalizability of our findings is in question and pre-CEP data are available, our analytic approach provides researchers with a blueprint for assessing the implications of the CEP, given their own local conditions.

Supplemental Material

Cory_Koedel_Online_Appendix – Supplemental material for The Effect of the Community Eligibility Provision on the Ability of Free and Reduced-Price Meal Data to Identify Disadvantaged Students

Supplemental material, Cory_Koedel_Online_Appendix for The Effect of the Community Eligibility Provision on the Ability of Free and Reduced-Price Meal Data to Identify Disadvantaged Students by Cory Koedel and Eric Parsons in Educational Evaluation and Policy Analysis

Footnotes

Acknowledgements

We thank the Missouri Department of Elementary and Secondary Education for data access and Yang An, Cheng Qian, and Jing Song for research assistance. We thank participants at the 2019 CALDER and APPAM conferences and especially Kristin Blagg, Carrie Conaway, Erica Greenberg, Michael Hurwitz, CJ Libassi, and Leigh Wendenoja for comments and suggestions. All opinions expressed in this article are those of the authors and do not necessarily reflect the views of our funders, the Missouri Department of Elementary and Secondary Education, or the institutions to which the author(s) are affiliated. All errors are our own.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The study received financial support from CALDER, which is funded by a consortium of foundations (for more information about CALDER funders, see ).

Notes

Authors

CORY KOEDEL is an associate professor in the Department of Economics and Truman School of Public Affairs at the University of Missouri. His research focuses on teacher quality and compensation, and more broadly, schooling efficacy in K–12 and higher education.

ERIC PARSONS is an associate teaching professor in the Department of Economics at the University of Missouri. His research focuses on teacher quality, high-achieving students, and more broadly, schooling efficacy in K–12 education.

References

Altindag

D. T.

Baek

Lee

Merkle

(in press). Free lunch for all? The impact of universal school lunch on student misbehavior. Economics of Education Review.

Bass

D. N.

(2010). Fraud in the lunchroom? Education Next, 10(1), 67–71.

Blagg

(2019, January 25). Which students count as low income? New national data shine light on proxy for poverty. Urban Wire: Education and Training. https://www.urban.org/urban-wire/which-students-count-low-income-new-national-data-shine-light-proxy-poverty

Camera

(2019). Miscounting poor students [U.S. news & world report]. https://www.usnews.com/news/education-news/articles/2019-01-07/why-its-getting-harder-to-count-poor-children-in-the-nations-schools

Chingos

M. M.

(2016). No more free lunch for education policymakers and researchers. Evidence Speaks Reports, 1(20), 1–4.

Chingos

M. M.

(2018). A promising alternative to subsidized lunch receipt as a measure of student poverty [Policy report]. Brookings Institute.

Domina

Pharris-Ciurej

Penner

A. M.

Penner

A. K.

Brummet

Porter

S. R.

Sanabria

(2018). Is free and reduced-price lunch a valid measure of educational disadvantage? Educational Researcher, 47(9), 539–555.

Dotter

(2013). Breakfast at the desk: The impact of universal breakfast programs on academic performance [Unpublished manuscript]. http://www.appam.org/assets/1/7/Breakfast_at_the_Desk_The_Impact_of_Universal_Breakfast_Programs_on_Academic_Performance.pdf

Food Research & Action Center. (2017, March). Community eligibility continues to grow in the 2016-2017 school year [Policy report]. https://hungerfreeok.org/food-hardship-america/

10.

Food Research & Action Center. (2019, May). Community eligibility: The key to hunger-free schools, school year 2018-19 [Policy report]. https://frac.org/research/resource-library/community-eligibility-the-key-to-hunger-free-schools-school-year-2018-2019

11.

Gindling

T. H.

Mata

Kitchin

Avila

(2018). Some causes of undercount of low income students under the Community Eligibility Provision in Baltimore City Public Schools [Department of Economics Working Paper Series Paper No. 18-01]. University of Maryland, Baltimore County.

12.

Gordon

N. E.

Ruffini

K. J.

(2018). School nutrition and student discipline: Effects of schoolwide free meals [National Bureau of Economic Research Working Paper No. 24986]. https://www.nber.org/papers/w24986

13.

Greenberg

(2018). New measures of student poverty: Replacing free and reduced-price lunch status based on household forms with direct certification [Education policy program policy brief]. Urban Institute.

14.

Greenberg

Blagg

Rainer

(2019). Measuring student poverty [Policy report]. Urban Institute.

15.

Grich

(2019, November 28). New strategies for measuring poverty in schools. FutureEd. https://www.future-ed.org/how-states-measure-poverty-in-schools/

16.

Harwell

LeBeau

(2010). Student eligibility for a free lunch as an SES measure in education research. Educational Researcher, 39(2), 120–131.

17.

Hoffman

(2012). Free and reduced-price lunch eligibility data in EDFacts: A White paper on current status and potential changes [Unpublished manuscript]. United States Department of Education. https://files.eric.ed.gov/fulltext/ED556048.pdf

18.

Massachusetts Department of Elementary and Secondary Education. (2017). Low-income student calculation study [Policy report].

19.

Michelmore

Dynarski

(2017). The gap within the gap: Using longitudinal data to understand income differences in educational outcomes. AERA Open, 3(1), 1–18.

20.

Moore

Conway

Klyer

Gothro

(2016). Direct certification in the national school lunch program: State implementation progress, school year 2014-2015 [Report to Congress]. U.S. Department of Agriculture.

21.

Neuberger

Segal

Nchako

Masterson

(2015). Take up of community eligibility this school year: More than 6 million children have better access to school meals [Policy report]. Center on Budget and Policy Priorities.

22.

Pogash

(2008, March 1). Free lunch isn’t cool, so some students go hungry. The New York Times. https://www.nytimes.com/2008/03/01/education/01lunch.html

23.

Schwartz

A.E.

Rothbart

M.W.

(2020). Let them eat lunch: The impact of universal free meals on student performance. Journal of Policy Analysis and Management, 39(2), 376–410.

24.

Simpson

E. H.

(1951). The interpretation of interaction in contingency tables. Journal of the Royal Statistical Society, Series B, 13(2), 238–241.

25.

Snyder

T. D.

de Brey

Dillow

S. A.

(2019). Digest of education statistics 2017. National Center for Education Statistics, U.S. Department of Education.

26.

Sweeney

(2018, August 20). The problem with school lunch: How the wealth gap is shaming students. Huffington Post. https://www.huffingtonpost.in/entry/school-lunches-wealth-gap_n_5b72ee42e4b0bdd0620d0b43?guccounter=1&guce_referrer=aHR0cHM6Ly93d3cuZ29vZ2xlLmNvbS8&guce_referrer_sig=AQAAAB6ILKyrHYNOt25zakKLfE5rq5GdZJt9yZVR_qYy_xK5a97pRV_jXs2Nq5mgQA0Y2x0C_epFw8J8YuFjPpvBZ7z92dl83Agu6_9RvpKZTzlSCyJgAarr3xW5KcI6V8JrL-3pS9aetED_uqC450tVBpyAvOL2el6BcKJD6as3TEP

27.

U.S. Department of Agriculture. (2018, October). Direct certification in the national school lunch program: State implementation progress report to congress—School year 2015-2016 and school year 2016-2017. https://www.fns.usda.gov/direct-certification-national-school-lunch-program-report-congress-state-implementation-progress-1

28.

Zedlewski

S. R.

Martinez-Schiferl

(2010). Low-income Hispanic children need both private and public food assistance (Policy Brief 2). Urban Institute.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.08 MB