The Hidden Costs of Corroboration: Estimating the Effects of Financial Aid Verification on College Enrollment

Abstract

Every year, the U.S. Department of Education selects hundreds of thousands of low-income students to provide additional documentation to corroborate their financial aid eligibility in a process known as verification. Although many are concerned about the potential deleterious effects of being selected, to date, studies are limited to descriptive analyses. To fill this gap in the literature, we use population-level, multicohort data to estimate the effects of financial aid verification on initial college enrollment for recent high school graduates in Tennessee. An entropy balance weighting approach indicates that students selected for verification are 3.8 percentage points (4.9%) less likely to enroll in college with underserved populations and late Free Application for Federal Student Aid (FAFSA) filers most negatively affected.

Keywords

higher education finance educational policy econometric analysis policy analysis financial aid college access

Postsecondary education plays a critical role in an individual’s economic security in the 21st century. The benefits associated with higher education have also been widely documented for communities and societies at large (Association of Governing Boards, 2017; McMahon, 2009; Oreopoulos & Salvanes, 2011). Despite the importance of postsecondary education, the college application and enrollment processes are rife with barriers to access. Financial aid and cost of attendance remain particularly perplexing factors in students’ college decisions. Information constraints about the financial aid process (Cabrera & La Nasa, 2000; Page & Scott-Clayton, 2016) and misinformation about college costs (Dynarski & Scott-Clayton, 2013; Horn et al., 2003; Scott-Clayton, 2012) present significant barriers to students. Dynarski and Scott-Clayton (2006) found that barriers in the financial aid process have a disproportionate impact on students with the least ability to pay.

Perhaps the most vexing aspect of the financial aid process is filing the Free Application for Federal Student Aid (FAFSA). Students must complete the FAFSA to determine their eligibility for federal financial aid and many state financial aid programs. It is well-documented that the FAFSA is highly complex, making it difficult for students and families to complete without assistance (Bettinger et al., 2012; Dynarski & Scott-Clayton, 2013; Page & Scott-Clayton, 2016). Our study focuses on a specific aspect of the FAFSA filing process called verification. Through the verification process, Federal Student Aid (FSA) and postsecondary institutions seek to substantiate information reported on a student’s FAFSA to ensure accuracy. FSA, an office of the U.S. Department of Education (ED), selects students for verification who are then required to submit additional documentation to their postsecondary institution’s financial aid office to confirm that the data reported on the FAFSA were correct and complete.

Figure 1 provides a visual representation of the additional steps that verification creates in the college enrollment process. Students not selected for verification simply decide whether and where to enroll based on their original financial aid offer(s). The process becomes more complex for students selected for verification. Students either comply with verification requirements or lose eligibility for their federal (and, potentially, state) financial aid. Even students who want to comply with verification requirements may fail to do so if they are unable to provide the additional documentation requested (e.g., if a dependent student is not able to get the required documentation from his or her parents). Students who comply with verification requirements may see a change to their expected family contribution (EFC) based on the additional documentation they provide, which may result in an adjusted financial aid offer. At this point, students with a new EFC decide whether to enroll based on the adjusted financial aid offer. Students who do not experience a change in EFC decide whether to enroll based on the original financial aid offer.

Figure 1.

College enrollment process including selection for verification.

Our study aims to comprehensively assess the effect of selection for verification on postsecondary enrollment for recent public high school graduates in Tennessee. Our review of the extant literature on this topic suggests that our study is the first to go beyond descriptive documentation of the disparate impacts of verification to provide estimates of the effect of selection for verification on students’ college enrollment decisions. These analyses rely on a population-level, multiyear data set of Tennessee public high school graduates who filed the FAFSA and took the ACT exam at least once. Access to the academic information and questionnaire data from the ACT enabled us to learn more about who is selected for verification and how it impacts postsecondary enrollment across different sectors. This data set also enabled us to test for effect heterogeneity by specifying models conditional on students’ predicted probabilities of college enrollment, as detailed in the methods section. Accordingly, this study is framed by the following research questions:

What is the effect of selection for verification on postsecondary enrollment for recent high school graduates in Tennessee?

Is there any evidence of effect heterogeneity when accounting for students’ predicted probabilities of college enrollment and when they filed the FAFSA?

Does selection for verification impact the sector in which a student enrolls?

The purpose of verification is to safeguard taxpayers’ investment in postsecondary access by ensuring that federal financial aid is awarded only to eligible students and the awarded amounts aligned with students’ documented levels of need. To protect the integrity of the verification process, FSA releases very little information about how students are selected for verification (Keller, 2017; Smith, 2018; Warick, 2018a, 2018b). FSA has confirmed that most students are selected for verification through a targeted selection process relying on a risk score from a machine learning model (FSA, 2019; Keller, 2017; “Verification,” n.d.). Students with a higher risk score are more likely to have incorrect FAFSA information and, thus, are more likely to be selected for verification (FSA, 2019). Importantly, though, students can also be randomly selected for verification (Douglas-Gabriel, 2017; FSA, 2019; Keller, 2017). The goal of this machine learning–based approach is to flag students who are likely to have incorrect FAFSA information by identifying discrepancies within a student’s FAFSA or between the student’s FAFSA and the information FSA collects from other federal agencies (FSA, 2019; Keller, 2017; “Verification,” n.d.). In each award year, FSA publishes a list of targeted verification items, acceptable documentation for verification, and a deadline for students to complete the verification process in the Federal Register. The documents required for verification vary by year and student but may include income tax documents, high school transcripts, or even college enrollment documentation for other household members (FSA, 2013, 2014, 2015, 2016, 2017; “Verification,” n.d.). The specific verification items for each award year in our data set are listed in Supplementary Appendix Table 1 in the online version of the journal.

There are two recent changes to the FAFSA filing process that are worth noting, given their potential relevance to our study. For the first four cohorts in our sample, the FAFSA opened on January 1, and students were required to provide tax information from the prior calendar year. For the high school graduating class of 2017, the FAFSA opened on October 1, 2016. This change (i.e., moving from January 2017 to October 2016) meant that filers for the academic year 2017–2018 used income tax information from calendar year 2015 (prior-prior year). The second important consideration for this analysis is the suspension of the Data Retrieval Tool (DRT), which allows students to import their tax information directly from the Internal Revenue Service (IRS) into the FAFSA. Due to potential vulnerability in taxpayer information, the DRT function was suspended on March 3, 2017, and remained unavailable through the rest of the 2017–2018 cycle. The DRT function was restored on the FAFSA opening date for the 2018–2019 cycle (October 1, 2017). Because tax information had to be manually entered by students completing the FAFSA rather than being directly imported from the IRS (resulting in a higher likelihood of error during data entry), the suspension of the DRT function may contribute to the higher verification rate observed for the high school graduating class of 2017 (Douglas-Gabriel, 2017). Despite the suspension of the DRT, our estimates of the effects of verification on postsecondary enrollment remained consistent in 2017 compared with previous years.

Financial aid offices at postsecondary institutions are responsible for communicating with students about verification and requesting the necessary documentation. This process typically begins in the spring or summer for students planning to enroll in the fall, though the process can now begin earlier due to the extended FAFSA cycle since award year 2017–2018. Historically, FSA required institutions to verify no more than 30% of aid applicants, though institutions had the authority to verify additional applicants or items as needed (Cochrane et al., 2010; National Association of Student Financial Aid Administrators [NASFAA], 2018; Warick, 2018a). Beginning in award year 2012–2013, however, FSA removed the 30% cap and now requires institutions to verify all students who are selected for verification (NASFAA, 2018; Warick, 2018a). The removal of the 30% cap places a disproportionate burden on 2-year institutions, many of which have a higher share of students selected for verification relative to 4-year institutions (NASFAA, n.d.). Specifically, in a survey of 45 institutions representing 700,000 students, NASFAA found that all of the 2-year institutions that responded had a verification rate more than 30% in award year 2018–2019, with an average selection rate of 37% (NASFAA, n.d.). Beginning with award year 2019–2020, FSA set a new goal of selecting approximately 22% of aid applicants for verification each year (FSA, 2019). Although institutions are still required to verify all students who are selected by FSA, decreasing the target verification rate from 30% to 22% has already reduced the burden of verification for institutions and students in its first year of implementation (FSA, 2019).

Beyond the administrative burden imposed upon institutions, verification also affects the students who are selected, namely, low-income students. Historically, while FSA has strived to select about 30% of all aid applicants for verification, more than 50% of Pell-eligible students are selected for verification in a typical award year (DeBaun, 2018; NASFAA, 2018; Warick, 2018a). This is by design; verification is intended to protect taxpayers against improper disbursement of need-based Pell grants, so naturally, low-income students who are ostensibly eligible to receive a Pell grant are largely the students who are selected for verification. As further evidence of the disparate impact of selection for verification, many researchers have estimated the extent of “verification melt,” defined as the percent of FAFSA filers who drop out of the process and, thus, are unable to access federal aid. Estimates of verification melt range from 22 to 28 percentage points, with a Pell receipt rate of about 55% for students selected for verification and about 80% for students not selected for verification (DeBaun, 2018; Smith, 2018; Warick, 2018a). Although FSA recognizes these figures as feasible, they report verification melt rates of around 11% (FSA, 2019). The amount of money these students forfeit because they do not complete verification is substantial. Martorell and Friedmann (2018) estimate that the 20% of students at California community colleges who appeared to be Pell-eligible in fall 2014 but did not receive a Pell disbursement would have been eligible for approximately US$130 million in federal financial aid. Moreover, if state financial aid programs require students to complete verification, as is the case for most state financial aid programs in Tennessee, then students who do not complete verification will also lose access to state aid.

Although the evidence is clear that verification has costs for both institutions and students, the benefits of verification are impossible to quantify with the current data publicly available. Certainly, the intent of verification is not to harm students or punish institutions. The verification process was designed to ensure that financial aid goes to the students who need it most. The federal government wants to identify and correct any misreported information on the FAFSA (whether intentional or unintentional) so that the Pell grant and other need-based Title IV programs can achieve their goal of increasing college access for low-income students.

That said, there is no way to definitively know whether verification prevents improper disbursement of federal financial aid because ED does not make data about the characteristics or outcomes of selected students available to the public. NASFAA (n.d.) found that 84% of verified applicants at surveyed institutions (91% of verified applicants at 2-year public institutions) did not experience an EFC adjustment large enough to change their Pell grant award, suggesting that financial aid fraud may not be pervasive. However, ED has reported a relatively high share of improper payments relative to total program funding for Pell grants in recent years. In fiscal year 2017, this share was 8.21% of Pell grant funding, resulting in improper payments totaling US$2.21 billion (ED, 2017). Although the numbers were similar in fiscal year 2018 (ED, 2018), ED reported much lower numbers in fiscal year 2019, potentially due to corrective action implemented in the previous year. For fiscal year 2019, the share of improper payments was 2.23% of Pell grant funding, totaling US$646.14 million (ED, 2019). Perhaps the more interesting finding from ED’s financial report, however, is that a large share of the improper payments in fiscal years 2017 and 2018 can be attributed to administrative or process errors (63% and 95%, respectively), not failure to verify financial data (ED, 2017; ED, 2018). This share is much lower in fiscal year 2019 (17%), which again may be attributable to corrective action taken in fiscal year 2018 (ED, 2019). These data provide a scope for the potential benefit of verification, but without robust data about verification’s impact on financial aid awards and student outcomes, it is difficult to estimate the true taxpayers’ financial benefit of verification.

Literature Review

FAFSA Filing

The federal government leverages several different types of aid, including grants, loans, and tax subsidies, to support students’ access to higher education (Dynarski & Scott-Clayton, 2013; Serna, 2016). The FAFSA is the keystone to accessing many of these federal resources, specifically the Pell grant, work-study funds, and subsidized loans. The significance of the FAFSA for access to federal financial aid is clear, but the form itself is challenging to complete (Bettinger et al., 2012; Page & Scott-Clayton, 2016), and the output can be difficult to interpret (Dynarski & Scott-Clayton, 2013). This can be further confounded by students’ and parents’ misunderstanding about college costs, which is exacerbated for first-generation and low-income students (Horn et al., 2003; Scott-Clayton, 2012).

Bettinger et al. (2012) conducted a randomized controlled trial providing students and families with FAFSA assistance when filing their taxes. They found a substantive increase in FAFSA completion and an 8 percentage point increase in a student’s probability of attending college (Bettinger et al., 2012). In a text messaging intervention in Texas, Page et al. (2020) sought to inform students of important FAFSA filing steps and priority deadlines, provide feedback on the aid process, and facilitate communication between students and high school counselors to improve FAFSA filing among high school seniors. The authors found modest effects on FAFSA submission and completion with stronger effects for earlier FAFSA filing and timely college enrollment. McKinney and Novak (2015) unpack FAFSA filing further by investigating the importance of time for FAFSA submission, finding that later filing has a clear negative association with average total state and institutional aid receipt. Both interventions and recent federal policy changes have begun to alleviate barriers in FAFSA filing and completion, though more work remains to ensure equitable access for all students.

Although FAFSA completion rates are on the rise, we know that many high school seniors are still not filing the FAFSA. Page and Scott-Clayton (2016) estimate that among 30% of nonfilers, one third would qualify for a Pell grant. Similarly, Page et al. (2017) found FAFSA completion rates to be significantly lower in high-poverty areas. FAFSA nonfiling is also relevant to currently enrolled students. McKinney and Novak (2015) found that a large share of community college attendees did not file a FAFSA, compared with smaller shares at public and private 4-year institutions. These findings suggest that students who stand to benefit most from financial aid are often those least informed about the financial aid process and least likely to complete the FAFSA. This reaffirms the need to expand awareness of federal financial aid and assistance with the FAFSA process to groups of students who have traditionally not participated in higher education.

FAFSA Verification

The current literature on verification is sparse. Because the purpose of verification is to ensure federal financial aid is being spent on students with financial need, we know that the majority of students selected for verification are low-income and eligible for the Pell grant or a need-based Stafford Loan (Cochrane et al., 2010; Evans et al., 2017). A study by Cochrane et al. (2010) found that across 13 California community colleges, students selected for verification were about 7% less likely to receive a Pell grant than students not selected for verification. However, 91% of students who completed the verification process ultimately received a Pell grant (Cochrane et al., 2010). Although not focused on quantifying and exploring the effects of verification, Page et al. (2020) found that students flagged for verification were about 5 percentage points less likely to enroll in college on time than nonflagged students. Taken together, these findings suggest that verification may inhibit students from accessing financial aid. Considering, however, that most students who complete verification do not experience a large enough change in EFC to alter their aid eligibility suggests that verification creates an unnecessary barrier to financial aid access for low-income students (Cochrane et al., 2010).

Evans et al. (2017) provide a more comprehensive investigation of EFC change using student-level administrative data from an anonymous 4-year public institution on the west coast. Evans et al. (2017) found that about half of all students selected for verification experience a change in EFC and that verification results in both increases and decreases in EFC. Of all students selected for verification, 32.2% had increased EFCs, whereas 19.4% saw decreased EFCs. Evans et al. (2017) also found that selection for verification results in a significant increase in the likelihood that a student will experience a change in EFC (27.8 percentage points) and Pell eligibility status (6.1 percentage points) compared with their peers who were not selected for verification (Evans et al., 2017).

Besides being a burdensome and bureaucratic process for students, verification is also an expensive and time-consuming process for institutions. In a survey of more than 600 college financial aid professionals, only a third of respondents indicated that the time and effort that students and schools spend on verification is reasonable (The Institute for College Access and Success [TICAS], 2016). In terms of its financial burden, Cochrane et al. (2010) estimate that verification cost the 13 community colleges in their sample between US$1.7 million and US$2.5 million in 2007–2008. National estimates indicate that institutions spend approximately US$432 million each year to verify FAFSA information (Asher, 2009; Davidson, 2015).

In sum, we know relatively little about the impact of verification on students, despite the fact that it affects hundreds of thousands of students each year and may change a student’s eligibility for aid. Failure to comply with the requirements of verification makes a student ineligible for Title IV federal financial aid such as the need-based Pell grant. Whether students are required to complete verification to receive state financial aid varies by state. In Tennessee, it depends on the aid program and the institution’s financial aid policy. To receive aid from any program with a need-based eligibility criterion, students must complete verification. For programs without a need-based eligibility criterion, institutions have the authority to set their own policy about whether these students must be verified to receive state financial aid. All that to say, verification is an incredibly consequential process, particularly for low-income students who depend on need-based aid to make college affordable. Anecdotal evidence and descriptive data suggest that verification has a negative impact on students’ college enrollment, but little rigorous research exists to support that claim. This study will begin to fill that gap in the literature by providing empirical evidence of the impact of selection for verification on college enrollment for recent public high school graduates in Tennessee.

Data and Method

Data

Our data primarily come from the Tennessee Higher Education Commission (THEC) and the Tennessee Student Assistance Corporation (TSAC). THEC and TSAC collect and maintain administrative data from several sources, including FAFSA data from Institutional Student Information Records (ISIRs) from FSA, initial postsecondary enrollment from the National Student Clearinghouse (NSC), and exam scores and student questionnaire responses from the ACT. The study’s sample includes public high school students who graduated between 2013 and 2017, filed a FAFSA, and took the ACT exam at least once, which results in a sample of 217,389 students. We chose to restrict the sample in these ways because the majority of our control variables come from the ACT student questionnaire and our treatment of interest is derived from the ISIR. Another reason we decided to limit the sample to students who filed the FAFSA and took the ACT is because we wanted to identify students who are committed to the possibility of going to college. We assume that the students in our sample are truly college-intending because they have completed two crucial steps in the college-going process. As such, our results are only generalizable to the population of students who both file a FAFSA and take a college entrance examination, though this is a common sample restriction both within higher education and even in the Tennessee context (Bruce & Carruthers, 2014; Pallais, 2009).

Outcome and Predictor Variables

The dependent variable of interest in our study is postsecondary enrollment in the fall immediately following high school graduation. The NSC data we used for this project are the files used to calculate Tennessee’s college-going rate, which is defined as the percentage of high school graduates that enroll in college in the summer or fall immediately following high school graduation. Thus, the NSC data we used do not contain any longitudinal enrollment information; we can only observe enrollment in the first semester after a student graduates from high school. This data limitation prevents us from exploring whether verification is associated with delayed enrollment. Because we have access to NSC data, we can track enrollment in more than 95% of postsecondary institutions within the United States (National Student Clearinghouse, 2018) with most of the missing coverage existing in the for-profit sector (Dynarski et al., 2015). To quantify the magnitude of potential lack of coverage in the for-profit sector, we obtained the proportion of first-time fall enrollees in this sector since 2002 using data from the Integrated Postsecondary Education Data System (IPEDS). The results indicate that the share of total first-time enrollees who enroll at a for-profit institution in Tennessee changed from 14% in 2002 to 5.8% in 2018, keeping in mind that many of those first-time students are not recent high school graduates. The implications of these shares for our study are that in the worst-case scenario, wherein we fail to observe all for-profit enrollees, our analytic sample will still consistently account for at least 90% of the population of first-time college students with the exception of 2013, which will be 89.6% of this population. As NSC data can track 95% of enrollees, we are confident that the magnitude of this potential issue will not be meaningful enough to affect our inferences and conclusions.

Our independent variable of interest is selection for verification by the federal government, which is reported on a student’s ISIR. Since award year 2012–2013, institutions have been required to verify all students who are selected for verification by FSA. However, institutions also have the authority to verify additional students or items as needed and, in fact, are required to verify any information on a student’s FAFSA that they believe may be incorrect. Thus, our treatment indicator likely underestimates the number of students who were subjected to verification. We believe that the federal verification flag is the most policy-relevant indicator, as this is the indicator that states and institutions have access to on a student’s ISIR and does not change during the FAFSA completion process. Thus, any intervention by a campus, state system, or independent researcher would likely rely upon this indicator to determine which students may benefit from additional assistance.

We leverage the ACT for most independent variables, including high school grades, ACT exam scores, postsecondary aspirations, and student demographic information such as race, gender, and family income. In our main specification, we avoid using predictor variables from the ISIR because we only have access to what is likely a student’s final financial aid record for the academic year. As Evans et al. (2017) highlight, the FAFSA submission and verification processes are dynamic. Although the verification selection flag does not change even if a student successfully completes the verification process, students, FSA, and institutions may make changes to the information on the FAFSA over time, yet we are only able to observe one ISIR record per student. Because of this, and because the verification process itself may result in changes to a student’s Pell eligibility status or EFC, we chose instead to rely mostly on independent variables from the ACT student questionnaire. We note, however, that our results are robust to including additional variables from the ISIR.

Method

The study’s overarching purpose is to estimate the effect of selection for verification on students’ likelihood of enrolling in college. As stated in the introductory section, existing evidence suggests that lower-income students, who may qualify for need-based aid, are more prone to be selected for verification. This suggests that students who are selected for verification (henceforth, the treated subset) may not only be systematically different from students not selected for verification (henceforth, the control subset) but also may be less likely to enroll in college in the first place, even in the absence of selection for verification. From this view, model estimation should control for these potential systematic differences between control and treated students before measuring the effect of verification on the probability of college enrollment.

To address this estimation challenge, this study relies on entropy balancing (EB) and doubly robust modeling. Although EB derives from the propensity score modeling framework, it differs from this approach by creating a balanced control sample that mirrors not only the characteristics of the treated participants’ first central moment (i.e., mean) but also their second and third moments (i.e., variance and skewness, respectively), as suggested by Hainmueller and Xu (2013).

In EB the counterfactual unit is estimated as follows:

E [\hat{Y (0) | T = 1}] = \frac{\sum_{{i | t = 0}} Y_{i} w_{i}}{\sum_{{i | t = 0}} w_{i}}

(1)

where Y(0) is the outcome of control students had they been subjected to verification. This counterfactual outcome is obtained given a weight $w_{i},$ that is retrieved from each participant’s estimated propensity to be selected for verification $[(e (x) = p r (t = 1 | x_{i})]$ given a set of theoretically and empirically relevant characteristics $(x_{i})$ that may not only affect treatment status but may also have an effect on outcome variation. In accordance with the notion of average treatment effect on the treated (ATET), every control participant had a weight $w_{i}$ with a positive non-zero value indicating that control students had a non-zero chance of exposure to the treated condition (Rosenbaum & Rubin, 1983), whereas treated individuals $w_{i} = 1$ are assumed to be a good representation of the population. In the ATET it is the control participants’ characteristics that will be weighted to mirror their treated counterparts’ characteristics. From this view, if the only difference between treated and control students is their verification status, then treatment is assumed to be independent of individuals’ attributes and any observed difference in outcomes are due to selection for verification.

Different from other propensity score weighting schemes (e.g., inverse probability weighting), $w_{i}$ is built by minimizing the distance of each control from one treated unit, conditioning on up to three moments per x_i such that each $w_{i} | t = 0$ will follow

\sum_{i | t = 0} w_{i} c_{r i} (x_{i}) = m r

(2)

where the right-hand side term will contain the balance constraints (up to the three central moments). This process will ensure that for all control units there will be a non-zero $w_{i}$ subject to moment constraints that will minimize the distance between a given treated participant’s predictor. This minimization or optimization process follows Newton’s optimization method in a similar fashion as the other data-driven optimization methods as implemented in the synthetic control method (e.g., Abadie et al., 2010). See Hainmueller and Xu (2013) for more details regarding the creation of the EB weights and the optimization process.

Model Building Process and Sensitivity Analyses

Our approach aimed to create control participants whose baseline indicators look statistically the same as those of their treatment counterparts. An important concern with this approach is that some unobserved variation in the places where students live or receive their education may both affect not only their chances of being flagged for verification but also the resources they have at their disposal to navigate this extra step in the financial aid process. An important source of concern in this respect is the high school a student attends, as there is variability in the resources available through a student’s high school (e.g., access to a guidance counselor or college access counselor). Accordingly, we estimated the balancing weights within high schools in addition to controlling for high school-by-cohort groups in our main models. This analytic process implies that our models created entropy weights to make control students resemble their flagged peers within each high school within a given academic year. For example, all 2013–2014 high school seniors attending a given high school were weighted to resemble their treatment peers within that same high school $i$ . This process was repeated for the remaining academic years and high schools.¹

College Access Index

The approach we used to measure the impact of verification on enrollment prospects consisted of interacting the treatment status with an index measuring the predicted probability of attending college. Given that this predicted probability (henceforth, college access index) inherently captures students’ observed sociodemographic and socioeconomic conditions, the index is a more efficient approach to test for heterogeneity as opposed to running models disaggregated by race/ethnicity and/or other indicators, such as socioeconomic status or first-generation status.²

Specifically, our index for the predicted probabilities of attending college was calculated using a logit model where the outcome captured whether a given student enrolled in college immediately after high school graduation (1) or did not enroll in college immediately after high school graduation (0). This can be expressed as $logit (Π_{i}) = {x^{'}}_{i},$ with $x_{i}$ representing the vector of covariates used to estimate the entropy weights used in our main models (see Table 1) and $β$ its corresponding vector of regression coefficients. Subsequently, we solved for the probability $Π_{i}$ expressed as

Π_{i} = \frac{\exp {{x^{'}}_{i} β}}{1 + \exp {{x^{'}}_{i} β}} .

(3)

Table 1

Entropy Balancing

	Not flagged before weighting			Not flagged after weighting			Flagged for verification
	(Control)			(Control)			(Treated)
Variable	M	Variance	Skewness	M	Variance	Skewness	M	Variance	Skewness
Female	0.534***	0.249	−0.135	0.555	0.247	−0.223	0.555	0.247	−0.223
Asian	0.020**	0.020	6.803	0.018	0.018	7.175	0.018	0.018	7.176
Black	0.201***	0.160	1.496	0.287	0.205	0.941	0.287	0.205	0.941
Hispanic	0.029***	0.029	5.571	0.041	0.040	4.604	0.041	0.040	4.604
White	0.734***	0.195	−1.059	0.636	0.232	−0.563	0.636	0.232	−0.563
First-generation	0.402***	0.241	0.398	0.520	0.250	−0.081	0.520	0.250	−0.081
Have child	0.007***	0.007	12.050	0.028	0.027	5.734	0.028	0.027	5.734
Citizen	0.955***	0.043	−4.372	0.946	0.051	−3.960	0.946	0.051	−3.960
HSGPA 1st quartile	0.184***	0.150	1.632	0.227	0.175	1.306	0.227	0.175	1.306
HSGPA 2nd quartile	0.213***	0.168	1.404	0.240	0.182	1.220	0.240	0.182	1.220
HSGPA 3rd quartile	0.229***	0.177	1.289	0.217	0.170	1.376	0.217	0.170	1.376
HSGPA 4th quartile	0.107***	0.095	2.546	0.125	0.109	2.269	0.125	0.109	2.269
Highest ACT composite	21.100***	26.980	0.380	19.580	22.700	0.582	19.580	21.230	0.563
Number of ACT exams	2.008***	1.221	1.230	1.827	1.041	1.436	1.827	1.028	1.428
2014 HS graduation cohort	0.188**	0.153	1.597	0.194	0.156	1.548	0.194	0.156	1.548
2015 HS graduation cohort	0.195***	0.157	1.537	0.241	0.183	1.213	0.241	0.183	1.213
2016 HS graduation cohort	0.189***	0.153	1.593	0.218	0.171	1.365	0.218	0.171	1.365
2017 HS graduation cohort	0.245***	0.185	1.185	0.155	0.131	1.909	0.155	0.131	1.909
Family income less than US$24,000	0.148***	0.126	1.982	0.163	0.136	1.829	0.163	0.136	1.829
Family income US$24,000–US$36,000	0.118***	0.104	2.371	0.194	0.156	1.548	0.194	0.156	1.548
Family income US$36,000–US$50,000	0.094***	0.085	2.789	0.172	0.142	1.741	0.172	0.142	1.741
Family income US$50,000–US$60,000	0.076***	0.071	3.187	0.101	0.091	2.646	0.101	0.091	2.646
Family income US$60,000–US$80,000	0.114***	0.101	2.424	0.092	0.084	2.823	0.092	0.084	2.823
Family income US$80,000–US$100,000	0.102***	0.092	2.627	0.047	0.044	4.306	0.047	0.044	4.306
Family income US$100,000–US$120,000	0.075***	0.069	3.239	0.024	0.024	6.199	0.024	0.024	6.199
Family income US$120,000–US$150,000	0.046***	0.044	4.337	0.012	0.012	8.782	0.012	0.012	8.782
Family income more than US$150,000	0.055***	0.052	3.913	0.012	0.012	8.884	0.012	0.012	8.886
Plans to work more than 20 hr/week	0.198***	0.159	1.514	0.236	0.180	1.243	0.236	0.180	1.243
Tuition is the most important factor	0.288***	0.205	0.936	0.324	0.219	0.751	0.324	0.219	0.751
Expects to earn a certificate	0.027***	0.026	5.843	0.032	0.031	5.286	0.032	0.031	5.286
Expects to earn an associates	0.053***	0.050	3.976	0.065	0.061	3.525	0.065	0.061	3.525
Expects to earn a bachelor’s	0.445**	0.247	0.220	0.452	0.248	0.193	0.452	0.248	0.193
Expects to earn a graduate degree	0.362***	0.231	0.575	0.317	0.217	0.786	0.317	0.217	0.786
Plans to apply for financial aid	0.758***	0.184	−1.202	0.792	0.165	−1.435	0.792	0.165	−1.435
No college in mind	0.220***	0.172	1.351	0.198	0.159	1.513	0.198	0.159	1.513
Unsure of college major	0.173***	0.143	1.726	0.138	0.119	2.102	0.138	0.119	2.102
Plans to live at home	0.180*	0.148	1.667	0.185	0.151	1.625	0.185	0.151	1.625
Class rank 1st quartile	0.332***	0.222	0.715	0.259	0.192	1.098	0.259	0.192	1.098
Class rank 2nd quartile	0.330***	0.221	0.726	0.339	0.224	0.679	0.339	0.224	0.679
Class rank 3rd quartile	0.165***	0.138	1.807	0.196	0.158	1.531	0.196	0.158	1.531
Class rank 4th quartile	0.027***	0.026	5.849	0.033	0.032	5.202	0.033	0.032	5.202
Took a college prep curriculum	0.495***	0.250	0.020	0.432	0.245	0.275	0.432	0.245	0.275
Wants help with math in college	0.444***	0.247	0.226	0.479	0.250	0.084	0.479	0.250	0.084
Wants help with reading in college	0.289***	0.206	0.930	0.310	0.214	0.823	0.310	0.214	0.823
Wants help with writing in college	0.278***	0.201	0.990	0.299	0.210	0.876	0.299	0.210	0.876
Wants help with study skills in college	0.468***	0.249	0.130	0.497	0.250	0.012	0.497	0.250	0.012
Wants help with career planning in college	0.413	0.242	0.355	0.412	0.242	0.357	0.412	0.242	0.357
Mean distance to colleges listed on FAFSA	106.8***	29,667.000	4.897	95.170	26,043.000	5.331	95.170	28,353.000	6.093
Automatic zero EFC	0.275***	0.199	1.011	0.207	0.164	1.444	0.207	0.164	1.443
Independent	0.037***	0.036	4.886	0.113	0.100	2.451	0.113	0.100	2.450
Simplified needs test	0.360***	0.230	0.584	0.534	0.249	−0.138	0.535	0.249	−0.138
Number of FAFSA transactions	2.064***	1.497	1.829	3.263	4.980	1.713	3.264	3.380	1.453
Family size	3.913	1.796	0.684	3.929	3.119	1.455	3.928	2.801	0.611
Family concurrently enrolled in college	1.302	0.296	1.800	1.291	0.316	1.887	1.291	0.324	2.502
Parents married	0.605***	0.239	−0.430	0.446	0.247	0.216	0.446	0.247	0.217
Parent(s) never married	0.147***	0.125	2.000	0.176	0.145	1.705	0.176	0.145	1.704
Parents divorced or separated	0.193***	0.156	1.554	0.258	0.192	1.106	0.258	0.192	1.106
Parent widowed	0.018	0.017	7.315	0.021	0.021	6.639	0.021	0.021	6.638

Note. 75,222 in the treated group and 142,167 in the control group. Entropy balancing optimization took 16 iterations and resulted in a max difference of .00959. Balancing occurred within high school-by-cohort groups. Students flagged for verification enroll in college 74.4% of the time, whereas students not flagged for verification enroll 82.6% of the time. T tests conducted between treatment and control groups with statistically significant differences denoted. Family income groups are reported exactly as the question is asked on the ACT questionnaire; consequently, the income groups overlap at the extrema. HS = high school; FAFSA = Free Application for Federal Student Aid; EFC = expected family contribution.

p < .05. **p < .01. ***p < .001.

The left-hand side in Equation 3 is the typical probability scale that ranges from 0 to 1, with values closer to 1 indicating higher probabilities of college enrollment. The resulting $Π_{i}$ values were then separated into five categories, with the highest quintile values as the reference group, which accounts for students with the highest predicted probabilities of college enrollment, sans our treatment of interest (i.e., selection for verification). As shown in Table 2, we interacted the college access index with verification status to assess effect heterogeneity. These models, in addition to showing aggregate estimates, were disaggregated by month of FAFSA filing, to further assess whether when students file the FAFSA is associated with variations in these point estimates.

Table 2

Effect of Verification by FAFSA Filing Months and College Access Index

	Overall	First month	Second month	Third month	After third month	Overall w/ISIR
Model	(1)	(2)	(3)	(4)	(5)	(6)
Panel A
Verification	−0.038*** (0.002)	−0.026*** (0.002)	−0.052*** (0.004)	−0.053*** (0.011)	−0.064*** (0.006)	−0.052*** (0.003)
Control mean	0.779	0.852	0.737	0.738	0.591	0.791
Observations	217,389	117,832	55,672	12,144	31,741	217,389
R²	.215	.180	.214	.332	.229	.230
Oster’s delta	2.26	2.06	2.22	2.42	2.70	4.97
Panel B
Verification × Quintile 1	−0.058*** (0.006)	−0.051*** (0.009)	−0.064*** (0.011)	−0.024 (0.032)	−0.045* (0.021)	−0.131*** (0.008)
Verification × Quintile 2	−0.062*** (0.005)	−0.052*** (0.007)	−0.063*** (0.011)	−0.076** (0.028)	−0.050* (0.022)	−0.075*** (0.008)
Verification × Quintile 3	−0.037*** (0.004)	−0.028*** (0.005)	−0.038*** (0.010)	−0.039 (0.026)	−0.035 (0.023)	−0.027*** (0.007)
Verification × Quintile 4	−0.020*** (0.004)	−0.016*** (0.004)	−0.012 (0.010)	−0.012 (0.025)	−0.042 (0.025)	−0.006 (0.007)
Observations	217,389	117,832	55,672	12,144	31,741	217,389
R²	.215	.180	.214	.332	.229	.230

Note. Dependent variable is college enrollment the fall after high school graduation. Each model includes weights created by the entropy balancing procedure and dummied controls for each quintile of the college access index shown in Equation 3. Each model includes high school-by-cohort fixed effects. Standard errors are clustered at the high school–by-cohort level. Column 6 includes additional variables from the FAFSA, including dependency status, simplified needs test status, the number of FAFSA transactions, family size, family concurrently enrolled in college, and parents’ marital status. Reference category for Panel B is the fifth quintile of the college access index. The test for differences across coefficients in Columns 2 to 4 of Panel A yielded an F statistic of 16.2, which is high enough to reject the null of equality of coefficients. FAFSA = Free Application for Federal Student Aid; ISIR = Institutional Student Information Records.

p < .05. **p < .01. ***p < .001.

Sensitivity Tests Based on Unobservable Selection and Coefficient Stability

Considering that the estimation strategy relies on a selection-on-observables approach, the study’s main results presented in Table 2 include sensitivity tests based on unobservable selection and coefficient stability (Oster, 2017). This sensitivity test assesses how large the effect of unobservables would need to be for the coefficient associated with verification estimated with observed indicators to be zero. Following Altonji et al. (2005) and implemented in Oster (2017), the estimate is obtained as follows:

\frac{c o v (w_{2}, v e r i f i c a t i o n)}{v a r (w_{2})} = δ \times \frac{c o v (w_{1}, v e r i f i c a t i o n)}{v a r (w_{1})},

(4)

where $δ$ captures how large the effect of unobservables $(w_{2})$ needs to be relative to the effect of observables $(w_{1})$ for the coefficient associated with verification to be zero. This implies that this test is particularly appropriate when the treatment effect of interest is statistically significant.

The estimation of $δ$ was obtained using fully specified models, as shown in Table 2. The high school-by-cohort fixed effects were specified in the form

\begin{array}{l} y_{i j} = β_{0} + β_{1} V e r i f i c a t i o n_{1, i j} + \dots + β_{k} X_{k, i j} \\ + γ_{2} D 2_{i} + γ_{3} D 3_{i} + \dots + γ_{n} D n_{i} + u_{i j}, \end{array}

(5)

where the $γ_{2}, \dots, γ_{n}$ coefficients are high school-by-cohort fixed effects coefficients that control for unobserved heterogeneity in a given high school in a given year (Stock & Watson, 2015).

In the test for coefficient stability of the treatment effects (which in this study is $β_{1}$ in Equation 5 and Model 1 in Panel A in Table 2), Oster (2017) argues that unobservable selection may be captured by changes in the coefficient of determination $(R^{2}),$ which measures the proportion of variance explained by the observables. Oster goes on stating that a model’s $R^{2}$ rarely reaches its maximum value of 1 but that one can increase this value to an upper bound of .3. This is conceptually important given that by increasing this value 30%, researchers would be scaling the coefficient of proportionality to a new value referred to as R_max that hypothetically includes these unobserved confounders in the model. Following Oster’s recommendation, the sensitivity tests shown in Table 2 were computed with a R_max value of each model’s observed $R^{2}$ multiplied by 1.3 $(R^{2} * 1.3 = R_{\max}) .$ If coefficients present small to no changes when the explained variance grows, then a high degree of selection on unobservables proportionate to the estimated degree of selection on observables would be necessary as captured in $δ$ in Equation 4. In this view, $δ$ captures what degree of selection on unobservables is required to either make the verification coefficient zero or to explain any of the observed gaps in the estimated outcomes associated with selection for verification. For example, if the estimated delta is 4.5, one can conclude that selection for verification based on unobservable determinants would have to be 4.5 times as informative as selection based on the observed characteristics for the observed verification coefficient to be zero or for the observed gaps to be due to unobservables.

Results

Table 1 summarizes the results of the EB weighting approach implemented within high school and cohort to make students in the control group resemble their peers who were flagged for verification. These results are based on the creation of a balanced set of 142,167 control participants whose first three central moments closely resemble the characteristics of the 75,222 Tennesseans who were flagged for verification. Table 1 also includes the unweighted means of students in the control group with indications of statistical significance to show when treatment and control groups differed in a t test of means.

Overall, note that only four of the 58 predictors of verification included in the models did not reach statistically significant differences between treatment and control students. (These differences in the likelihood of selection for verification are also shown in Supplementary Appendix Table 2 in the online version of the journal.) In comparing the unweighted means between the two groups, students selected for verification are slightly more likely to be female and much more likely to be Black or first generation in college. Moreover, treated students are less academically successful in high school, as students from the lower end of the grade point average distribution are overrepresented, and treated students also have an average ACT composite score that is approximately 1.5 points lower than the average ACT composite score of control students. Although the data are self-reported on the ACT and likely contain some measurement error (Anderson & Holt, 2017), students selected for verification report incomes that are much lower than their counterparts in the control group, which is aligned with the goals of the FAFSA verification process. Notably, students selected for verification have more modest postsecondary aspirations, as a lower proportion of treated students expects to earn a graduate degree and a higher proportion expects to earn a certificate or associate degree. Treated students also seem more certain about the college they plan to attend, as well as the program of study they will pursue. While statistically significant differences between the unweighted treatment and control groups exist across 93% of these variables, no differences remained after the EB procedure. This indicates that entropy weighting was successful in creating a control group that resembled the baseline indicators of their flagged counterparts.

Table 2 shows our main results. Panel A includes results from the model specified in Equation 5. Panels A and B include controls for high school-by-cohort groups to account for the pooled data as well as controls for each quintile of the access index. Panel B also includes interactions between each quintile of the access index and the treatment to measure effect heterogeneity. Column 1 presents the coefficient on verification in the full sample, after accounting for observable differences between the treatment and control groups. This approach suggests that students selected for verification are 3.8 percentage points (4.9%) less likely to enroll in college the fall immediately following high school graduation. Despite our concerns about the endogeneity that arises from including ISIR variables in the model, the results in Column 6 suggest that our estimates are robust to their inclusion, as the point estimate increases from 3.8 percentage points to 5.2 percentage points.

These effects vary by the FAFSA filing month, as shown in Columns 2 to 5 with students filing later in the cycle more negatively affected by verification.³ Students who filed the FAFSA in the first month after it became available were 2.6 percentage points (3.1%) less likely to attend college if selected for verification. By comparison, students who filed the FAFSA more than 3 months after it was made available were 6.4 percentage points (10.8%) less likely to attend college, if selected. With each passing month in the FAFSA application cycle, the verification coefficient becomes increasingly negative while the control mean enrollment rate decreases monotonically, suggesting that students who file the FAFSA later are more negatively impacted by verification. We believe this may be attributable to the reduced time to provide the additional documentation required for verification, or it may be that students who file later are less certain about their postsecondary plans and, thus, are more negatively impacted by another impediment in the college access pipeline.

At the bottom of Panel A, we present results of sensitivity tests we conducted following Oster’s (2017) approach of evaluating robustness to omitted variable bias by accounting for both the coefficient and R² movements. Our estimates of Oster’s delta consistently indicated that across all model specifications, the unobservable characteristics would have to be at least twice as informative or important as observable characteristics to reduce the treatment effect to zero. Oster (2017) recommends that researchers should set the R_max value to 1.3 times R² (as we did) and demonstrate that delta is greater than 1 to claim robustness to omitted variable bias. Because our delta is greater than 2 in every specification (and greater than 4 in our specification that includes ISIR variables), we believe it would take a substantial amount of omitted variable bias to invalidate our finding that selection for verification has a significantly negative effect on college enrollment.

The results presented in Panel B show how the effects of verification vary across quintiles of the access index. Examining results for the whole sample in Column 1, it is clear that the students who are least likely to enroll in college (i.e., those in the first quintile of the access index) are the very students who are most affected by verification. Students from the lowest two quintiles who were flagged for verification were approximately 6 percentage points less likely to enroll in college than students from the highest quintile (i.e., the reference group). F tests comparing these groups confirm that a statistically significant difference exists. The same pattern exists for students who file the FAFSA in the first 2 months after it becomes available. Interestingly, the differences between the access index quintiles become less pronounced for students who file the FAFSA in the third month or later (Columns 4 and 5). It may be the case that student characteristics commonly associated with college access, which are captured in our access index, matter less for students who file later. In other words, students who file later are similarly affected by selection for verification regardless of their demographics, academic achievement, or high school environment. The estimates for students who file later are also less precise than the estimates for students who file earlier, because there are fewer students in our sample who file so late in the FAFSA cycle. While the models in Columns 4 and 5 still have a reasonably large sample size, the number of observations in either column is considerably smaller than in Column 2 or 3.

So far, our most comprehensive model specifications indicate that the coefficient on verification translates to a 3.8 to 5.2 percentage point reduction in the probability of college enrollment (with higher estimates obtained from the second month and on, as shown in Models 3–5 in Table 2). These estimates, however, examine whether someone enrolled in college not where the student enrolled. Panel A in Table 3 sheds light on the sectors in which students are most affected by verification. Columns 1 and 2 show that the overall effect of 3.8 percentage points is predominantly driven by a 2.9 percentage point (8.1%) decrease in the probability of enrollment at 2-year institutions. Although smaller in magnitude, enrollment in the 4-year sector also decreased by 1 percentage point (2.4%). Looking at the public and private margins of attendance, we see that the effect is largely driven by a 3.5 percentage point (5.2%) decrease in enrollment at public institutions. The effect on enrollment at private institutions is statistically insignificant. Similarly, there is no discernable effect of selection for verification on the probability of enrolling in college out-of-state.

Table 3

Effects of Verification by Sector and FAFSA Filing Month

	Enroll 2yr	Enroll 4yr	Enroll private	Enroll public	Enroll out-of-state
Model	(1)	(2)	(3)	(4)	(5)
Panel A
Verification	−0.029*** (0.002)	−0.010*** (0.002)	−0.002 (0.001)	−0.035*** (0.002)	−0.002 (0.001)
Control mean	0.358	0.421	0.108	0.671	0.071
R²	.122	.317	.062	.114	.063
Panel B
First month	−0.018*** (0.003)	−0.008** (0.003)	−0.003 (0.002)	−0.023*** (0.003)	0.001 (0.002)
Second month	−0.040*** (0.005)	−0.012** (0.004)	−0.004 (0.003)	−0.048*** (0.005)	−0.005* (0.002)
Third month	−0.037*** (0.011)	−0.016 (0.010)	0.010 (0.007)	−0.063*** (0.011)	0.005 (0.006)
After third month	−0.048*** (0.006)	−0.016** (0.005)	−0.004 (0.003)	−0.060*** (0.006)	−0.001 (0.003)
F test	11.31	0.99	1.22	14.14	1.17

Note. Dependent variable is college enrollment the fall after high school graduation by sector specified. Each model includes weights created by the entropy balancing procedure, dummied controls for each quintile of the college access index shown in Equation 3, and high school-by-cohort fixed effects. Standard errors are clustered at the high school-by-cohort level. Observation count for Panel A is 217,389. Observation counts for Panel B are 117,832 in Month 1, 55,672 in Month 2, 12,144 in Month 3, and 31,741 in After third month. F statistics for F tests of equality of coefficients across months are presented in Panel B.

p < .05. **p < .01. ***p < .001.

To determine whether the effect of verification across FAFSA filing months varies by sector, we ran models disaggregated by month for each of the five outcomes. Column 1 in Panel B shows that the effects of verification differ across months in a similar pattern presented in Panel A of Table 2. Students who file during the first month are less affected by verification than those who file later. The differences across these months are statistically significant with an F statistic of 11.31. Interestingly, there were not meaningful differences across the FAFSA filing months for enrollment in the 4-year sector. While some estimates in Column 2 reached conventional levels of statistical significance, the confidence intervals around the point estimates overlapped considerably, rendering them statistically indistinguishable from each other (as indicated by the F statistic of 0.99). In Column 4 (enrollment at public institutions), we see that recurring pattern yet again; students who file later in the FAFSA cycle are more negatively affected by selection for verification.

Limitations

Our use of population-level administrative data is a notable strength of this study. However, one limitation is that these results may not be generalizable to other states due to Tennessee’s unique policy context. All of Tennessee’s large financial aid programs, including the HOPE Scholarship, Tennessee Student Assistance Award, Tennessee Promise, and Tennessee Reconnect, require students to complete the FAFSA as part of the application process. This eligibility requirement for state financial aid makes FAFSA completion a critical part of the college-going process in Tennessee. Furthermore, Tennessee has invested substantial state resources in support of FAFSA filing. As a result, the FAFSA filing rate for high school seniors in Tennessee increased to a record high of 81.7% in the 2018–2019 FAFSA cycle. In states with a lower FAFSA filing rate, we might assume that the average FAFSA filer is more likely to go to college than the average FAFSA filer in Tennessee due to self-selection bias. We acknowledge that our results only represent one state, and despite our efforts to produce a causal estimate with minimal bias, we recommend caution in extrapolating our findings to other contexts.

The counterfactual framework employed in this study made the observable characteristics included in the models balanced among students flagged for verification and their nonflagged counterparts. Moreover, the delta estimates obtained using Oster’s (2017) approach consistently suggested that unobservable characteristics would have to be at least twice as influential as the observable characteristics included in our models to reduce our verification coefficient to zero. We employed both of these strategies to ensure that we accounted for as many differences as possible between the two groups. While we include an extraordinary number of observable characteristics in our modeling strategy, we recognize that unobserved confounders may still exist. Despite these limitations, we believe that our research is an important first step in quantifying the effects of verification and hope that future studies will be able to build upon this work by using identification strategies that better address the potential influence of unobservables.

Discussion and Implications

Our analyses quantify the impact that selection for verification has on college enrollment for recent high school graduates in Tennessee. The aggregate estimates that exclude ISIR variables indicate that approximately 2,800 students across the five cohorts in our sample did not enroll in college because of selection for verification. FSA (2019) reports that about 11% of students flagged for verification experience “verification melt,” which makes them ineligible for federal financial aid (this is a more conservative estimate compared with others discussed previously, which reach as high as 28%). As Tennessee requires students to complete the FAFSA to be eligible for state financial aid, the financial consequences of verification melt in our sample will be much more negative than in other states with fewer or less generous financial aid programs, even if verification does not derail students’ college enrollment plans. Not only would students unable to complete the FAFSA lose federal and state need-based aid but they may also need to rely on private lenders and/or work more hours, thus reducing their chances of graduating on time or even continuing with their college education. Going back to our aggregate estimates, the reduction of about 3,000 enrollees only depicts one problem. Other financial burdens associated with verification for students who continued their enrollment plans without completing the FAFSA remain masked, and further research is needed.

Another important finding is that some subpopulations are more negatively affected by selection for verification than others. In fact, students who are the least likely to attend college irrespective of verification are the very students who are most harmed. Moreover, this study lends credence to the notion that the timing of FAFSA filing is a strong signal of college matriculation. Unfortunately, our results suggest that the verification process is a much more significant hurdle for those students who file later, arguably due to the reduced time to meet the paperwork requirements. The difference in the magnitude of the effect of selection for verification on college enrollment suggests that institutions and financial aid staff may want to strategically target resources toward late FAFSA filers to have the largest impact on college enrollment outcomes.

Our results align with existing research suggesting that students from minoritized backgrounds and first-generation students are more affected by financial aid policies than their nonminoritized and continuing-generation counterparts. The additional burden verification places on students from less-privileged backgrounds, who typically have fewer resources to help them navigate the verification process and are less likely to enroll in college to begin with, will continue to fuel inequities in college access and affordability. While access to college is only the first step toward attaining a credential, mitigating barriers by changing policies or targeting support toward those who need it most may enhance the enrollment prospects of thousands of students in Tennessee and potentially hundreds of thousands of students across the country. Considering the limited research on the effect of verification and the single-state nature of our estimates, we recommend that future research extend our analyses to additional states and student populations (e.g., nonrecent high school graduates) to assess the external validity of our estimates.

We hope that our findings can help education and nonprofit entities identify which students would most benefit from targeted interventions or additional assistance during the verification process. One example of how this research may inform practice is that colleges could devote more resources to student subgroups that are particularly vulnerable to the effects of verification (e.g., offering support through the financial aid office or a philanthropic third party). More to the point, as early FAFSA filing is associated with less of a verification penalty, students who are more likely to be negatively affected by being flagged should be encouraged to file as early as possible, even if they might be uncertain about enrolling in college. In fact, as more states adopt mandatory FAFSA policies, it may behoove policymakers to consider when students are required to file the FAFSA, rather than just if they do.

We acknowledge that verification serves an important role in protecting taxpayers’ investment in higher education, but we believe our findings provide convincing evidence that selection for verification is associated with worse enrollment outcomes for students and is more harmful for some students than others. Of course, more research is necessary to determine whether these outcomes persist beyond initial enrollment, but these facts coupled with the substantial costs institutions and FSA incur to coordinate the verification process makes us and others (Smith, 2018; Warick, 2018a) dubious of its benefits, as currently constituted. At the very least, we hope that our research can provide insight into conversations about the value of verification and how the process can continue to be redesigned to protect against fraud while minimizing harm to selected students.

Supplemental Material

sj-pdf-1-epa-10.3102_0162373721989304 – Supplemental material for The Hidden Costs of Corroboration: Estimating the Effects of Financial Aid Verification on College Enrollment

Supplemental material, sj-pdf-1-epa-10.3102_0162373721989304 for The Hidden Costs of Corroboration: Estimating the Effects of Financial Aid Verification on College Enrollment by Jason C. Lee, Madison Dell, Manuel S. González Canché, Alex Monday and Amanda Klafehn in Educational Evaluation and Policy Analysis

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Manuel S. González Canché

Notes

Authors

JASON C. LEE is a senior policy analyst at the State Higher Education Executive Officers Association. His research focuses on higher education finance and policy with an emphasis on evaluating which postsecondary policies and programs hinder postsecondary access and success.

MADISON DELL is a PhD student studying education at Stanford University. Her research interests lie at the intersection of higher education, economics, and public policy, with a focus on interventions that support students’ access to and success in higher education.

MANUEL S. GONZÁLEZ CANCHÉ is an associate professor at the University of Pennsylvania. He is interested in offering less-biased estimates of individuals’ decisions and/or policy changes on education and labor market outcomes, particularly concerning low-income and marginalized groups.

ALEX MONDAY is a PhD student in higher education at the University of Georgia’s Institute of Higher Education. His research interests include college access and success, student decision-making, and how policy impacts equity in higher education.

AMANDA KLAFEHN is the senior director of research and strategy at the Tennessee Higher Education Commission. Her research interests include college access and financial aid policy, specifically the intersection of K–12 and higher education policy supporting students’ awareness, preparation, and decision-making for postsecondary plans.

References

Abadie

Diamond

Hainmueller

(2010). Synthetic control methods for comparative case studies: Estimating the effect of California’s tobacco control program (NBER Working Paper No. 12831). National Bureau of Economic Research.

Altonji

J. G.

Elder

T. E.

Taber

C. R.

(2005). Selection on observed and unobserved variables: Assessing the effectiveness of Catholic schools. Journal of Political Economy, 113(1), 151–184.

Anderson

D. M.

Holt

J. K.

(2017). Do high school students know their parent’s income? [Working paper]. https://www.siue.edu/ierc/pdf/Income%20Accuracy%20Brief%202017-4.pdf

Asher

(2009, June). FAFSA simplification plan will make it easier for students to get college aid. The Institute for College Access and Success. https://ticas.org/sites/default/files/legacy/ticas_files/File/ticas_d/FAFSA_STA_6-24.pdf

Association of Governing Boards. (2017). Return on investment in college education. https://agb.org/reports-2/return-on-investment-in-college-education/

Bettinger

E. P.

Long

B. T.

Oreopoulos

Sanbonmatsu

(2012). The role of application assistance and information in college decisions: Results from the H&R Block FAFSA experiment. The Quarterly Journal of Economics, 127(3), 1205–1242.

Bruce

D. J.

Carruthers

C. K.

(2014). Jackpot? The impact of lotter scholarship on enrollment in Tennessee. Journal of Urban Economics, 81, 30–44.

Cabrera

A. F.

La Nasa

S. M.

(2000). Overcoming the tasks on the path to college for America’s disadvantaged. New Directions for Institutional Research, 27(3), 31–43.

Cochrane

D. F.

LaManque

Szabo-Kubitz

(2010). After the FAFSA: How red tape can prevent eligible students from receiving financial aid. The Institute for College Access and Success. https://ticas.org/affordability-2/after-fafsa/

10.

Davidson

J. C.

(2015). Improving the financial aid process for community college students: A literature review of FAFSA simplification, information, and verification. Community College Journal of Research and Practice, 39(5), 397–408. https://doi.org/10.1080/10668926.2013.820157

11.

DeBaun

(2018, November 12). Verification melt rate ticks up to 25 percent. National College Attainment Network. https://www.ncan.org/news/news.asp?id=456084&hhSearchTerms=%22fafsa+and+verification%22

12.

Douglas-Gabriel

(2017, November 28). Colleges puzzled by surge in FAFSA verification requests. The Washington Post. https://www.naicu.edu/news-events/headline-news/2017/11/colleges-puzzled-by-surge-in-fafsa-verification-re

13.

Dynarski

S. M.

Hemelt

S. W.

Hyman

J. M.

(2015). The missing manual: Using National Student Clearinghouse data to track postsecondary outcomes. Educational Evaluation and Policy Analysis, 37(1), 53S–79S. https://doi.org/10.3102/0162373715576078

14.

Dynarski

S. M.

Scott-Clayton

J. E.

(2006). The cost of complexity in federal student aid: Lessons from optimal tax theory and behavioral economics. National Tax Journal, 59(2), 319–356.

15.

Dynarski

S. M.

Scott-Clayton

J. E.

(2013). Financial aid policy: Lessons from research. The Future of Children, 23(1), 67–91.

16.

Evans

B. J.

Nguyen

T. D.

Tener

B. B.

Thomas

C. L.

(2017). Federal Pell grant eligibility and receipt: Explaining nonreceipt and changes to EFC using national and institutional data. Journal of Student Financial Aid, 47(3), 45–69.

17.

Federal Student Aid. (2013, February). Application and verification guide 2013-2014. https://ifap.ed.gov/sites/default/files/attachments/fsahandbook/1314AVG.pdf

18.

Federal Student Aid. (2014, October). Application and verification guide 2014-2015. https://ifap.ed.gov/sites/default/files/attachments/fsahandbook/1415AVG.pdf

19.

Federal Student Aid. (2015, September). Application and verification guide 2015-2016. https://ifap.ed.gov/sites/default/files/attachments/fsahandbook/1516AVG.pdf

20.

Federal Student Aid. (2016, November). Application and verification guide 2016-2017. https://ifap.ed.gov/sites/default/files/attachments/fsahandbook/1617AVG.pdf

21.

Federal Student Aid. (2017, May). Application and verification guide 2017-2018. https://ifap.ed.gov/sites/default/files/attachments/fsahandbook/1718AVG.pdf

22.

Federal Student Aid. (2019, December). GS4 Verification 2019–20 and 2020–21. https://youtu.be/HyKvqc11UYY

23.

Hainmueller

(2013). Ebalance: A Stata package for entropy balancing. Journal of Statistical Software, 54(7), 1–18.

24.

Horn

L. J.

Chen

Chapman

(2003). Getting ready to pay for college: What students and their parents know about the cost of college tuition and what they are doing to find out (NCES-2003-30). U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics.

25.

The Institute for College Access and Success. (2016, November). On the sidelines of simplification: Stories of navigating the FAFSA verification process. https://ticas.org/affordability-2/sidelines-simplification/

26.

Keller

(2017, September 25). FAFSA verification by the numbers. National College Attainment Network https://www.ncan.org/news/news.asp?id=&hhSearchTerms=%fafsa+and+verification%4567852222

27.

Martorell

Friedmann

(2018, April). Money left on the table: An analysis of Pell grant receipt among financially-eligible community college students in California. Wheelhouse: The Center for Community College Leadership and Research, University of California, Davis School of Education.

28.

McKinney

Novak

(2015). FAFSA filing among first-year college students: Who files on time, who doesn’t, and why does it matter? Research in Higher Education, 56, 1–28.

29.

McMahon

W. W.

(2009). Higher learning, greater good: The private and social benefits of higher education. Johns Hopkins University Press.

30.

National Association of Student Financial Aid Administrators. (2018, October). NASFAA issue brief: Verification. http://www.nasfaa.org/uploads/documents/NASFAA_Issue_Brief_Verification.pdf

31.

National Association of Student Financial Aid Administrators. (n.d.). NASFAA data show verification does not impact most financial aid awards. https://www.nasfaa.org/uploads/documents/verification_one_pager.pdf

32.

National Student Clearinghouse. (2018). Enrollment coverage workbook. https://nscresearchcenter.org/workingwithourdata/

33.

Oreopoulos

Salvanes

K. G.

(2011). Priceless: The nonpecuniary benefits of schooling. The Journal of Economic Perspectives, 25(1), 159–184. https://doi.org/10.1257/jep.25.1.159

34.

Oster

(2017). Unobservable selection and coefficient stability: Theory and evidence. Journal of Business & Economic Statistics, 37(2), 187–204. https://doi.org/10.1080/07350015.2016.1227711

35.

Page

L. C.

Castleman

B. L.

Meyer

(2020). Customized nudging to improve FAFSA completion and income verification. Educational Evaluation and Policy Analysis, 42(1), 3–21. https://doi.org/10.3102/0162373719876916

36.

Page

L. C.

Lowry

D. J.

Nurshatayeva

(2017). An examination of the relationship between school district FAFSA completion rates and district poverty levels. National College Access Network. https://www.luminafoundation.org/files/resources/relationship-between-fafsa-completion-and-poverty.pdf

37.

Page

L. C.

Scott-Clayton

(2016). Improving college access in the United States: Barriers and policy responses. Economics of Education Review, 51, 4–22.

38.

Pallais

(2009). Taking a chance on college: Is the Tennessee Education Lottery Scholarship program a winner? Journal of Human Resources, 44, 199–222.

39.

Rosenbaum

P. R.

Rubin

D. B.

(1983). The central role of the propensity score in observational studies for causal effects. Biometrika, 70(1), 41–55.

40.

Scott-Clayton

(2012). Information constraints and financial aid policy (NBER Working Paper No. 17811). National Bureau of Economic Research.

41.

Serna

G. R.

(2016). The federal role in financing 21st century higher education: Effectiveness, issues, and alternatives. University of Virginia Miller Center for Public Affairs.

42.

Smith

A. A.

(2018, October 12). Income verification for federal aid hinders low-income students. https://www.insidehighered.com/news/2018/10/12/college-officials-concerned-about-high-number-low-income-students-selected-federal

43.

Stock

J. H.

Watson

M. W.

(2015). Introduction to econometrics (3rd ed.). Pearson.

44.

U.S. Department of Education. (2017). FY 2017 agency financial report. https://www2.ed.gov/about/reports/annual/2017report/4c-otherinfo-payment-integrity.pdf

45.

U.S. Department of Education. (2018). FY 2018 agency financial report. https://www2.ed.gov/about/reports/annual/2018report/4c-otherinfo-payment-integrity.pdf

46.

U.S. Department of Education. (2019). FY 2019 agency financial report. https://www2.ed.gov/about/reports/annual/2019report/4c-otherinfo-payment-integrity.pdf

47.

Verification. (n.d.). https://finaid.org/fafsa/verification/

48.

Warick

(2018a, November). FAFSA verification: Good government or red tape? National College Access Network. https://cdn.ymaws.com/www.ncan.org/resource/resmgr/publications/verificationwp2018.pdf

49.

Warick

(2018b, November 27). How FAFSA verification harms low-income students and what ED can do to help. National College Attainment Network. https://www.ncan.org/news/news.asp?id=&hhSearchTerms=%fafsa+and+verification%4560782222

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.08 MB

0.19 MB