Customized Nudging to Improve FAFSA Completion and Income Verification

Introduction

the efficacy of public policies is often hindered by complicated application processes that make it difficult for people who are eligible for public benefits to access them. Hassles associated with applying for public benefits can impede impoverished families from accessing supplementary nutrition assistance for their children or from sending their children to high-quality schools within their school districts (Bertrand, Mullainathan, & Shafir, 2004; Hastings & Weinstein, 2008; Weixler, Valant, Bassok, Doromal, & Gerry, 2019). A complex array of choices and cumbersome enrollment processes can prevent working adults from maximizing their employer retirement contributions or health care benefits (Kling, Mullainathan, Shafir, Vermeulen, & Wrobel, 2012; Madrian & Shea, 2001), whereas low visibility can preclude individuals from pursuing preventive health measures or from saving financially (Karlan, McConnell, Mullainathan, & Zinman, 2010; Milkman, Beshears, Choi, Laibson, & Madrian, 2012; Stockwell et al., 2012).

Numerous studies demonstrate that the application of behavioral economics principles to make it easier for people to access beneficial programs and opportunities can lead to improved outcomes for individuals. These strategies range from changing defaults so that employees are automatically enrolled in an employer-match retirement program unless they actively opt out of participating, to using prompts and reminders to increase the share of adults who get flu vaccines or contribute to their savings accounts, to using positive social norms to reduce residential home energy use (Allcott, 2011; Karlan et al., 2010; Stockwell et al., 2012). Behavioral science strategies to increase program participation have become increasingly integrated into public policy at various levels, including the federal government (Executive Office of the President, 2016).

In the context of postsecondary education, the federal and state governments allocate billions of dollars each year in the form of need-based financial aid to support low-income students to access and succeed in college. Rigorous research demonstrates that need-based aid can substantially increase the share of low-income students who enter and graduate from college (Castleman & Long, 2016; Dynarski, 2003; Kane, 2007; Page, Kehoe, Castleman, & Sahadewo, 2019; Page & Scott-Clayton, 2016; Scott-Clayton, 2011; Scott-Clayton & Zafar, 2016), yet each year hundreds of thousands of students who would qualify for financial aid do not complete the Free Application for Federal Student Aid (FAFSA). Recent estimates suggest that college students leave upward of US$2 billion on the table in grant assistance (King, 2004; Kofoed, 2016).

Researchers have long recognized that the complexity of the FAFSA can serve as a barrier to students applying for—and in turn receiving—financial aid (Bill & Melinda Gates Foundation, 2015; Dynarski & Scott-Clayton, 2006; Dynarski, Scott-Clayton & Wiederspan, 2013). Experimental research indicates that providing families with individual assistance completing the FAFSA can lead to substantial increases in rates of FAFSA submission as well as college attendance and persistence (Bettinger, Long, Oreopoulos, & Sanbonmatsu, 2012). Nevertheless, many students and families lack access to this type of professional assistance with the FAFSA. For instance, upward of half of high school seniors in the largest U.S. school districts do not complete the FAFSA prior to graduation (Bird et al., 2019), and nationwide, completion rates are lower in districts serving higher poverty populations (Page, Lowry, & Nurshatayeva, 2017).

Of those who do submit a FAFSA, a substantial portion may not actually receive financial aid as result of often-overlooked complexities that arise between when students submit the FAFSA and when they receive their financial aid offers from colleges. For instance, some students fail to fully complete the FAFSA because they miss simple steps like providing an electronic signature. Furthermore, even among those who complete the application, many are required by the U.S. Department of Education or the institutions to which they apply to verify the income and asset information they report on their application. The burden of income verification is disproportionately experienced by low-income applicants for financial aid. For example, Wiederspan (2019) estimates that among financial aid applicants within the state of Iowa, 57% of those eligible for Pell grant funds were selected for verification, compared with only 7% of non Pell-eligible students.

Despite the centralized nature of being selected for verification, the process for completing verification is completely decentralized. Students who are selected need to complete separate verification processes with each institution to which they have applied, and verification procedures can be different across institutions (Castleman & Page, 2014a). This additional step in the financial aid process may delay or even prevent FAFSA filers from receiving award packages from the colleges and universities to which they are accepted. According to recent popular reporting (e.g., Hoover, 2017), verification processing can take upward of 6 weeks, and processing times may be longer within institutions enrolling larger numbers of low-income students.

If students are hindered in completing all steps in the financial aid application process, they may miss out on aid entirely; receive less than they are eligible for because they file after priority deadlines; or face substantial delays in receiving aid, such that funds that they are to receive for educational costs such as books and other living expenses may not be disbursed until after the start of an academic term (Hoover, 2017). Indeed, failure to complete the FAFSA in a timely way during a student’s high school senior year is a contributor to “summer melt,” the phenomenon that college-intending students fail to transition successfully to postsecondary education in the fall after high school graduation (Castleman & Page, 2014a, 2014b).

Given the broad recognition of FAFSA as a challenging but critical step in the college-going process, numerous efforts at the local, state, and national levels aim to improve FAFSA filing among college-intending students. Prior experimental work demonstrates that a combination of information, reminders, and remote assistance for students about financial aid and other transitional tasks can improve college entry and persistence, particularly among economically disadvantaged students (Castleman & Page, 2015, 2016, 2017; Castleman, Page & Schooley, 2014; Page & Gehlbach, 2017). At a national level, the federal government has piloted efforts to provide student-level FAFSA completion information to educational agencies throughout the country. However, providing such information to school districts may be insufficient to increase FAFSA completion rates meaningfully. Indeed, anecdotal evidence based on conversations with personnel from districts connected to this FAFSA pilot project indicates that utilization rates of these data can be low. Numerous research studies demonstrate empirically that information alone may not be sufficient to encourage students to commit the time and effort necessary to complete important processes required to apply to college or for financial aid (Bergman, Denning, & Manoli, 2017; Bettinger et al., 2012).

A potentially important distinction, however, is what kind of information is likely to be most salient to individuals and might affect their decisions about whether to invest in consequential but complex actions related to pursuing postsecondary education. For instance, information about the benefits of pursuing higher education or the availability of financial aid may not resonate with individuals if they already have some basic understanding of these benefits. Similarly, information that is generic and not tailored to an individual’s background and circumstances may seem less salient. These hypotheses are supported by recent FAFSA completion nudge campaigns at the state and national levels that had precisely estimated zero impacts on financial aid receipt and college enrollment. The lack of direct relationship between students and the organizations sending the outreach as well as the generic nature of the messaging at scale likely contributed to the null impacts (Bird et al., 2019). By contrast, information that provides individuals with highly personalized, actionable information about concrete steps they can take to advance through complicated processes may support people to persevere through informational and behavioral bottlenecks (Bhargava & Manoli, 2015; Kling et al., 2012). For example, to the extent that students are unaware that there may be additional steps required on their part after FAFSA submission, up-to-date information about their application status may be helpful in increasing FAFSA completion. An important question, therefore, is whether interventions that provide more personalized, data-driven updates for students about the status of their FAFSA and that provide access to one-on-one assistance could generate substantial increases in FAFSA completion.

We address this question by designing a text message intervention that leveraged regularly updated administrative data to provide students with personalized, data-informed updates on their FAFSA submission and completion status; encouragement to make use of school and community supports available for FAFSA filing; and the ability to write back for one-on-one, text-based assistance with FAFSA. Our reliance on text messaging as the primary mode of outreach builds on prior work using text messaging to communicate about college-going tasks (e.g., Castleman & Page, 2015, 2016) as well as literature illustrating text messaging as a preferred mode of communication that introduces fewer barriers to use (e.g., those targeted for messaging do not need to have a phone with a data plan; Deil-Amen & Rios-Aguilar, 2014).

We hypothesize that by providing students with timely, personalized reminders about the importance of the FAFSA and their individual status in the filing process, this initiative will improve students’ successful completion of the FAFSA overall and within institutional and state priority deadlines. Furthermore, we hypothesize that early, successful completion of these processes will lead to higher and more stable levels of financial aid which, in turn, will improve the rates with which students matriculate to college.

To test these hypotheses, we implemented the text-based FAFSA campaign in the context of a cluster-randomized trial and in collaboration with eight Texas school districts that together served more than 17,000 class of 2015 high school seniors across 66 high schools. Within these districts, we experimentally selected high schools to participate in the text-based intervention, which provided financial aid focused outreach to students during the second half of their high school senior year. To preview our results, by the end of the intervention period in Texas, FAFSA submission and completion rates were a statistically significant 6 percentage points higher in the treatment schools. These treatment effects attenuated somewhat after the active intervention period to a nonsignificant 3 to 4 percentage points by late summer. This attenuation suggests that the intervention influenced a combination of overall FAFSA filing as well as FAFSA timing, with some students who would have filed regardless doing so earlier because of the outreach. Furthermore, the outreach improved immediate college matriculation by approximately 3 percentage points, a combination of a larger improvement in 4-year college enrollment and a modest decline in 2-year college enrollment.

We consider three potential mechanisms beyond simply completing the FAFSA through which the intervention could have improved rates of college enrollment. First, the messaging was inclusive of a separate application for undocumented students to access state-based financial aid (the Texas Application for State Financial Aid [TASFA]). Therefore, it may have increased TASFA completion and access to financial aid specifically for these students. Second, filing the FAFSA earlier may have helped students to access more generous aid. Third, the messaging may have made more salient the income verification process for the nearly 40% of FAFSA filers in our data selected for verification, and earlier filing may have afforded these students more time to navigate the verification process, as encouraged by the outreach. Although we are unable to observe outcomes related to TASFA filing, we provide evidence to support the second and third possible channels that we note.

We structure the remainder of the article as follows. In section “Research Sites and Intervention Description,” we describe the Texas sites and intervention, followed in section “Experimental Research Design, Data, and Analysis,” by a discussion of the Texas data and experimental research design. In section “Experimental Results,” we present experimental results. We conclude with a general discussion in section “Discussion.”

Research Sites and Intervention Description

During the 2014–2015 academic year, we partnered with eight public school districts in the Austin and Houston areas of Texas to implement a text messaging intervention aimed at improving FAFSA submission and completion rates among high school seniors. Across the participating districts, our sample includes 66 high schools serving more than 17,000 class of 2015 high school seniors. We present descriptive statistics for participating students and schools in Table 1. These districts collectively serve a majority–minority student population that is 26% White, 19% Black, and more than 50% Hispanic. Forty-one percent of sample students are individually flagged as economically disadvantaged, although the rate of economic disadvantage ranged substantially across participating schools from quite low to nearly 100%. ¹ Among participating schools, the average rate of college-going was 46% among class of 2013 graduates. This rate is well below the college-going rate of 66% among class of 2013 high school graduates across the United States ² and may be reflective of the large share of lower income students served by participating districts. This may also be reflective of the lower-than-average SAT (Scholastic Aptitude Test) performance among test takers in participating districts. The combined math and verbal SAT score of 881 places the typical student in these districts near the 30th percentile of performance. Finally, among class of 2014 students in the participating schools, an average of 48% of students completed a FAFSA by the end of 2014.

OPEN IN VIEWER

Table 1

Descriptive Statistics and Assessment of Balance Between Treatment and Control Groups

Variable	M	Treatment control differential
Student-level
White	0.257	0.003 (0.035)	0.003 (0.035)
Black	0.186	−0.005 (0.033)	−0.005 (0.033)
Hispanic	0.577	0.032 (0.056)	0.033 (0.056)
Other race/ethnicity	0.075	0.005 (0.016)	0.005 (0.015)
Female	0.503	−0.002 (0.009)	−0.001 (0.010)
Economically disadvantaged	0.407	−0.026 (0.048)	−0.055 (0.053)
GPA	2.80 (0. 814)	−0.012 (0.071)	−0.004 (0.125)
SAT (math + verbal)	881.26 (227. 261)	0.374 (29.884)	−11.472 (26.693)
School-level
College enrollment rate (2013)	0.456 (0.129)	0.01 (0.017)	0.01 (0.017)
Two-year college enrollment rate (2013)	0.178 (0.062)	−0.003 (0.013)	−0.003 (0.013)
Four-year college enrollment rate (2013)	0.279 (0.121)	0.013 (0.020)	0.013 (0.020)
FAFSA filing rate (2014)	0.484 (0.143)	0.018 (0.024)	0.018 (0.024)
Missing values imputed to zero			✓

Source. District administrative records.

Note. Treatment-control differentials derived from regression models where we regress each student-level covariate on an indicator for treatment assignment at the school level and a fixed effects for groups within which we randomized schools. Robust standard errors (in parentheses) clustered at the school level. We present results for observed data and for data after missing values are set to zero. N = 17,731 students. GPA = grade point average; SAT = Scholastic Aptitude Test; FAFSA = Free Application for Federal Student Aid.

†

p < .10. *p < .05. **p < .01. ***p < .001.

The intervention consisted of weekly, personalized text messages related to applying for college financial aid. To implement the project, we built on a contract that some of the districts already had with a data management and communications platform (OneLogos Education Solutions). ³ OneLogos has the capability to push out personalized text messages to students, to receive student responses for particular counselors, and to facilitate one-to-one, text-based communication between students and their counselors. A unique feature of this project is that, via the Texas Higher Education Coordinating Board (THECB) and the Apply Texas Counselor Suite Portal, Texas school districts have access to regularly updated student-level data on FAFSA filing and income verification status. ⁴ OneLogos was able to automate data pulls from this system to provide students with text message updates on the status of their FAFSA application. ⁵ Project implementation additionally was facilitated by the fact that, via access to students’ Apply Texas college application information, the districts already had access to student cell phone information for high school seniors who had established Apply Texas accounts as well as consent to message these students via text regarding the college-going process.

Beginning in January 2015 and continuing through late April 2015, high school seniors with Apply Texas accounts containing valid cell phone numbers received automated, customized messages approximately weekly related to college financial aid via the OneLogos platform. Please see Appendix A, for example, message content. ⁶

Some of these messages were general (i.e., the content was the same for all recipients), whereas in others message content was customized according to students’ actual status in the FAFSA filing process. The goals of these messages were to (a) remind students about the importance of the FAFSA and about the steps, timelines, and priority deadlines for applying for financial aid; (b) provide feedback on students’ progress in the aid application process; and (c) facilitate students’ communication with their school counselor to ask questions and obtain additional help and guidance. These messages included links to additional resources such as short informational videos on the FAFSA process created by Federal Student Aid (FSA). ⁷ Each student’s assigned school counselor was the ostensible sender of all text messages, and the messages encouraged students to reply via text (or follow-up in-person with the counselor) with questions or for further assistance with the financial aid process.

As noted above, we customized some of the messages based on student-level FAFSA filing status information made available to the school districts by the THECB. Specifically, we were able to classify students into the following categories:

Students’ FAFSA status information was updated in districts’ data systems every 1 to 2 weeks. As this information was updated, OneLogos updated the message stream that students received. In late April, we closed out the spring messaging with reminders about updating FAFSA information with 2014 taxes, if filed recently, and reminders about likely due dates for enrollment deposits.

During the course of the intervention in treatment schools, control schools did have access to the OneLogos platform and the texting capabilities included. As we discuss below, certain control schools did use this functionality during the intervention period for messaging focused on FAFSA filing. As we argue, if anything, control group access to and use of the texting platform should lead to an underestimate of the impact of an intervention such as this.

Experimental Research Design, Data, and Analysis

We implemented the text-based outreach in selected Texas school districts in the context of a school-level randomized controlled trial (RCT). Our sampling frame includes 66 unique high schools across eight school districts in the Houston and Austin areas.

We randomized schools in the participating districts to one of two experimental interventions, the second of which occurred in the subsequent academic year (here, we report only on the FAFSA-focused messaging intervention for the class of 2015). ¹¹ To ensure balance on key baseline information and to improve statistical power, we first matched sample schools into groups (“group”) of approximately five each, matching on 2013 school-level college enrollment data publicly available through the Texas Education Agency, and then randomly selected approximately three out of each of the five schools to be assigned to the FAFSA messaging intervention. For large districts, we prioritized matching schools within districts but in other cases, we grouped schools across districts. In sum, we randomly selected 39 schools to participate in the FAFSA messaging intervention and 27 schools to serve as control.

We assess baseline equivalence at both the student and school levels. At the student level, we regress each student-level baseline characteristic on school-level random assignment using a model that includes fixed effects for group and that clusters standard errors at the school level. At the school level, we regress school-level baseline measures on an indicator for treatment assignment and include fixed effects for group. We present results assessing balance in Table 1. All results indicate that our sample is well balanced according to both student- and school-level characteristics. Importantly, the schools are balanced on lagged measures of college enrollment (from the class of 2013) and FAFSA filing (from the class of 2014). ¹²

Analysis

We use regression and linear probability models to assess intervention implementation, engagement, and impact. To examine intervention participation and college enrollment outcomes, we fit models of the following general form on data at the student level:

Y i j k = α k + β 1 Treat j k + X i j k γ + S j k θ + ε i j k ,

(1)

where for student i in school j in group k, Y i j k is the outcome of interest, Treat j k is an indicator for treatment assignment at the school level, and X i j k and S j k are vectors of baseline characteristics at the student and school levels. We include fixed effects for group ( α k ) to account for the structure of the randomization and cluster standard errors at the school level. ¹⁵ Our parameter of primary interest is β 1 , which represents the impact of school-level random assignment to the intervention on a given student outcome. Because not all students within a school are treated, β 1 represents the intent-to-treat (ITT) impact. Given potential spillover effects from participating to nonparticipating students within the same school, we reason that the assumptions required to use an instrumental variables strategy to derive treatment-on-the-treated effects are not well met. In addition, the ITT is arguably the most policy-relevant effect. A school or other institution can adopt a text messaging initiative targeted to students, but it cannot force students to read or respond to that information. For these reasons, therefore, we focus exclusively on ITT effects.

To examine impacts on FAFSA submission and completion, we analyze data aggregated to the school level. The associated models take a similar form with outcomes assessed at the school rather than individual level and where we include baseline covariates at the school level only. We report impacts on FAFSA submission and completion at the end of the active intervention period as well as later in the summer.

Experimental Results

In Table 2, we present impacts of the intervention on receipt of text outreach through the texting platform. In the first column, we report impacts on whether students received any text outreach during the course of the intervention. Here, we observe that of students in the control group, approximately 38% did receive some text outreach through the system. This high rate of text receipt is not surprising, given that all schools (both treatment and control) had access to the OneLogos platform. We further examined the messages sent by the control schools to understand the nature of the control group messaging. Within four of the participating districts, messaging only occurred within treatment schools, with no use of the messaging system within the control schools. Within three of the participating districts, control schools used the messaging platform to a minimal degree, with students receiving at most one or two general messages regarding FAFSA. Finally, within one participating district, several control schools essentially crossed over and implemented a significant proportion of the intervention, as described. Given this control group crossover, we reason that the ITT effects on which we focus represent a lower bound on the impact of the messaging campaign.

OPEN IN VIEWER

Table 2

Take-Up and Participation Rates

	Received text outreach	Responded to text outreach	Request texts stop	N texts received by student	N texts sent by student
Treatment	0.317***	0.136***	0.028***	6.637***	0.246***
	(0.061)	(0.015)	(0.005)	(0.912)	(0.038)
Control group mean	0.376	0.050	0.004	2.452	0.093
N	17,731	17,731	17,731	17,731	17,731

Source. District administrative records and project technology partner.

Note. Treatment effects on take-up and participation derived from regression models where we regress each outcome on an indicator for treatment assignment at the school level and fixed effects for groups within which we randomized schools. Robust standard errors (in parentheses) clustered at the school level.

†

p < .10. *p < .05. **p < .01. ***p < .001.

In the treatment schools, the rate of text receipt was 32 percentage points higher, such that approximately 70% of students in the treatment schools received text-based outreach over the course of the intervention. In the remaining columns of Table 2, we observe various indicators of higher text-based engagement in the treatment compared with the control schools. In the control schools, approximately 5% of students texted in to a school counselor, compared with 19% in the treatment schools (column 2). In addition, treatment students received and sent more text messages than their control group counterparts (columns 4 and 5). Treatment group students were modestly more likely to opt out of receiving text outreach, although we interpret an overall treatment group opt-out rate below 3% as an indication that students were neutral to positive about receiving the text-based outreach. This opt out rate is on par with prior text-based interventions (e.g., Castleman & Page, 2015).

Most of the messages students received were prescheduled and sent automatically. Nevertheless, counselors were responsible to follow up on incoming student questions in response. In this way, the quality of the intervention is, at least in part, reliant on counselors’ timely responses to student messages. To understand the quality of counselor engagement with the messaging intervention, we examined the timing and content of the responses students received to the text messages that they sent. Within the treatment group, students sent a total of 3,457 messages during the course of the intervention period. A total of 687 (19%) did not receive any response, however, nearly half of these incoming messages were requests to stop out of the intervention, and most of the other messages did not require any kind of response. Eighty-seven messages that received no response include content where a text response would have been appropriate. Some of these messages asked for confirmation of the text sender and occurred toward the beginning of the text campaign. Others asked about specific FAFSA processes and should have received a substantive response.

A total of 970 (27%) incoming student messages received a same-day response, 754 (21% received a response within 3 days), and 391 (11%) received a follow-up message within 4 to 7 days. The remaining 745 incoming student messages received follow-up messages after a 1-week period, but many of these student messages did not necessarily need a response.

Students asked questions on a variety of topics including confirmation of the message sender’s identity and legitimacy; college application and financial aid application deadlines; general filing procedures; application procedures for undocumented students, specifically; timing of school-sponsored FAFSA workshops; fee waivers and scholarships; and complex family circumstances. These complex circumstances often related to students’ family structure and included issues such as a deceased parent, nontraditional legal guardianship; grandparents with custody; and how to handle an absent father who does not pay child support. Students also used the text messaging to schedule in-person meetings with their counselors. Finally, some of the student questions reflected misunderstandings with the FAFSA/TASFA process, such as one asking how much it cost to file the FAFSA.

Counselor responses essentially reflected student questions and provided replies that served the following purposes: confirming their identify, scheduling in-person meetings, providing general counseling related to both financial aid and college application procedures, confirming technical details like which tax year records to use in FAFSA filing, and catching and providing feedback on errors, such as students filing the FAFSA for the incorrect academic year.

Next, we turn to examine FAFSA submission and completion rates in Table 3. By the end of the FAFSA messaging, FAFSA submission and completion rates were approximately 6 percentage points higher in the treatment schools compared with the control schools. Compared with the control group rates of 40% and 36% for submission and completion, respectively, this is a meaningful impact that translates to an improvement of approximately 15% in FAFSA filing rates. After the intervention period ends, however, these treatment effects attenuated somewhat, although to note is that we are not able to statistically distinguish the late summer effects from those at the end of the intervention. Nevertheless, by late summer, students in treatment schools were 3.3 percentage points more likely to have submitted and 4.4 percentage points more likely to have completed the FAFSA. Thus, the intervention influenced a combination of overall FAFSA filing as well as FAFSA timing, with some students who would have filed regardless doing so earlier because of the outreach. Below, we discuss what implications FAFSA timing may have for financial aid received.

OPEN IN VIEWER

Table 3

Impacts on FAFSA Submission and Completion

	By end of FAFSA messaging(April 17)		By last time point observed(July 24)
	SubmitFAFSA	Complete FAFSA	SubmitFAFSA	Complete FAFSA
Treatment	0.062 †	0.059 †	0.033	0.044
	(0.037)	(0.035)	(0.038)	(0.036)
Control group mean	0.396	0.356	0.485	0.429
N of schools	66	66	66	66
R 2	.44	.439	.397	.42

Source. District administrative records and Federal Student Aid.

Note. Treatment effects on FAFSA submission and completion derived from regression models where we regress school-level completion rates on an indicator for treatment assignment at the school level and fixed effects for groups within which we randomized schools. Robust standard errors (in parentheses). Models control for school-level covariates reported in Table 1. Schools weighted by senior class enrollment. FAFSA = Free Application for Federal Student Aid.

†

p < .10. *p < .05. **p < .01. ***p < .001.

In Table 4, we report impacts on on-time college enrollment, defined as enrollment in the fall semester following high school graduation, based on data from the National Student Clearinghouse. ¹⁶ We find that the intervention improved timely college enrollment by approximately 3 percentage points. This overall enrollment effect is a combination of a larger positive impact (5 percentage points) on enrollment in 4-year institutions and an offsetting negative effect (–2 percentage points) on enrollment in 2-year institutions. Finally, we did not detect significant differences in these impacts according to characteristics such as economic disadvantage.

OPEN IN VIEWER

Table 4

Impacts on College Enrollment Outcomes

	Seamless college enrollment	Two-year college enrollment	Four-year college enrollment
Treatment	0.031 †	−0.020 †	0.052**
	(0.017)	(0.011)	(0.018)
Control group mean	0.498	0.234	0.264
N	17,731	17,731	17,731
R 2	.235	.023	.298

Source. District administrative records and National Student Clearinghouse.

Note. Treatment effects on college enrollment outcomes derived from regression models where we regress each outcome on an indicator for treatment assignment at the school level and fixed effects for groups within which we randomized schools. Models control for covariates reported in Table 1. Robust standard errors (in parentheses).

†

p < .10. *p < .05. **p < .01. ***p < .001.

Exploring Potential Mechanisms

Consistent with previous research (e.g., Bettinger et al., 2012), it is reasonable to expect higher rates of FAFSA completion to be followed by higher rates of college enrollment. It is perhaps surprising that the impacts on FAFSA filing and overall timely enrollment are so similar in magnitude. In fact, we reason that our estimated impacts on FAFSA filing potentially represent a lower bound for the intervention’s impact on students applying for college financial aid overall. This is because the text outreach was inclusive of financial aid application processes for undocumented students (e.g., completing the TASFA), the relevance of which was highlighted in the content of incoming student messages. Because the districts do not collect data on TASFA filing activity, however, any impact of the intervention on TASFA filing is not observable in our data.

Given that the outreach influenced a combination of overall FAFSA (and likely TASFA) filing and the timing with which some students would have otherwise filed, we consider two channels through which earlier FAFSA submission, in particular, may contribute to higher rates of college access. First, all else equal, students who file the FAFSA earlier may be able to access more generous financial aid. This may be particularly so in Texas, where many of the colleges and universities common among the students in our sample have early priority filing deadlines around mid-March. The Texas messages both encouraged early FAFSA filing in general and linked students directly to easy-to-digest information about institution-specific priority filing deadlines. If early filers are awarded more financial aid, on average, then early filing may be a mechanism for improved college access.

To explore this relationship, we use data from the 2011–2012 National Postsecondary Student Aid Survey (NPSAS:12) to examine average institutional grant aid award receipt for students, grouped by Expected Family Contribution (EFC), who filed the FAFSA at various points throughout the spring of their senior year in high school. ¹⁷ We restrict our sample to dependent students whose high school state of residence was Texas and who had filed the FAFSA. We then divided these students into groups within a US$500 EFC range (e.g., an EFC of US$0–US$499, and US$500–US$999) and calculated average institutional aid awards within groups. In Appendix Table B1, we report the average level of institutional aid for students who filed the FAFSA prior to February 1 and then report average differences in aid receipt between these students and students who filed in subsequent months.

Across most EFC categories, students who file after February 1 receive less in institutional grant aid, on average, compared with earlier filers. Disaggregated by institutional type (e.g., 2-year vs. 4-year institutions), ¹⁸ the differentials in institutional aid are driven by variation in awards within the 4-year sector. Across several EFC categories, these data suggest that students may miss out on substantial institutional aid by filing their FAFSA later in the spring, with the largest differences for students with the lowest family EFCs. ¹⁹

Second, by filing the FAFSA earlier, students have more time to successfully navigate the income verification process, if they are flagged to do so. In our experimental sample, of those students we observe to have submitted a FAFSA and for whom we are also able to observe verification status, ²⁰ nearly 40% are flagged for income verification. Furthermore, rates of selection for verification differ substantially by characteristics such as race and socioeconomic status. For example, in our data, White students who are not economically disadvantaged are selected for verification at a rate of approximately 26%. This rate is approximately 39% for non-White students who are not economically disadvantaged, and 42% for economically disadvantaged students regardless of race/ethnicity.

Being flagged for income verification means that a student must complete follow-up procedures with any institution to which she has applied to receive a financial aid package from that school. Thus, verification represents an additional hurdle for a sizable share of students, and if these students file the FAFSA later, they may have less time to successfully navigate the verification process in advance of institutional priority deadlines. To our knowledge, the impact of the verification process on college access has not been explored previously.

In our data, we are not able to observe whether a student successfully navigates the verification process, only whether a student is selected for verification. Nevertheless, if the text messaging helped students to navigate the verification process earlier and/or more successfully, we hypothesize that the intervention could have helped to mitigate any negative impact that verification may have on college access. We observe suggestive evidence to this effect.

In Table 5, columns 1 and 2 we examine the overall relationship between being selected for income verification and timely enrollment among those students who submitted the FAFSA. Among FAFSA filers in both treatment and control schools, three quarters of filers not selected for verification enroll on time. However, on-time enrollment is a significant 5 percentage points lower among those flagged for verification.

OPEN IN VIEWER

Table 5

Relationship Between On-Time College Enrollment and Selection for FAFSA Verification Overall and by School-Level Treatment Status

	(1)	(2)	(3)	(4)
FAFSA verification	−0.052***	−0.051***	−0.065***	−0.064***
	(0.012)	(0.011)	(0.018)	(0.018)
Treatment × FAFSA Verification			0.022	0.022
			(0.024)	(0.023)
Intercept	0.750***		0.750***
	(0.007)		(0.007)
Student-level covariates		✓		✓
N	6,174	6,174	6,174	6,174
R 2	.086	.132	.086	.132

Source. District administrative records and National Student Clearinghouse.

Note. Results from regression models including fixed effects for school. Student-level covariates are those reported in Table 1. Robust standard errors in parentheses. FAFSA = Free Application for Federal Student Aid.

†

p < .10. *p < .05. **p < .01. ***p < .001.

In columns 3 and 4, we present results from analogous models that include an interaction between verification and attending a treatment school. If students were more successful with the verification process in the treatment schools, we should see a positive coefficient on this interaction term. We observe suggestive evidence that this is the case. Both with and without covariate controls, the point estimates on the interaction term parameter indicates that the negative effect of verification selection is 2.2 percentage points smaller within the treatment schools. These estimates are noisy and not precise enough to be statistically significant. Nevertheless, they are suggestive of potential benefit, in that in the treatment schools, the negative effect of verification is approximately 34% smaller than in the control schools. Taken together, this financial aid–focused intervention may have influenced timely college enrollment through a combination of increasing students’ success in completing either the FAFSA or the TASFA, helping students to access more generous aid because of earlier filing, and helping students to more successfully navigate the FAFSA verification process, when required.

Discussion

We observe significant impacts of the text-based outreach on FAFSA submission and completion in addition to downstream impacts on timely college enrollment, especially within 4-year institutions. The intervention reached a large number of students at a relatively low cost. Specifically, across the participating Texas schools, the financial aid intervention reached approximately 7,500 high school seniors at a service provider contract cost of US$60,000, leading to a direct technology cost of approximately US$8/student reached. We can also frame costs in terms of expenditure per student enrolled in college. The treatment schools collectively served approximately 10,000 students. Among students served by these schools, we estimate that the intervention increased timely college enrollment by approximately 3 percentage points, or 300 students. Scaling our provider costs by this denominator equates to a cost of approximately US$200 per impacted college enrollee. Framed either way, the low-cost and moderate impacts together underscore the benefit of text-based outreach as a readily scalable strategy for improving student completion of important college-going milestones, such as timely FAFSA filing.

We regard students receiving the text-based outreach as college-intending. This is indicated through students’ establishment of an Apply Texas profile, an account that they use to apply to colleges and universities within the state. In previous work, we find that even among college-intending high school graduates who are accepted to a college or university and intend to enroll, many fail to actually matriculate (Castleman & Page, 2014b). Furthermore, low-cost interventions to provide outreach and support to students in the summer months can help to mitigate this “summer melt” (e.g., Castleman & Page, 2015). Yet, advising staff who participated in these summer interventions cautioned that the summer melt we observed actually began in the winter, when students and families should be navigating the FAFSA (Castleman & Page, 2014a). Consistent with this notion, the impacts that we observe of this financial aid–focused intervention on college enrollment are similar in magnitude to those for analogous efforts to mitigate summer melt. Thus, these results lend credence to the notion that more proactive college-going support ought to focus on the financial aid application process and that there is benefit to maintaining this focus on supporting students to complete the FAFSA (or other applications) prior to the end of the school year. By supporting students with this key milestone and shepherding them through the process earlier in the year, rates of college access may be improved among college-intending high school graduates. Furthermore, in our data, approximately two in five FAFSA filers were flagged for income verification, with prevalence higher for students indicated as low income. Thus, moving up students’ FAFSA filing timeline and allowing them more time during the school year to navigate verification (while also prompting the need for them to do it) may be a particularly important benefit of the federal government’s shift to earlier FAFSA filing based on prior-prior year tax data for the first time for the graduating class of 2017.

Finally, this work points to the importance of going beyond just data provision to improve educational systems. Rather, educational (and other social) systems need to have efficient procedures to make use of available information. A key innovation in this effort is the use of administrative data to better target outreach to students. Specifically, the system that we devised represents an efficient strategy for schools to make better (and automated) use of the student-level FAFSA status data that they receive. The positive impacts of these data-informed nudges stand in contrast to other recent FAFSA completion campaigns which provided generic, FAFSA-focused outreach to students via text message and other channels and which had no impact on financial aid receipt or college enrollment (Bird et al., 2019).

A qualification to this point is that we did receive some reports of discrepancy between students’ reported FAFSA filing activity and indicators represented in the data. For example, counselors sometimes received text or in-person communication from students indicating that they had submitted their FAFSA after having received a text message indicating that counselor’s records suggested otherwise. Data accuracy is no doubt a fundamental prerequisite to the long-run success of a system such as this. Nevertheless, we provide evidence in support of using data to inform students about and support them through one such process for accessing a key social benefit. The positive impacts that we observe here inspire thinking about other data-informed nudges to encourage and support students along their educational trajectories.