Sign Me Up: The Factors Predicting Students’ Enrollment in an Early-Commitment Scholarship Program

Abstract

We investigate factors influencing student sign-ups for Washington State’s College Bound Scholarship program and consider whether there is scope for the program to change college enrollment expectations. We find that student characteristics associated with signing the scholarship pledge closely parallel characteristics of low-income students who attend 4-year colleges, suggesting that signing the pledge is driven largely by preexisting expectations of college going. We also find evidence that student sign-up rates are lower than have been previously reported, which is important given the perception among program administrators that nearly all eligible students sign up.

Keywords

college financial aid low-income youth scholarship programs program take-up

College and credential completion rates have recently increased for the first time since the Great Recession (Shapiro et al., 2016, 2017a), but college attainment gaps among various demographic and socioeconomic groups persist, with minority and low-income students being far less likely to obtain college degrees (Shapiro et al., 2017a, 2017b; U.S. Department of Education, 2016). One factor that may contribute to this gap is low expectations of college attendance due to the unaffordability of college for disadvantaged youths. For instance, Perna (2008) finds that perceptions of college affordability are likely to matter to young people and that perceptions about college attainability help shape their high school trajectories (Hill & Wang, 2015; Howard, 2003).

A state policy response to the relatively low level of college attainment of disadvantaged students is to make college more affordable and also alert disadvantaged students early, letting them know that college is within their grasp. One such program is an early-commitment, need-based scholarship in which states or local areas commit early in a low-income student’s schooling to provide financial aid for college if, in return, the student pledges to do well in school and stay out of trouble. A central theory of action of these early-commitment programs is that the pledge will change students’ expectations about college while they still have time to prepare for college.

There is very little evidence about the factors that predict whether students sign up for these early-commitment pledge programs.¹ This is an important gap in the financial-aid and college-going literature given that students can financially and academically benefit from such programs only if they sign up for them.

This paper evaluates Washington State’s College Bound Scholarship (CBS), an early-commitment scholarship that promises full tuition to low-income middle school students who participate. By examining student-level historical data and conducting qualitative interviews, this paper addresses the following questions: (a) What student and middle school characteristics are associated with signing the CBS pledge? (b) How closely do these characteristics parallel preprogram predictors of college enrollment? (c) What do program administrators report doing to encourage student uptake of the program? We address the first two questions through quantitative analyses, using administrative data from Washington State. The third question is addressed by a qualitative component of our work that involves semistructured interviews with administrators from the College Success Foundation (which, as we describe later, is involved with outreach efforts to promote the CBS) and middle school guidance counselors.

We find that factors associated with signing up are similar to factors associated with enrollment in college, and this pattern holds for each of the first three cohorts of CBS-eligible students. Indeed, there is little evidence that the demographic patterns predicting sign-ups change over the first three cohorts of program administration, calling into question the extent to which the pledge program is changing college-going expectations. Additionally, we find there is a positive correlation between predicted and actual sign-up rates at the school level, but the correlation (.40) is not overwhelmingly high, suggesting that targeting resources toward encouraging student sign-ups at schools with surprisingly low sign-up rates could significantly boost overall program enrollment.

Conceptual Model/Theory of Action for Early-Commitment Pledge Programs

Human capital theory posits that individuals make decisions about human capital investments that are thought to enhance productivity and earnings (Altonji, Blom, & Meghir, 2012; Becker, 1993). Early-commitment pledge programs may change the calculus when it comes to decisions about investing in college preparation by providing an early signal that the financial resources to attend college are within a student’s grasp.

However, the extent to which pledge programs influence college going is also likely to be influenced by the information that students receive about programs as well as the guidance they get about how to prepare for college success (Oreopolous & Dunn, 2012). The role played by school administrators in providing students information about college is likely to be particularly important for low-income students given that research (Avery & Kane, 2004) finds that they are substantially less likely to have an accurate picture of college admissions processes or costs.

The CBS provides clear guidance about academic and behavioral requirements for the receipt of a scholarship, along with the promise that students adhering to the requirements are likely to have the resources necessary to attend college (assuming they remain income eligible in 12th grade); for most eligible students, the program essentially eliminates the cost of attending a Washington public college (aside from room and board). Thus, the CBS should increase forward-looking students’ expectations about college (and hence make them more likely to prepare for college in high school, apply to colleges, and enroll).

Students must sign up in middle school for the CBS in order to be eligible for the scholarship in the 12th grade (more details about the CBS are available in Appendix A online). We believe signing up is an early indicator of college-going expectations. But students eligible for the CBS will also have heterogeneous expectations about college: Some, for instance, are likely to go to college irrespective of the CBS program. The hope is that the program, as it is rolled out over time, is encouraging eligible students who had not previously considered college to make college an expectation. Thus, given the theory of action, we test two hypotheses:

Hypothesis 1: The factors that predict college going among eligible students will also predict the likelihood of signing up for the CBS program.

Hypothesis 2: As the program is rolled out over time, the relationship between the factors predicting college going and signing up for the program will be lessened.

The first hypothesis reflects the fact that students with a high propensity to go to college should also have a high propensity to sign up for a program that provides them with a college scholarship. The second hypothesis reflects the hope that as the program is implemented and word spreads about the availability of this source of college funding, students who would not previously have considered college as an option begin to do so, weakening the relationship between the factors predicting college and program sign up.

Literature Review

Several states have implemented financial aid programs designed to address college enrollment and attainment gaps by offering in-state low-income adolescents an early promise of funding for college in exchange for a student’s early-commitment pledge (typically during seventh to ninth grades). Pledges commonly ask students to do well in high school, be a good citizen, and apply to college. Currently there are three states (Indiana, Oklahoma, and Washington) that are operating early-commitment pledge programs that meet Blanco’s (2005) “three core criteria for early commitment programs: that they make a guarantee of aid; that aid is designated only for economically disadvantaged students; and that students are identified in elementary, middle school, or early high school” (p. 9).²

These early-commitment pledge programs are similar to other promise programs that spread rapidly in recent decades (Dynarski, 2004; LeGower & Walsh, 2017).³ The effects of these promise programs, that is, programs in which states or local municipalities make a commitment to fund at least some part of in-state college tuition, have been studied extensively. The findings tend to show positive effects for in-state college matriculation (with some diversion of students from out of state to in state) and credit accumulation (e.g., Bartik, Hershbein, & Lachowka, 2017; Carruthers & Özek, 2016; Cornwell, Mustard, & Sridhar, 2006; Page, Iriti, Lowry, & Anthony, in press; Perna & Leigh, 2017; Scott-Clayton, 2011; Sjoquist & Winters, 2014). Yet, these programs may also exacerbate college attainment gaps between advantaged and disadvantaged students. For instance, Dynarski (2000) finds that “Georgia’s program has likely increased the college attendance rate of all 18- to 19-year-olds by 7.0 to 7.9 percentage points” yet “widened the gap in college attendance between blacks and whites and between those from low- and high-income families” (p. 629).⁴

Although early-commitment pledge programs are similar to other promise programs insofar as they typically offer aid at in-state colleges for students achieving modest academic thresholds, they differ from other promise programs in a key way: They require students to sign an early-commitment pledge as a condition of eligibility. This programmatic element is designed to increase college going and, importantly, close college attendance gaps. In particular, they are thought to help low-income students directly by making college more affordable and indirectly by signaling to them early enough that college is financially within their reach. In doing so, the early promise of a college scholarship is expected to raise students’ expectations about the feasibility of college attendance and create a strong incentive for them to do well in high school and fulfill pledge requirements.

It is worth emphasizing that the opt-in nature of the pledge requirement is a feature designed to target students, particularly those who, due to financial concerns, might not have college-going expectations entering high school. College-going aspirations are expected to change high school behavior, for example, to lead students to take high school courses that ready them for college. But if opting in has no effect on subsequent behavior in high school, then it is likely that early-commitment pledge programs will be less effective than other promise programs wherein eligibility is automatic, because some students will fail to make themselves eligible for scholarships by not signing up; Thaler and Sunstein (2003) show in a variety of contexts that programs requiring participants to opt in tend to have lower participation rates than programs in which participation is the default option.

There is a striking dichotomy between the aspirations of younger students to attend college and the actuality of college enrollments.⁵ And when college funding is available, low-income students are less likely to know about it, are less familiar with the process for attaining it, and/or lack a good understanding of the academic requirements needed to be admitted and to succeed in college (Bettinger, Long, Oreopoulos, & Sanbonmatsu, 2012; Dynarski & Scott-Clayton, 2006; Hahn & Price, 2008; Kirst, Venezia, & Antonio, 2004; B. Long & Riley, 2007; Rosenbaum, 2001). Not surprisingly, then, by senior year, low-income students are substantially less likely to have taken courses necessary to succeed in college (Choy, 2001; Jacob & Linkow, 2011). Nevertheless, Castleman and Goodman (2018) found that intensive high school counseling benefits low-income students, and the program they studied “successfully shifts enrollment toward four-year colleges,” which further resulted in “persistence through the third year of college” (p. 19).

Existing studies on early-commitment opt-in pledge programs are limited in their ability to identify treatment effects as they lack data needed to form appropriate comparison groups for those students who are eligible to participate in these programs. St. John and colleagues (St. John et al., 2004; St. John, Fisher, Lee, Daun-Barnett, & Williams, 2008; St. John, Gross, Musoba, & Chung, 2005; St. John, Hu, & Weber, 2001; St. John, Musoba, & Simmons, 2003) investigated the impact of Indiana’s program yet lacked both data on cohorts of students before the introduction of the pledge program and data needed to identify if a student was eligible for the program. As a result, they were forced to compare students who signed the pledge with a comparison group of students who may or may not have been eligible.⁶ Fortunately, as we describe later, we have access to both CBS eligibility indicators and data on prepolicy cohorts of students. This allows us to explore the extent to which factors predicting whether students sign the pledge look similar to the factors associated with college enrollment from before the implementation of the CBS. In other words, we are able to investigate the degree to which the aspirations surrounding college attendance of different groups of students do or do not change.⁷

Methods and Data

Quantitative Analytic Methods

We begin with a quantitative analysis of the individual and school characteristics that predict the likelihood of a student signing the CBS pledge in middle school. We estimate the specifications shown in Equations 1a and 1b using data on the first three cohorts of CBS-eligible students:

Signe d_{i m} = F (β_{0} + β_{1} X_{i} + M S C E_{m} + ε_{i m}) .

(1a)

Signe d_{i m} = F (β_{0} + β_{1} X_{i} + β_{2} S_{m} + ε_{i m}) .

(1b)

The subscript i denotes the student and m denotes the middle school attended in eighth grade.⁸ The dependent variable for this analysis, Signed_im, is an indicator variable that equals 1 if the student signed the pledge by the end of eighth grade (or ninth grade for Cohort 1⁹). The student-level predictors (i.e., the X_i vector) include standardized scores on Washington’s reading and mathematics assessments—known as the Washington Assessment of Student Learning (WASL)—when the student was in sixth grade and indicators for taking the WASL tests out of grade level and taking a modified version of the WASL,¹⁰ student’s age in May of eighth grade, and indicators for female, race-ethnicity group (Hispanic; non-Hispanic African American, Asian, Hawaiian or Pacific Islander, or Native American or Alaskan Native; or non-Hispanic and two or more races), disability status, migrant status, homeless status, “highly capable” status, “transitional bilingual” status, language spoken at home other than English, and attendance at a public school part-time (as a homeschooler or private school enrollee).

We estimate Equation 1a using a logit specification with middle school fixed effects, MSCE_m, to account for time-invariant school-level factors that may influence students’ signing the pledge.¹¹ We cluster the standard errors of all the regressions at the middle school level to account for the fact that guidance counselors and school culture are both thought to be important determinants for whether middle school students sign the pledge; that is, there is likely to be within-cluster error correlation (Cameron & Miller, 2015).

We estimate Equation 1b (which drops middle school fixed effects and adds middle school characteristics, S_m) using a hierarchical logistic regression (i.e., the logit analog of hierarchical linear modeling) with random intercepts. The advantage of the fixed-effects specification (Equation 1a) is that it accounts for both observable and unobserved time-invariant school-level factors. The disadvantage of Equation 1a over Equation 1b is that the former does not estimate the way that student sign-up rates are influenced by observable school-level characteristics. The following middle school characteristics are included in Equation 1b: percentage of students receiving free or reduced-price lunch (FRPL), mean standardized student score on the sixth-grade mathematics WASL among eighth-grade students in the school, eighth-grade enrollment in fall (divided by 100), whether the school has a guidance counselor, the proximity to college slots (measured as the number of undergraduate students at a 4-year college within a 50-mile radius¹²), and an indicator for the school’s region (Puget Sound [including King, Pierce, Kitsap, Thurston, and Snohomish counties], the remainder of western Washington [i.e., west of the Cascade Mountains], or eastern Washington). After running these logistic regressions, we compute the marginal effect of a one-unit increase in each variable (holding all other variables constant) on the probability that the student signs the pledge.

Next, we estimate versions of Equations 1a and 1b where we replace the outcome with an indicator for enrolling in any college within 4 years of ninth-grade entry, Enrolled_im.¹³

E n r o l l e d_{i m} = F (γ_{0} + γ_{1} X_{i} + M S C E_{m} + ε_{i m}) .

(2a)

E n r o l l e d_{i m} = F (γ_{0} + γ_{1} X_{i} + γ_{2} S_{m} + ε_{i m}) .

(2b)

For these regressions, we use data from two pre-CBS cohorts.¹⁴ These regressions allow us to evaluate whether there are meaningful differences between the β₁ and β₂ vectors of coefficients using “Signed” as the dependent variable and the γ₁ and γ₂ vectors of coefficients using “Enrolled” as the dependent variable. If these sets of coefficients are similar, it would indicate that the types of students who are more prone to attend college are also more likely to sign the pledge. Substantial differences in the coefficients may also provide useful information.¹⁵

To assess the similarity of the estimated vectors of coefficients using “Enrolled” rather than “Signed” as the dependent variable, we examine the correlation between the sets of marginal effects found for the two dependent variables. Then, to gain a better sense of whether the CBS program is encouraging students who are demographically unlikely to attend college, we compare the predicted probabilities generated by the enrollment and pledge models. We investigate whether the predicted enrollment and pledge probabilities align or diverge for each cohort of students eligible to sign the CBS pledge. Specifically, we assess the extent to which the estimated coefficients from specification 2b (the coefficients from the college enrollment model) predict the estimated probabilities of signing the pledge for each CBS-eligible cohort (generated from estimating specification 1b, the pledge model, separately by cohort).¹⁶

More formally, we generate two predicted probabilities for each individual in each postpolicy cohort: ${\hat{P}}_{P i c m} = F ({\hat{β}}_{0 c} + {\hat{β}}_{1 c} X_{i} + {\hat{β}}_{2 c} S_{m})$ , which is the predicted probability of signing the pledge using the pledging coefficients estimated based on members of postpolicy pledge cohort c, and ${\hat{P}}_{E i m} = F ({\hat{γ}}_{0} + {\hat{γ}}_{1} X_{i} + {\hat{γ}}_{2} S_{m})$ , which is the predicted probability of a postpolicy student enrolling in college based on prepolicy enrollment coefficients. We then assess the correspondence between these predicted probabilities by estimating the following regression for each postpolicy pledge cohort:

{\hat{P}}_{P i c m} = α_{0 c} + α_{1 c} {\hat{P}}_{E i m} + α_{2 c} {\hat{P}}_{E i m}^{2} + α_{3 c} {\hat{P}}_{E i m}^{3}

(3)

We estimate Equation 3 as a cubic as we are particularly interested in whether there are differential changes in the likelihood of signing the pledge along the distribution of probabilities of enrolling in college.

Data for Quantitative Analysis

The data we utilize are compiled by Washington State’s Education Research and Data Center (ERDC). ERDC maintains individual student-level K–12 records for all public school students in the state that can be linked to information about enrollment in 2- or 4-year colleges in Washington state as well as those outside the state (through the National Student Clearinghouse). The ERDC data include K–12 student information dating back to the 2005–2006 school year, providing us data on two cohorts of students who did not have the opportunity to receive a CBS scholarship (those who were in eighth grade in 2005–2006 and 2006–2007) and three cohorts who were eligible to sign up (those who were in eight grade in 2007–2008, 2008–2009, or 2009–2010). We restrict the analysis to students who are known to be FRPL eligible, and therefore eligible to sign the pledge, in seventh or eighth grade (or eighth or ninth grade for the first postpolicy cohort due to a temporary change in eligibility for this cohort), which brings our analytical sample size to 191,337 students.¹⁷

The CBS program is overseen by the Washington Student Achievement Council (WSAC), which “provides strategic planning, oversight, and advocacy to support increased student success and higher levels of educational attainment in Washington” (WSAC, n.d.). As part of its efforts to support the CBS, WSAC monitors sign-up rates across the state and by district. WSAC reported substantial success in increasing the sign-up rate since the CBS program began. Its initial calculations, reported in Figure 1, show that the sign-up rate was 57% for the first cohort of eligible students and rose to 85% by the sixth cohort. In 2016, its website stated,

In 2015, 91% of the Class of 2019 (8th graders whose deadline was June 30, 2015) submitted complete applications. This year, 110 districts had sign-up rates of 92% or higher. Of these, 77 school districts saw 100% of their eligible students sign up.¹⁸

Figure 1.

Sign-up rates according to the Washington Student Achievement Council (personal communication, Rachelle Sharpe, senior director of student financial aid and support services, Washington Student Achievement Council, August 23, 2013).

Figure 1 suggests improvement over time in sign-up rates and little room for further improvement in a large number of school districts, as these approached 100% participation/sign-up. Our estimates, discussed next, confirm improvement over time, but as we describe, we found that the baseline/first-year sign-up rate to be significantly lower than the figures previously reported by WSAC, suggesting room for growth in sign-up rates.

In particular, our calculations were somewhat different than those previously used by WSAC (WSAC has, since the release of our working paper, adjusted its method for calculating sign-up rates). To calculate the sign-up rate, we utilize student-level FRPL data, which capture eligibility in both seventh and eighth grades corresponding to the policy’s eligibility requirements. In Table 1, we show our calculations of the sign-up rate for the first three eligible “postpolicy” cohorts. In row (A), we find that the number of students signing the pledge increased from 14,176 to 18,802 across these three cohorts.¹⁹ By contrast, WSAC reported (as shown in Figure 1) that the number of students signing the pledge increased from 15,947 to 20,903 across these same three cohorts. Our total counts of students who signed the pledge are lower because we do not count pledges from foreign exchange students, students in foster care in seventh or eighth grade, students who have irreconcilable birthdays across observations, or students who did not attend eighth grade in a Washington school. (Note that these same students are not included in our denominator.) Also, we do not count pledges for which there was no corresponding student in our K–12 database.

Table 1

Sign-Up Rates

	Expected high school graduation year
	Prepolicy		Postpolicy
	2010	2011	2012	2013	2014
	Seventh grade in 2004–2005 or eighth grade in 2005–2006	Seventh grade in 2005–2006 or eighth grade in 2006–2007	Eighth grade in 2007–2008 or ninth grade in 2008–2009	Seventh grade in 2007–2008 or eighth grade in 2008–2009	Seventh grade in 2008–2009 or eighth grade in 2009–2010
(A) Number who signed the pledge	25	234	14,176	15,143	18,802
(B) Number whom we identify as “clearly eligible”^a	39,218	37,450	38,659	37,519	38,491
(C) Number who signed the pledge and whom we identify as “clearly eligible”^a	24	216	13,037	13,943	17,524
Sign-Up Rate 1: (A)/(B)	0.1%	0.6%	36.7%	40.4%	48.9%
Sign-Up Rate 2: (C)/(B)	0.1%	0.6%	33.7%	37.2%	45.5%
Sign-Up Rate 2 for Puget Sound counties	0.1%	0.6%	31.0%	34.5%	45.4%
Sign-Up Rate 2 for other western Washington counties	0.1%	0.3%	28.1%	32.7%	42.1%
Sign-Up Rate 2 for eastern Washington	0.0%	0.6%	41.7%	44.5%	48.4%

“Clearly eligible” defined as a student that was marked as eligible for free and reduced lunch program at any point during seventh- or eighth-grade year (eighth or ninth grade for the first postpolicy cohort).

Next, in row (B), we show that the number of clearly eligible students remained stable, ranging from 37,519 to 38,659 across the first three cohorts. This denominator is substantially larger than the denominator previously reported by WSAC (i.e., 28,093 to 29,856 as shown in Figure 1). WSAC used the number of FRPL students in seventh grade as recorded in October of the corresponding school year. The result is an early snapshot of eligibility that does not include students who are added to FRPL programs throughout the remainder of their seventh grade or at any point during their eighth grade.²⁰

We believe the sign-up rates we report represent an upper bound given that they do not include students who are made eligible to sign the pledge by virtue of participation in the Supplemental Nutrition Assistance Program, the Food Distribution Program on Indian Reservations, or Temporary Assistance for Needy Families or who sign the pledge despite not being income eligible.²¹ The sign-up rate we calculate by dividing (A) by (B) is 36.7% for the first cohort and 48.9% for the third cohort. These figures that are substantially lower than the 57% and 70% rates reported by WSAC in Figure 1, although the increase in our calculated rate over time is very close to that reported by WSAC (12.2 percentage points from first to third cohort vs. WSAC’s report of a 13-percentage-point change).

In an effort to better track progress on the CBS sign-up rates, soon after the release of our working paper on July 28, 2016, WSAC changed the way it calculates sign-up rates to have its denominators reflect student mobility into CBS eligibility and called the new rates “reconciled rates” (Shankster, Larson, & Kwakye, 2017). The distinction in the way sign-up rates are calculated may well be important because school districts and schools until 2017 could have thought that they had only a few eligible students who had not signed the CBS pledge when in fact there may be many more.²²

In the last rows of Table 1, we show that there were sizable differences in sign-up rates across the three regions of the state, particularly in the first cohort, with eastern Washington leading and western Washington counties that surround the Puget Sound counties trailing. It is interesting to note that although eastern Washington had a high sign-up rate for the scholarship program compared to western Washington counties (41.7% vs. 28.1% for the first scholarship cohort), both regions have similar college enrollment rates (28.2% vs. 27.1% for the first scholarship cohort). This heterogeneity in sign-up rates suggests the possibility of uneven communication and understanding of the program across the state and provides an argument for the qualitative research that we carry out.

Table 2 presents the descriptive statistics for our analytic sample, which consists of those students who are clearly eligible to sign up. Among eligible students, the share that enrolled in any college within 4 years after starting ninth grade increased across the two prepolicy cohorts (from 23.6% to 25.7%) and then increased again for the first postpolicy cohort to 27.2%. Eligible students saw small gains in their test scores relative to not-clearly-eligible students (e.g. students who qualify based on family income but do not receive free or reduced-priced lunch). Finally, note that eligible students tend to live closer to more undergraduate students than not-clearly-eligible students (as indicated by the standardized number of undergraduates within 50 miles being greater than zero for eligible students). See Appendix B online for a discussion of how missing data are handled.

Table 2

Descriptive Statistics for Quantitative Analysis Sample

Variable	Pre-imputation mean and standard deviation					Number of observations in pooled sample with nonmissing data
	Prepolicy		Postpolicy
	Seventh grade in 2004–2005 or eighth grade in 2005–2006	Seventh grade in 2005–2006 or eighth grade in 2006–2007	Eighth grade in 2007–2008 or ninth grade in 2008–2009	Seventh grade in 2007–2008 or eighth grade in 2008–2009	Seventh grade in 2008–2009 or eighth grade in 2009–2010	Prepolicy (two cohorts pooled)	Postpolicy (three cohorts pooled)
Dependent variable
Signed the pledge	0.10%	0.60%	33.70%	37.20%	45.50%	76,668	114,669
Enrolled in college in 4 years of ninth grade	23.60%	25.70%	27.20%	Incomplete		76,668	114,669
Student characteristics
Reading WASL^a	−0.38 (0.96)	−0.36 (0.95)	−0.32 (1.00)	−0.32 (0.98)	−0.32 (0.97)	66,038	969,223
Math WASL^a	−0.4 (0.93)	−0.39 (0.94)	−0.37 (0.94)	−0.39 (0.92)	−0.38 (0.90)	66,102	96,375
Took WASL out of grade level	45.7%	55.3%	24.7%	22.8%	19.0%	76,668	114,669
Took a modified version of WASL	8.5%	8.9%	8.3%	9.0%	8.5%	76,668	114,669
Age in eighth grade	14.4 (0.80)	14.4 (0.50)	14.4 (0.50)	14.4 (0.50)	14.4 (0.50)	76,668	114,669
Female	48.0%	48.7%	48.4%	48.7%	48.4%	76,668	114,669
Hispanic	25.6%	28.4%	30.2%	31.9%	31.6%	76,668	114,669
Non-Hispanic African American	7.4%	7.0%	6.6%	6.5%	6.1%	76,668	114,669
Non-Hispanic Asian	5.9%	5.7%	5.7%	5.6%	5.4%	76,668	114,669
Non-Hispanic Hawaiian or Pacific Islander	0.1%	0.2%	0.2%	0.3%	0.3%	76,668	114,669
Non-Hispanic Native American or Alaskan	3.2%	2.3%	2.0%	2.0%	1.8%	76,668	114,669
Non-Hispanic and more than one race	7.8%	10.1%	10.3%	10.4%	10.8%	76,668	114,669
Disability	20.3%	20.4%	21.6%	23.0%	24.1%	76,668	114,669
Migrant	7.3%	7.9%	8.4%	9.0%	8.6%	76,668	114,669
Homeless	8.0%	10.3%	11.5%	12.6%	13.6%	76,668	114,669
Gifted (“highly capable”)	2.5%	3.9%	3.9%	6.1%	6.2%	76,668	114,669
Receives bilingual services	13.5%	16.0%	18.5%	21.7%	23.2%	76,668	114,669
Language spoken at home not English	22.9%	25.8%	28.1%	30.5%	31.3%	76,668	114,669
Attends public school part-time	0.3%	0.3%	0.5%	0.4%	0.6%	76,668	114,669
Attended eighth grade in Puget Sound county	48.7%	50.1%	49.6%	49.4%	49.3%	76,668	114,669
Attended eighth grade in western Washington county	19.6%	19.1%	19.5%	19.9%	20.3%	76,668	114,669
Attended eighth grade in eastern Washington	31.7%	30.8%	30.9%	30.7%	30.5%	76,668	114,669
Middle school characteristics
Average Math WASL^a	−0.24 (0.20)	−0.20 (0.17)	−0.21 (0.17)	−0.19 (0.16)	−0.22 (0.16)	76,651	114,650
Eighth-grade fall enrollment	243 (106)	238 (101)	236 (106)	235 (106)	233 (105)	74,916	113,251
Percentage of student body on FRPL	47.7 (21.7)	47.6 (21.4)	48.5 (21.7)	50.6 (20.8)	51.8 (21.1)	75,874	114,247
Undergraduate enrollment in a 50-mile radius (standardized across schools within cohorts)	0.10 (1.01)	0.16 (1.01)	0.15 (1.00)	0.15 (1.00)	0.15 (1.00)	76,439	114,438
Has a guidance counselor	90.9%	92.1%	93.6%	94.0%	92.7%	76,668	114,669

Note. Standard deviations are in parentheses. Students are eligible to sign the pledge in eighth or ninth grade for Cohort 1 and in seventh or eighth grade for subsequent cohorts. WASL scores are based on sixth-grade administration for postpolicy cohorts and seventh grade for prepolicy cohorts. WASL = Washington Assessment of Student Learning; FRPL = free or reduced-price lunch.

Scores are standardized within test type, test grade, and cohort.

Qualitative Data and Analytic Methods

The goal of our qualitative analysis is to complement the quantitative research on uptake of the CBS by providing a deeper understanding of how school-level administrators understand the CBS, how information is conveyed to students, and what efforts schools are making to get students to sign the pledge. As the empirical evidence in Hurwitz and Howell (2014) suggests, school administrators, and guidance counselors in particular, play a vital role in laying the groundwork for students to be on a college-ready track and successfully enroll in college. We believe such qualitative information is key to understanding the CBS take-up rate.

The qualitative analysis approach was intended to align with the structure of the CBS; that is, a set of regional officers oversee a geographical area and provide guidance and support to school administrators. School administrators then (almost always) delegate the program administration to their guidance counselor or administrative staff. Interviews were conducted at all three levels to see if there was a disconnect that was limiting program implementation or that could help explain any potential differences between the qualitative and quantitative findings.

Our qualitative analyses draws upon data from semistructured interviews. The first set of interviews included the five CBS regional officers and their director on the College Success Foundation staff. The interviews were conducted either individually or in a small group between November and December 2015. Interview protocols, which are included in Appendix C online, were designed to ensure consistent data collection on critical themes across respondents and to facilitate systematic analysis. At the same time, they allowed for unanticipated themes to emerge. A senior researcher led interviews while an analyst took notes. Interviews lasted between 60 and 120 min.

These interviews touched on a variety of topics, including

Efforts to inform and engage various stakeholders in the CBS program (schools, students, parents, community organizations),²³

Practices to encourage student uptake,

Factors limiting student uptake,

Evidence that the program is affecting students’ behavior, and

Factors limiting the program’s success.

After the interviews, the research team distilled notes into analytic themes. We then identified areas of agreement, areas of disagreement, and patterns with respect to these themes. Overall, these interviews provided a high-level perspective on the CBS’s sign-up activities around the state. These interviews were used to inform development of the protocol for the second set of interviews with middle school principals and guidance counselors.

For a second set of interviews, we selected a diverse sample of middle schools that were based on geographic region and the extent of their correspondence (or divergence) in the school’s actual sign-up rate to their predicted sign-up rates. We predicted each school’s sign-up rate for 2011–2012 as follows:

\hat{S i g n e d_{m}} = F ({\hat{β}}_{0} + {\hat{β}}_{1} {\bar{X}}_{m} + {\hat{β}}_{2} S_{m}),

(4)

where $\hat{S i g n e d_{m}}$ is the school’s predicted sign-up rate, ${\bar{X}}_{m}$ indicates the average of the school’s students’ characteristics and S_m are the school’s characteristics in 2011–2012, and ${\hat{β}}_{0}$ , ${\hat{β}}_{1}$ , and ${\hat{β}}_{2}$ are estimated coefficients from Equation 1b, based on data from students in the first three CBS-eligible cohorts.²⁴ Table 3 shows the distribution of the 27 schools in which we conducted interviews. We conducted 30 principal and guidance counselor interviews between March and August 2016, including 25 guidance counselors (or other CBS program leaders) and five principals. We also interviewed one college readiness coordinator identified through professional contacts.

Table 3

Respondent Sample by Region and Predicted Versus Actual Sign-Up Rate

School characterization	Puget Sound	Remainder of western Washington	Eastern Washington	Total
Predictably low sign-up rate	1	3	2	6
Surprisingly low sign-up rate	2	2	2	6
Typical school	2	2	2	6
Surprisingly high sign-up rate	2	1	0	3
Predictably high sign-up rate	2	2	2	6
Total	9	10	8	27

Note. Predictably-low-sign-up schools are in the bottom 20% for both actual and predicted sign-up rates. Surprisingly-low-sign-up schools are in the bottom 20% for actual sign-up rate and in the top 20% for predicted sign-up rate. Typical schools are in the middle quintile for both actual and predicted sign-up rate. Surprisingly-high-sign-up schools are in the top 20% for actual sign-up rate and in the bottom 20% for predicted sign-up rate. Predictably-high-sign-up schools are in the top 20% for both actual and predicted sign-up rates.

Analysis of principal and guidance counselor interviews proceeded in three steps. First, the research team developed the infrastructure of a database corresponding to the interview protocols, with items organized around the research questions and key themes that emerged from the regional officer interviews. Second, a trained analyst coded interview notes as they were generated, entering information into the database with a senior adviser conducting periodic checks for quality and consistency of the data. Finally, researchers extracted and analyzed data across cases on the key questions and themes. Analytic themes that emerged from these interviews are discussed in detail in the Results section and the Discussion and Conclusion section.

Results

Findings for Research Question 1: What Student and Middle School Characteristics Are Associated With Signing the CBS Pledge?

Table 4 reports three model specifications for the likelihood of students signing the pledge. Column (1) of Table 4 presents raw (not regression-adjusted) differences in sign-up rates for each student characteristic, column (2) shows the school fixed-effects specification (Equation 1a), and column (3) shows the specification with school characteristics (Equation 1b). Although there are some differences in the estimated coefficients between the models that do not condition on other characteristics (column [1]) and those that do (columns [2] and [3]), there is a fairly consistent pattern of the coefficients decreasing in magnitude but maintaining their direction and significance with the addition of statistical controls. Those coefficients that do change substantially—such as taking a modified version of the WASL and attending public school part-time—are for smaller student subgroups.

Table 4

Marginal Effects of Student and School Characteristics on Signing the Pledge

	(1)	(2)	(3)
Variable	Raw differences not conditioned on other characteristics	Logit model with middle school fixed effects	Mixed effects logit model with school characteristics
Student characteristics
Math WASL (standardized)	.049 (.004)***	.045 (.003)***	.044 (.003)***
Reading WASL (standardized)	.050 (.004)***	.029 (.003)***	.027 (.003)***
Female	.073 (.004)***	.057 (.003)***	.056 (.003)***
Hispanic	.089 (.015)***	.016 (.005)***	.016 (.005)***
Non-Hispanic African American	.062 (.012)***	.080 (.007)***	.081 (.008)***
Non-Hispanic Asian	.098 (.013)***	.050 (.008)***	.054 (.005)***
Non-Hispanic Hawaiian or Pacific Islander	−.102 (.029)***	−.070 (.027)***	−.073 (.024)***
Non-Hispanic Native American or Alaskan	−.120 (.019)***	−.060 (.014)***	−.057 (.012)***
Non-Hispanic and more than one race	−.019 (.009)**	.022 (.005)***	.020 (.005)***
Disability	−.091 (.005)***	−.016 (.005)***	−.021 (.004)***
Gifted (“highly capable”)	.129 (.015)***	.082 (.009)***	.077 (.009)***
Migrant	.149 (.023)***	.029 (.008)***	.031 (.008)***
Receives bilingual services	.115 (.013)***	.036 (.006)***	.028 (.006)***
Attended eighth grade in remainder of western Washington	−.055 (.020)***		−.026 (.020)
Attended eighth grade in eastern Washington	.087 (.023)***		.060 (.023)**
Middle school characteristics
Average math WASL (standardized)			−.001 (.021)
Percentage of student body on FRPL			.001 (.000)***
Undergraduate enrollment in a 50-mile radius (standardized)			.040 (.016)**
Has a guidance counselor			.029 (.020)
McFadden’s pseudo R²		.116	.046
Number of observations		114,193	114,669

Note. Standard errors are in parentheses. WASL scores are based on sixth-grade administration for postpolicy cohorts and seventh grade for prepolicy cohorts. “Raw differences” are computed by an ordinary least squares regression of the outcome on the student characteristic (with no other controls). All models have standard errors clustered at the middle school level. Results for taking a modified version of the WASL, taking an out-of-grade-level test, age in eighth grade, being homeless, speaking a language other than English at home, attending public school part-time, and middle school eighth-grade enrollment are available upon request. WASL = Washington Assessment of Student Learning; FRPL = free or reduced-price lunch.

p ≤ .10. **p ≤ .05. ***p ≤ .01 (two tailed).

Some of the groups that are traditionally considered to be “advantaged” in terms of educational attainment are more likely to sign the pledge. For example, as shown in column (1), we find that females, Asian Americans, and students categorized as gifted (“highly capable” in the state’s terms) are 7.3, 9.8, and 12.9 percentage points more likely to sign up, respectively, than males, non-Hispanic Whites, and students not categorized as gifted. Similarly, we find that a one-standard-deviation increase in a student’s math or reading test score is associated with a 5-percentage-point increased probability of signing the pledge.

There are, however, some notable exceptions to the pattern of traditionally advantaged groups being most likely to sign up. For instance, as Table 4 shows, we also find that Hispanics, non-Hispanic African Americans, migrants, and transitional bilingual students are more likely to sign the pledge.²⁵ Similarly, the regional findings are somewhat surprising. As Table 4 shows, students who completed eighth grade in predominantly rural eastern Washington are more likely to sign up than their peers who completed eighth grade in the more urban and suburban Puget Sound region.

Column (2) of Table 4 shows the mean marginal effects corresponding to the parameters estimated in Equation 1a. Similar to column (1), we find that a one-standard-deviation increase in a student’s math test score is associated with a 4.5-percentage-point increase in the probability of signing the pledge, holding constant all other characteristics. Female students are 5.7 percentage points more likely to sign the pledge than otherwise comparable males. Relative to non-Hispanic White students, Hispanics and non-Hispanic African Americans, Asians, and multiracial youth are more likely to sign up, whereas Native American or Alaskan Native youth are less likely to sign the pledge. Net of these student characteristics, the middle school attended clearly affects the propensity to sign up; the null hypothesis that the addition of the school fixed effects is jointly zero is easily rejected (the F statistic is 17).

In column (3) of Table 4, we present the results including middle school characteristics rather than middle school fixed effects, corresponding to Equation 1b. We find higher sign-up likelihoods in schools that have more undergraduates within 50 miles and in schools with more students who are FRPL eligible, controlling for the student’s own characteristics. Interestingly, we find the indicator for a school having a guidance counselor is positive, but not statistically significant. Not surprisingly, the specification with middle school fixed effects (column [2]) has a substantially stronger fit than the model with five school characteristics (column [3]).²⁶

Findings for Research Question 2: How Closely Do the Characteristics That Predict Sign-Ups Parallel Pre-CBS Program Predictors of College Enrollment?

Table 5 presents a parallel analysis for the college enrollment of the prepolicy cohorts, which allows us to examine the extent to which the factors predicting CBS sign-ups mirror factors predicting college enrollment prior to the program’s introduction. Similar to what we found for predicting CBS sign-ups, we observe a general pattern of decreasing magnitude of the coefficients from the model that is not conditioned (column [1]) to those with either middle school fixed effects or characteristics (columns [2] and [3], respectively), with little difference between the latter two specifications. In the first column, we find that the disparities among CBS-eligible students mirror those typically found when examining all students (e.g., females, Asian Americans, and gifted students are substantially more likely to enroll in college “on time”). When estimating the logit specifications, we find a remarkable correspondence between the factors associated with signing the pledge and enrolling in college. The correlation between the marginal effects in the second columns of Tables 4 and 5 (i.e., the models with middle school fixed effects) is 0.75, and the correlation between the marginal effects shown in the third columns of Tables 4 and 5 (i.e., the models with middle school characteristics) is 0.69. These significant correlation coefficients clearly support the first hypothesis (see the Conceptual Model/Theory of Action section) that the factors related to the propensity to go to college also predict the likelihood of signing up for the CBS program.

Table 5

Marginal Effects of Student and School Characteristics on Enrolling in College

	(1)	(2)	(3)
Variable	Raw differences not conditioned on other characteristics	Logit model with middle school fixed effects	Mixed effects logit model with school characteristics
Student characteristics
Math WASL (standardized)	.157 (.003)***	.092 (.004)***	.092 (.004)***
Reading WASL (standardized)	.146 (.003)***	.068 (.003)***	.068 (.003)***
Female	.092 (.004)***	.059 (.003)***	.059 (.003)***
Hispanic	−.032 (.008)***	−.030 (.006)***	−.028 (.005)***
Non-Hispanic African American	−.006 (.008)	.061 (.007)***	.064 (.008)***
Non-Hispanic Asian	.225 (.014)***	.123 (.008)***	.141 (.010)***
Non-Hispanic Hawaiian or Pacific Islander	−.140 (.044)***	−.076 (.063)	−.078 (.048)
Non-Hispanic Native American or Alaskan	−.104 (.012)***	−.069 (.014)***	−.065 (.012)***
Non-Hispanic and more than one race	−.064 (.007)***	−.030 (.007)***	−.030 (.006)***
Disability	−.169 (.004)***	−.072 (.006)***	−.067 (.005)***
Gifted (“highly capable”)	.335 (.014)***	.165 (.010)***	.190 (.012)***
Migrant	−.005 (.013)	.006 (.008)	.009 (.008)
Receives bilingual services	−.031 (.007)***	−.011 (.006)*	−.011 (.006)*
Attended eighth grade in remainder of western WA	−.004 (.009)		.046 (.014)***
Attended eighth grade in eastern Washington	.021 (.011)**		.070 (.014)***
Middle school characteristics
Average math WASL (standardized)			.071 (.025)***
Percentage of student body on FRPL			−.001 (.000)***
Undergraduate enrollment in a 50-mile radius (standardized)			−.021 (.010)**
Has a guidance counselor			.036 (.012)***
McFadden’s pseudo R²		.146	.117
Number of observations		75,785	76,668

Note. Standard errors are in parentheses. WASL scores are based on sixth-grade administration for postpolicy cohorts and seventh grade for prepolicy cohorts. “Raw differences” are computed by an ordinary least squares regression of the outcome on the student characteristic (with no other controls). All models have standard errors clustered at the middle school level. Models are estimated on prepolicy cohorts, and the outcome for enrolling in college considers on-time college enrollment at any college. Results for taking a modified version of the WASL, taking an out-of-grade-level test, age in eighth grade, being homeless, speaking a language other than English at home, attending public school part-time, and middle school eighth-grade enrollment are available upon request. WASL = Washington Assessment of Student Learning; FRPL = free or reduced-price lunch.

p ≤ .10. **p ≤ .05. ***p ≤ .01 (two tailed).

The estimated regression lines based on Equation 3, which models the extent to which the predictions generated from the sign-up models (Equation 2) are predicted by the predictions generated by using the prepolicy college-going regressions combined with the postpolicy student characteristics, are shown in Figure 2. The figure also shows scatterplots of probabilities of pledging and enrolling in college, using a 1 percent random sample of students. Black open circles indicate students in Cohort 1, green solid circles indicate Cohort 2, and red Xs are for Cohort 3. The intercept is highest for Cohort 3, reflecting the general increase in the likelihood of signing the pledge. For all three cohorts, the regression lines have a positive slope indicating that those more likely to enroll in college are more likely to sign the pledge. The predicted probabilities generated by using the prepolicy college-going coefficients are statistically significant predictors of sign-up probabilities, again supporting Hypothesis 1 more formally.

Figure 2.

Relationship between probability of enrolling in college and probability of signing the pledge.

Moreover, the estimated regression lines in the figure are approximately parallel, suggesting little change in this underlying relationship, which argues against Hypothesis 2. Specifically, we might expect a change in the relationship between the factors predicting college going and signing up as the CBS program matures and changes college-going expectations (i.e., we should see a change in the relationship across eligible cohorts). To more formally test Hypothesis 2, we estimate a variant of Equation 3 next, which is a linear model that is fully interacted by cohort. In the equation, C₂ and C₃ are indicators for students in Cohorts 2 and 3.

\begin{array}{l} {\hat{P}}_{P i m} = α_{0} + α_{1} {\hat{P}}_{E i m} + α_{0 C 2} C_{2} + α_{1 C 2} {\hat{P}}_{E i m} C_{2} + α_{0 C 3} C_{3} + \\ α_{1 C 3} {\hat{P}}_{E i m} C_{3} \end{array}

This model enables us to test linear interactions between cohorts and the predicted probability of a post-policy student enrolling in college based on pre-policy enrollment coefficients, ${\hat{P}}_{E i m}$ . These interactions are represented by the coefficients α_1C2 and α_1C3. Both interactions are significant at the .01 level. However, the significant interactions are directionally opposite with the interaction term for Cohort 2 being –.015 and for Cohort 3 being .056. This inconsistent relationship is suggestive of little overall change in the relationship between signing the pledge and enrolling in college. Thus, as was suggested by Figure 2, these findings conflict with the second hypothesis, as the relationship between the factors predicting college going and signing up for the program are even stronger for the third cohort of eligible students than they are for the first cohort of students.

Findings for Research Question 3: What Do Program Administrators Report Doing to Encourage Student Uptake in the Program?

With respect to school-level sign-up rates, Figure 3 shows the relationship between a school’s actual sign-up rate and its predicted sign-up rate for the 605 schools in Washington state that have program-eligible eighth-grade students. A few things merit note here. First, although the actual and predicted rates are correlated, there is substantial deviation (the raw correlation is .40). This result is consistent with our finding earlier that school fixed effects were significant predictors, controlling for student characteristics. Our qualitative analysis allows us to explore explanations for this variation.

Figure 3.

Correspondence of schools’ predicted and actual sign-up rates, by region.

Given the improvement in sign-up rates demonstrated in Figure 2 and the variation among schools in expected versus actual sign-up rates documented in Figure 3, we examine interview data to determine what activities College Bound administrators believe are most important for the program’s success. Regional officers reported that schools’ success in signing up students depends upon (a) district-level “buy-in” and support for the program, coupled with a school-level champion who takes “ownership” of the program; (b) counselors or other school staff having access to FRPL data in order to target individual students who are eligible for the program; (c) guidance counseling staff that are not overburdened with other responsibilities and have good relationships with students; and (d) being in a community or district with a strong college-going culture.

All schools included in the qualitative study reported using the fall FRPL list to identify eligible students, but more than two thirds of guidance counselors interviewed reported using additional means to identify eligible students who were not on the list. Most prominent among these other tactics were sending forms to all students for families to self-identify (40%), giving applications to any new student, and teachers identifying potential students (20% each).

Guidance counselors reported using a variety of approaches to promote the program and encourage sign-ups. The most common activities reported by respondents included individual meetings with students (80%) and parent–teacher conferences (60%); less common were phone calls home (40%), other group approaches (32%), other individual approaches (24%) and classroom approaches (20%); the tactics least commonly reported included schoolwide events (12%) and placing scholarship forms in orientation packets (8%) or FRPL application packets (4%).²⁷ Considering students at their own schools, a principal and a guidance counselor suggested that those likely to sign the pledge were those who are already “looking beyond high school” and “excited about college.”

Guidance counselors have an interest in signing up as many students as possible to ensure they do not miss any eligible students, and there is no significant consequences associated with signing up ineligible students. It is perhaps not surprising that fully half of the guidance counselors interviewed reported signing students up without checking their eligibility. Respondents at more than a third of schools reported sharing applications with all students. Two additional schools previously sent applications to all families but stopped because of complaints by ineligible families or because they did not want to promote “false hope.” One school asks every student to turn in a form and lets WSAC verify the eligibility, whereas at another school, if a parent asks about scholarships, the school requests that the parent sign the form and sends it to WSAC to verify eligibility. Another school signs up every student who ever qualified for FRPL.

Beyond the work of the counselors themselves, another important component for CBS program success—identified both in the literature and by CBS regional officers—is district support for the program. A third of counselor and principal respondents could identify a district-level “champion,” and almost one fifth reported that their district places accountability requirements on schools’ CBS sign-up performance. Of course, this also shows that two-thirds of respondents did not report district accountability requirements related to CBS sign-ups.

Reflecting on the program’s progress and results, respondents were not in close agreement about what works, but they did largely concur about primary challenges to the CBS program’s success. Among the various strategies to support the program, respondents most often identified one-on-one conversations with parents (42%) or students (35%) as the most effective approach. Other strategies considered most effective by at least one individual included classroom approaches, large incentives, persistence, and having a WSAC guest speaker come to the school. The greatest challenge to encouraging sign-ups, identified by more than half of respondents, was getting signed forms back from students. Other challenges included, for instance, unresponsive parents, parents not understanding eligibility requirements, the amount of time required for follow-up, and the fact that many middle school students are not terribly future oriented, so college going is not at the forefront of their minds. For example, one counselor, who said the availability of the program could “light kids on fire,” echoed others in noting the challenge of promoting college going among middle schoolers who often “have no sense of the future.”

A handful of respondents (13%) stated outright that the program had little to no impact on the culture at their school, but most regional officers considered the CBS program to be a success. Yet they also noted areas where it could be improved. In particular, to improve and maintain high sign-up rates, respondents pointed to two opportunities. First, because district leadership and school champions play such important roles, turnover among these positions is a concern. Regional officers emphasized the need for more systematic approaches, whereby all school faculty and staff have a role to play in encouraging program participation and sign-up rates serve as part of a school’s evaluation and accountability systems. Second, echoing the principals and counselors, there was broad agreement that stakeholders need easier access to better and more timely data to identify eligible students, target them for signing up, and support them as they move toward college enrollment.

Regional officers emphasized that program sign-up is just the first step. They promoted more attention to students’ actual uptake of the scholarship and success in college. Yet, maintaining students’ and families’ awareness of the program and encouraging them to adequately plan and prepare for college present a different set of issues from encouraging program sign-up. Similarly, if the goal of the program is college completion, students will require additional supports to help them enroll and persist in college.

Discussion and Conclusion

Early-commitment pledge programs are fairly new, so it is not surprising that we know relatively little about program effects and almost nothing about students’ decisions to sign up. This is an important gap in the literature given that this type of program can help students only if they choose to participate. In this paper, we provide the first evidence of the factors that predict the likelihood of students signing Washington’s CBS pledge.

We document that while the state has made considerable progress in increasing the number of eligible middle school students signing the pledge, sign-up rates were far below 100% in the early years of the program. This finding conflicts with earlier state reports, suggesting near-universal sign-up rates in many districts, which were based on a snapshot of eligible students rather than all eligible students. The difference between these state reports and the proportion of eligible students that we observe to have signed up may be important, as regional officers and guidance counselors working to encourage both CBS sign-ups and college going might have allocated their efforts differently if they received more accurate information about sign-up rates.

That said, although all schools report using FRPL lists to identify eligible students, more than two thirds of the middle school counselors interviewed also report using additional means of identifying eligible students. This suggests that counselors and other program administrators recognized that the FRPL list does not capture all eligible students; in fact, about a fifth of counselors believed that calculated rates were overestimates of program uptake. When asked about the characteristics of students likely to sign up for the CBS, the most common responses offered by principals and guidance counselors were academic strength, a college orientation or expectations, and engaged parents.²⁸

The largest process breakdown identified by program administrators (counselors and principals) was the difficulty in getting signed forms back from students. To account for this, counselors often attempted to engage parents directly rather than via the student. The most common approach to engaging parents was via school conferences (70%). Less common approaches include phone calls (44%), letters home (33%), and including scholarship forms in school orientation or FRPL packets (11%). Nevertheless, the most common approach, conferences, suffers from the fact that not all parents attend conferences and that conferences typically cover many topics. In this regard, students represent a linchpin through which information, communications, and program application paperwork flow.

The quantitative findings are largely in line with interview reports about what drives students to sign up for the College Bound program although not without a few exceptions. Regional officers emphasized the importance of school and district leadership, resources, and climate in supporting sign-ups. Specifically, regional officers agreed that schools are the most critical partner in targeting eligible students, where they highlighted the need for a “champion” to take ownership of the program at the school level and take responsibility for its success there. In addition to the school champion, respondents noted the importance of a strong community or school college-going culture, which may be reflected in the positive and significant coefficient for nearby undergraduate enrollment shown in Table 4.

Somewhat surprising is the lack of statistical significance of guidance counselors in predicting sign-ups, as regional officers had pointed to these individuals as critical to the program’s success, regardless of whether they served as the champion. It is worth noting, however, that these respondents did not consider the mere presence of a counselor to be adequate. Rather, they noted that heavy workloads can leave some counselors struggling to prioritize the program. Similarly, they described variability across counselors in their ability to develop meaningful relationships with students, which they viewed as key to encouraging students to sign up. Finally, they felt that counselors’ access to accurate and timely FRPL data was necessary for success but noted that many did not have such access, a problem echoed by the counselors themselves.

Our quantitative findings show a positive correlation between a school’s predicted sign-up rate and the school’s actual sign-up rates, but the correlation, .40, is not overwhelmingly high. This, combined with the fact that schools have lower sign-up rates than were reflected in public reports, suggests it may be beneficial to target resources toward encouraging student sign-ups at schools with surprisingly low sign-up rates. In doing so, we may consider the tactics suggested by program administrators, who agree that individualized approaches are critical for signing up more students and that good data are necessary for such implementation.

Consistent with our first hypothesis and the qualitative findings about having a “college orientation,” the individual student characteristics that are associated with signing the pledge are closely aligned with the characteristics that predict whether low-income students matriculate to 4-year colleges. High-achieving eligible students, for instance, are both more likely to sign the pledge and more likely to go to college.

In contrast to our second hypothesis, the relationship between the factors predicting college enrollment and signing the pledge did not appreciably change over the first three cohorts of program administration. Significantly more students are signing up for the CBS program over time, but the fact that the underlying relationship between the factors predicting enrollment and pledge signing are similar for the early eligible cohorts calls into question whether the pledge program is changing college expectations significantly for those students with a predictably low propensity to go to college.

The fact that sign-ups are not near to universal is actually somewhat surprising given the low cost to sign up and potentially high financial reward for doing so. One possibility is that the school outreach is too narrowly targeted to students. Middle school students unquestionably play a central role in the sign-up decision. However, given that program administrators identified getting signed forms back from students as the largest process breakdown and that parents/families clearly play a role in shaping students’ college expectations (Kirk, Lewis-Moss, Nilsen, & Colvin, 2011), focusing on family dynamics may prove fruitful. And although some guidance counselors report targeting parents/guardians, this is not an explicit part of the CBS program; encouraging a parent/family outreach component to the program could be considered as a means to reach those students who are least likely now to go to college.

That said, we believe it is premature to jump to strong conclusions about the efficacy of early-commitment programs relative to other approaches to increase college access, given that the ultimate test of the success of the program is whether it is encouraging disadvantaged students to enroll and graduate from college. This is the subject of ongoing work.

Supplemental Material

DS_10.1177_2332858419857703 – Supplemental material for Sign Me Up:: The Factors Predicting Students’ Enrollment in an Early-Commitment Scholarship Program

Supplemental material, DS_10.1177_2332858419857703 for Sign Me Up:: The Factors Predicting Students’ Enrollment in an Early-Commitment Scholarship Program by Dan Goldhaber, Mark C. Long, Ann E. Person, Jordan Rooklyn and Trevor Gratz in AERA Open

Footnotes

Acknowledgements

Generous financial support for this project was provided by the U.S. Department of Education (Grant R305A140380). Excellent research assistance was provided by Nick Huntington-Klein and Bingjie Chen. We thank Vivien Chen for and Alex Johann for help with data questions and structure, and Li Feng, Teresa Greene, Rachelle Sharpe, Lexi Shankster, and audience members at the Association for Education Finance and Policy 2016 conference for helpful comments on an earlier draft of this paper.

Notes

ORCID iD

Mark C. Long

Authors

DAN GOLDHABER is the director of the Center for Analysis of Longitudinal Data in Education Research at the American Institutes for Research and the director of the Center for Education Data and Research at the University of Washington. His work focuses on issues of educational productivity and reform at the K–12 level; the broad array of human capital policies that influence the composition, distribution, and quality of teachers in the workforce; and connections between students’ K–12 experiences and postsecondary outcomes.

MARK C. LONG is the Evans School’s associate dean for faculty affairs and a professor of public policy and governance and adjunct professor of economics. His research examines the effects of public policies on economic opportunity and efficient social mobility, with emphasis on estimating the benefits and costs of those policies. His education-related research focuses on (a) the effects of high school course taking and school and college quality on test scores, educational attainment, labor market outcomes, and family formation and other behaviors; (b) the effects of college financial aid on college entry and household savings; (c) gender disparities in educational attainment; and (d) the effects of affirmative action and alternative college admissions policies on college entry.

ANN E. PERSON works at Mathematica as the director for the Center for Improving Research Evidence and serves as a project director for an evaluation of a grant to a three-state consortium of community colleges under the U.S. Department of Labor’s Trade Adjustment Assistance Community College and Career Training grant program. Her research interests include postsecondary education, employment training programs, college and career readiness, employment, and disability studies.

JORDAN ROOKLYN is affiliated with the Center for Education Data and Research as well as the Evan’s School of Public Policy at the University of Washington. Her research interests include program evaluation, financial aid, college readiness, the value of a statistical life, and regression discontinuity design.

TREVOR GRATZ works at the Center for Education Data and Research at the University of Washington. His research interests include educational outcomes for students with disabilities, career and technical education, K–12 human capital, college affordability, and college readiness.

References

Altonji

J. G.

Blom

Meghir

(2012). Heterogeneity in human capital investments: High school curriculum, college major, and careers. Annual Review of Economics, 4(1), 185–223.

Avery

Kane

T. J.

(2004). Student perceptions of college opportunities: The Boston COACH program. In Hoxby

(Ed.), College choices: The economics of where to go, when to go, and how to pay for it (pp. 355–394). Chicago, IL: University of Chicago Press.

Baker

D. B.

Gratama

C. A.

Ford

Chigizola

Calvery

S. V.

Zamora

(2013). College Bound Scholarship program: Final report. Bothell, WA: BERC Group.

Bartik

T. J.

Hershbein

Lachowska

(2017). The effects of the Kalamazoo Promise Scholarship on college enrollment, persistence, and completion. Working Paper 15-229, W. E. Upjohn Institute for Employment Research, Kalamazoo, MI. https://doi.org/10.17848/wp15-229

Becker

(1993). Human capital: A theoretical and empirical analysis with special reference to education (3rd ed.). Chicago, IL: University of Chicago Press.

Bettinger

Long

B. T.

Oreopoulos

Sanbonmatsu

(2012). The role of application assistance and information in college decisions: Results from the H&R Block FAFSA experiment. Quarterly Journal of Economics, 127(3), 1205–1242.

Blanco

C. D.

(2005). Early commitment financial aid programs: Promises, practices, and policies. Retrieved from https://www.wiche.edu/info/publications/EarlyCommitmentFinancialAid.pdf

Cameron

A. C.

Miller

D. L.

(2015). A practitioner’s guide to cluster-robust inference. Journal of Human Resources, 50(2), 317–372.

Cameron

A. C.

Trivedi

P. K.

(2005). Microeconomics: Methods and applications. New York, NY: Cambridge University Press.

10.

Carruthers

C. K.

Özek

(2016). Losing HOPE: Financial aid and the line between college and work. Economics of Education Review, 53, 1–15.

11.

Castleman

Goodman

(2018). Intensive college counseling and the enrollment and persistence of low income students. Education Finance and Policy, 13(1) 19–41.

12.

Choy

S. P.

(2001). Students whose parents did not go to college: Postsecondary access, persistence, and attainment. In Kridl

Livingston

(Eds.), The condition of education (pp. 18–43). Washington, DC: National Center for Education Statistics.

13.

Cornwell

Mustard

D. B.

Sridhar

D. J.

(2006). The enrollment effects of merit-based financial aid: Evidence from Georgia’s HOPE Program. Journal of Labor Economics, 24(4), 761–786.

14.

Dynarski

(2000). Hope for whom? Financial aid for the middle class and its impact on college attendance. National Tax Journal, 53(3), 629–662.

15.

Dynarski

S. M.

(2004). The new merit aid. In Hoxby

C. M.

(Ed.), College choices: The economics of where to go, when to go, and how to pay for it (pp. 63–97). Chicago, IL: University of Chicago Press.

16.

Dynarski

S. M.

Scott-Clayton

J. E.

(2006). The cost of complexity in federal student aid: Lessons from optimal tax theory and behavioral economics. Working Paper 12227, National Bureau of Economic Research, Cambridge, MA.

17.

Goldhaber

Long

Gratz

Rooklyn

(2019). Pledging to do “good”: An early commitment pledge program, college scholarships, and high school outcomes in Washington state. Working Paper 178, National Center for Analysis of Longitudinal Data in Education Research, Washington, DC.

18.

Hahn

R. D.

Price

(2008). Promise lost: College-qualified students who don’t enroll in college. Washington, DC: Institute for Higher Education Policy.

19.

Hill

N. E.

Wang

M. T.

(2015). From middle school to college: Developing aspirations, promoting engagement, and indirect pathways from parenting to post high school enrollment. Developmental Psychology, 51(2), 224.

20.

Howard

T. C.

(2003). “A tug of war for our minds”: African American high school students’ perceptions of their academic identities and college aspirations. High School Journal, 87(1), 4–17.

21.

Hurwitz

Howell

(2014). Estimating causal impacts of school counselors with regression discontinuity designs. Journal of Counseling & Development, 92(3), 316–327.

22.

Jacob

B. A.

Linkow

T. W.

(2011). Educational expectations and attainment. In Duncan

Murnane

(Eds.), Wither opportunity? Rising inequality, schools, and children’s life chances (pp. 133–164). New York, NY: Russell Sage Foundation.

23.

Kirk

C. M.

Lewis-Moss

R. K.

Nilsen

Colvin

D. Q.

(2011). The role of parent expectations on adolescent educational aspirations. Educational Studies, 37(1), 89–99.

24.

Kirst

M. W.

Venezia

Antonio

A. L.

(2004). What have we learned, and where do we go next? In Kirst

M. W.

Venezia

(Eds.), From high school to college: Improving opportunities for success in postsecondary education (pp. 285–319). San Francisco, CA: Jossey-Bass.

25.

LeGower

Walsh

(2017). Promise scholarship programs as place-making policy: Evidence from school enrollment and housing prices. Journal of Urban Economics, 101, 74-89.

26.

Long

B. T.

(2004). How do financial aid policies affect colleges? The institutional impact of the Georgia HOPE Scholarship. Journal of Human Resources, 39(4), 1045–1066.

27.

Long

B. T.

Riley

(2007). Financial aid: A broken bridge to college access? Harvard Educational Review, 77(1), 39–63.

28.

Long

M. C.

Kennedy

(2015). Optimal spatial distribution of colleges. Working paper, Association for Public Policy Analysis and Management, Washington, DC.

29.

National Center for Education Statistics. (1996). National Education Longitudinal Study: 1988–1994. Descriptive summary report. Washington, DC: U.S. Department of Education.

30.

Oreopoulos

Dunn

(2012). Information and college access: Evidence from a randomized field experiment. Working Paper No. 18551, National Bureau of Economic Research, Cambridge, MA.

31.

Page

L. C.

Iriti

J. E.

Lowry

D. J.

Anthony

A. M.

(in press). The promise of place-based investment in postsecondary access and success: Investigating the impact of the Pittsburgh Promise. Education Finance and Policy.

32.

Perna

L. W.

(2008). Understanding high school students’ willingness to borrow to pay college prices. Research in Higher Education, 49(7), 589–606.

33.

Perna

L. W.

Leigh

E. W.

(2017). Understanding the promise: A typology of state and local college promise programs. Educational Researcher. Advance online publication. https://doi.org/10.3102/0013189X17742653

34.

Power

M. A.

(2011). Case study: Multiple agency collaboration to enroll students in Washington State’s College Bound Scholarship program 2008 through 2011. WERA Educational Journal, 4(1), 13–14.

35.

Rosenbaum

J. E.

(2001). Beyond college for all: Career paths for the forgotten half. New York, NY: Russell Sage Foundation.

36.

Scott-Clayton

(2011). On money and motivation: A quasi-experimental analysis of financial incentives for college achievement. Journal of Human Resources, 46(3), 614–646.

37.

Shankster

Larson

Kwakye

(2017). College Bound postsecondary enrollment: An exploration of students’ understanding and beliefs. Olympia, WA: Washington Student Achievement Council.

38.

Shapiro

Dundar

Huie

Wakhungu

P. K.

Yuan

Nathan

Bhimdiwali

(2017a). Completing college: A national view of student completion rates. Fall 2011 cohort. Signature Report No. 14, National Student Clearinghouse Research Center, Herndon, VA.

39.

Shapiro

Dundar

Huie

Wakhungu

Yuan

Nathan

Hwang

Y. A.

(2017b). Completing college: A national view of student attainment rates by race and ethnicity. Fall 2010 cohort. Signature Report No. 12b, National Student Clearinghouse Research Center, Herndon, VA.

40.

Shapiro

Dundar

Wakhungu

P. K.

Yuan

Nathan

Hwang

(2016). Completing college: A national view of student attainment rates. Fall 2010 cohort. Signature Report No. 12, National Student Clearinghouse Research Center, Herndon, VA.

41.

Sjoquist

D. L.

Winters

J.V.

(2014). Merit aid and post-college retention in the state. Journal of Urban Economics, 80, 39–50.

42.

St. John

E. P.

Fisher

A. S.

Lee

Daun-Barnett

Williams

(2008). Educational opportunity in Indiana: Studies of the Twenty-first Century Scholars Program using state student unit record data systems. Ann Arbor: University of Michigan.

43.

St. John

E. P.

Gross

J. P.

Musoba

G. D.

Chung

A. S.

(2005). A step toward college success: Assessing attainment among Indiana’s Twenty-first Century Scholars. Indianapolis, IN: Lumina Foundation for Education.

44.

St. John

E. P. S.

Weber

(2001). State policy and the affordability of public higher education: The influence of state grants on persistence in Indiana. Research in Higher Education, 42(4), 401–428.

45.

St. John

E. P.

Musoba

G. D.

Simmons

A. B.

(2003). Keeping the promise: The impact of Indiana’s twenty-first century scholars program. Review of Higher Education, 27(1), 103–123.

46.

St. John

E. P. S.

Musoba

G. D.

Simmons

Chung

C. G.

Schmit

Peng

C. Y. J.

(2004). Meeting the access challenge: An examination of Indiana’s Twenty-first Century Scholars Program. Research in Higher Education, 45(8), 829–871.

47.

Thaler

R. H.

Sunstein

C. R.

(2003). Libertarian paternalism. American Economic Review, 93(2), 175–179.

48.

U.S. Department of Education. (2016). Advancing diversity and inclusion in higher education. Washington, DC: Author. Retrieved from http://www2.ed.gov/rschstat/research/pubs/advancing-diversity-inclusion.pdf

49.

Washington Student Achievement Council. (n.d.). About the council. Retrieved from http://www.wsac.wa.gov

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.04 MB