Assessing the Effects of a Community-Based College Preparation Program on the Academic Performance and Noncognitive Factors of Middle School Students

Abstract

This study examines the intermediate effects of a community-based college preparation program in a fast growth, high-needs exurban district in Texas. Participants and a matched group of nonparticipants were compared on a variety of academic and noncognitive measures. Findings indicate program participation is associated with higher scores on the state reading assessment and self-reported expectations of college graduation but do not fully explain the previously documented long-term impact of the program. Using a developmental systems perspective, the authors illustrate interactions between multiple ecological contexts and how this approach can be useful when investigating the efficacy of college preparation programs.

Keywords

college access achievement gap out-of-school time middle school urban education noncognitive skills

Background

Although the overall number of students enrolled in college in the United States has increased substantially over the past 30 years (National Center for Education Statistics [NCES], 2012), low-income, first-generation, Black, and Latino students graduate from high school, are ready for college, enroll in college, and persist and graduate from college at lower rates than other students (Aud et al., 2011; Bureau of Labor Statistics [BLS], 2014; Engle, Bermeo, & O’Brien, 2006; Engle, Yeado, Brusi, & Cruz, 2012; Kena et al., 2014). At the same time, there is a growing recognition that improving educational attainment is a key ingredient for preserving our nation’s democratic ideals of opportunity and advancement and maintaining the global competitiveness of our workforce (Engle et al., 2012; Executive Office of the President, 2014). Addressing this problem is particularly challenging for schools in urban areas (Nagaoka, Roderick, & Coca, 2009), which typically “serve large numbers of poor and historically lower-performing [student groups]” (Mishook, 2012, p. 43).

According to the NCES (Kena et al., 2014), only 34% of 25- to 29-year-olds in the United States have earned a bachelor’s or higher degree, which places the United States 15th in degree attainment internationally (Organization for Economic Co-Operation and Development [OECD], 2011). Between 1990 and 2013, the gap in educational attainment between Whites and Blacks, and Whites and Hispanics actually increased. These attainment gaps can have long-term, profound economic consequences. In 2012, young adults with a bachelor’s degree earned more than twice as much as those without a high school diploma and 57% more than high school graduates. Moreover, the unemployment rate for those with at least a bachelor’s degree was substantially lower than for those with less education (Kena et al., 2014). Recent economic and demographic changes threaten to compound these disparities. In 2012, approximately 21% of school-age children in the United States were in families living in poverty, up from 17% two decades earlier, with significant disparities across racial/ethnic groups (Harvey, Slate, Moore, Barnes, & Martinez-Garcia, 2013; Kena et al., 2014).

Given these socioeconomic, educational, and demographic trends, and the increased emphasis placed on postsecondary education, many community-based organizations have been formed to increase the college-going rate of low-income and minority students. According to the National College Access Program Directory (2013), there are more than 2,500 college access programs dedicated to helping underrepresented middle and high school students aspire to, apply to, enter, and succeed in college. Although there is growing recognition that community-based organizations can be essential partners in the educational achievement of students, especially those with limited social and economic resources (U.S. Department of Education, Office of Planning, Evaluation, and Policy Development, 2010), determining the impact and effectiveness of these programs can be a challenge for both program and school leaders as few have been closely examined via a formal program evaluation or across various contexts (Tierney & Hagedorn, 2007).

College preparation, access, readiness, and success are a major focus of research, policy, and practice (Coles, 2012), with many studies and reports arriving at similar conclusions and making comparable recommendations although often from different perspectives. For example, school counselors and college access program staff often play complementary roles in guiding students toward school success and postsecondary enrollment. School counselors can have a significant positive impact on the college-going rates on their campuses (Hurwitz & Howell, 2013; McDonough, 2006; McKillip, Rawls, & Barry, 2012). This is especially important in schools serving predominately low-income and minority students, where many students and families are more dependent on institutions for information about college (Bryan, Holcomb-McCoy, Moore-Thomas, & Day-Vines, 2009), and where counselor to student ratios are twice the national average (Ramsey, 2008).

Overall, there is general agreement that college access programs and services targeted to underrepresented populations, both school-based and community-based, are most effective when they are comprehensive, include both academic and nonacademic support and enrichment, raise college aspirations and knowledge, incorporate a consistent and supportive relationship with a caring adult, adjusted to meet students’ individual needs, assist students and families in navigating the process, collect and use data to assess progress and inform decisions, include financial aid planning, engage families and peers (Sokatch, 2006), are culturally responsive, provide training to build the capacity of adults, include transition support, exist within a strong college-going culture, are well coordinated with each other, and are sustained over time (Bedsworth, Colby, & Doctor, 2006; College Board National Office for School Counselor Advocacy, 2010; Conley, 2007; Engle et al., 2006; Hooker & Brand, 2009; Lieber, 2009; Root Cause, 2010; Swail, Quinn, Landis, & Fung, 2012; Tierney & Hagedorn, 2007).

There is also a growing body of research linking noncognitive factors, academic performance, and success in school. Social-cognitive theory posits a reciprocal relationship between a person’s “cognitive and emotional processes, environmental and social factors and the behavior or performance of the person” (Alderman, 2013, p. 23). Recent work by the University of Chicago Consortium on School Research (Farrington et al., 2012) highlighted the important roles of noncognitive factors in academic performance, focusing especially on their potential as levers for increasing student educational attainment and providing educators with clear strategies to promote positive academic mind-sets and behaviors in a school and classroom context. The noncognitive factors measured as a part of this study, such as motivation and self-control, were selected because of their previously demonstrated associations with educational outcomes (Eccles, 2009; Eccles & Wigfield, 2001; Gutman & Schoon, 2013; Weiner, 1992) and because of their potential to indicate longitudinal change between program participants and nonparticipants over the course of secondary school.

Program Description

The mission of the community-based organization that is the focus of this study is to provide a path to and through college for students who will be first-generation college graduates. The nonprofit organization started in 2002 and now serves more than 700 students in several school districts in a large metropolitan area in Texas. Approximately 75% of the students in the program are Hispanic, 20% are African American, and 5% are White or of another ethnic origin. More than 95% of the students in the program qualify for free or reduced lunch.

The college preparation program admits a cohort of 60 or more students per district each year during the summer prior to sixth grade and serves them through college graduation. Program components include a combination of out-of-school time programming (Beckett et al., 2009) and comprehensive case management (Dynarski, Clarke, Cobb, Finn, Rumberger, & Smink, 2008).

Out-of-school time is used strategically to expand students’ aspirations, accelerate their learning, and keep them on track for a successful transition to college. Each summer until 10th grade, students in the program attend 5 weeks of intensive full-day summer school taught by high-performing college and high school students from around the country. These summer teaching interns are trained and supervised by experienced mentor teachers in their content areas. During the school year, students are expected to participate in after-school tutoring, homework help and enrichment, as well as four full-day Saturday enrichment programs. In high school, intensive interventions continue but in different configurations. After high school, monitoring and support continue, but the frequency and intensity depend on the student and circumstances.

Each student is also assigned an advisor, who serves as an adult advocate. Advisors play an important role in working with families, teachers, and counselors to guide students to register for rigorous coursework; monitor students’ attendance, behavior, and grades; hold students accountable; support their efforts; provide needed resources; and help them navigate the college application, financial aid, selection, and transition processes. Through their frequent interaction and sustained relationships with students, advisors have opportunities to model positive behavior and goal-oriented decision-making skills such as perseverance and self-advocacy. In addition, advisors continually compile and use information about students’ background, performance, and participation to customize interventions and evaluate effectiveness.

Context

The school district that the college preparation program serves is part of a large metropolitan area in Texas. The community is geographically connected and economically tied to a large urban center. Depending on the field of study, scholars might classify this community in different ways. Economists and public policy analysts might describe the area as exurban. Typically, exurban areas have at least 20% of their workers commuting to jobs in the urbanized area, exhibit low housing density, and have relatively high population growth (Berube, Singer, Wilson, & Frey, 2006). Educational scholars might characterize the community as urban emergent or urban characteristic because the district is located near a large city and faces some of the challenges typically associated with more densely populated urban contexts, such as high rates of poverty and large numbers of English language learners (Milner, 2012). Because of its proximity to the urban center, local newspaper and magazine publications describe the community as a suburban district with students that perform well below the state average.

The district has grown and changed rapidly. In 2007, the district served approximately 5,000 students. By 2013, the district was serving more than 8,000 students, with more than 80% being students of color and 30% being English Language Learners. In addition, there is a high rate of student mobility between districts, with many students transferring back and forth between the adjacent large urban district and the growing outlying districts.

Although suburban and exurban areas are often thought of as being populated by the affluent, such is not the case for the district in this study. In spite of the presence of two high-tech corporations, the infusion of new jobs, and growth in the community, economic conditions for the students and families the school district serves have not improved. In fact, they have gotten worse. In the past 5 years, the percentage of students living in poverty has climbed from 69.5% to more than 80%. While poverty has risen, some measures of academic achievement have declined. In 2012, less than one third of the district’s high school graduates were considered college-ready in both math and English language arts.

Historically, poverty has been associated with inner-city neighborhoods and sparsely populated rural areas. However, for the past three decades, the poor population has grown fastest in the suburbs. As poverty becomes increasingly regional in scope and reach, more people and places are being touched by economic hardship, with serious implications for policy makers and planners, educators, and community leaders (Kneebone & Berube, 2013).

In an unpublished evaluation study (Malerba & Caplovitz, 2011) that examined the first three cohorts of students enrolled in the program, participants were 3.5 times more likely to graduate from high school and 1.98 times more likely to enroll in college than comparison students from the same sixth-grade campuses, after accounting for prior academic performance, attendance, and discipline. These positive results raised important questions regarding the timing and strength of various possible factors associated with program outcomes. This particular study is the first in a series of cohort studies designed to examine factors contributing to the program’s success.

Purpose

The purpose of this study is to examine the intermediate effects of a college access program to better understand its previously documented long-term program effects, which include increased high school graduation and college enrollment rates among participants (Malerba & Caplovitz, 2011). In the present study, program participants and a matched group of nonparticipants were compared on measures of academic achievement, school attendance, and noncognitive factors during the first 2 years of the program.

Specifically, this study attempted to answer the following questions:

Research Question 1: Are program participants significantly different from nonparticipants on demographic (gender, ethnicity, socio-economic status), academic (grades and test scores), attendance, and noncognitive characteristics (motivation, educational, and professional goals) prior to program participation?

Research Question 2: After 2 years of the program, how do participants compare with nonparticipants on academic measures (grades and test scores) after matching for demographic characteristics, prior academic performance, and attendance?

Research Question 3: After 2 years of the program, how do participants compare with nonparticipants on attendance after matching for demographic characteristics, prior academic performance, and attendance?

Research Question 4: After 2 years of the program, how do participants compare with nonparticipants on self-reported noncognitive measures, such as motivation, educational and professional goals, and self-discipline, after matching for demographic characteristics, prior academic performance, and attendance?

Research Question 5: After 2 years of the program, how do participants compare with nonparticipants on teacher-reported noncognitive measures after matching for demographic characteristics, prior academic performance, and attendance?

Research Question 6: After 2 years of the program, how do program participants compare with similar nonparticipants on changes in noncognitive measures?

Theoretical Framework

Different theoretical frameworks have been used to frame issues surrounding college access for low-income students. Savitz-Romer and Bouffard (2012) suggest examining college access and success using a developmental approach. They argue that attention to adolescent development is one of the main things missing from the current conversation about college readiness. They point out that many college preparation programs focus on academic achievement, college knowledge, and financial literacy and support, but neglect strategies to support the development of personal resources and capacities that “build college-going identities, develop positive motivational habits and regulatory skills, and instill agency” (Savitz-Romer & Bouffard, 2012, p. 31). Many college access programs have eligibility requirements and school counselors with high case loads are often able to serve only those students who knock on their doors. Thus, many of the students receiving support are those who already possess the confidence, vision, and planning skills to seek out support, while other youth remain underserved. According to Savitz-Romer and Bouffard (2012), the foundation of a developmental approach to college access rests on three core principles: developmental stages matter, young people can and should be active agents in the process, and going to college is an interconnected process related to social, emotional, and cognitive development as well as social contexts.

Arnold, Lu, and Armstrong (2012) contend that the ecological systems theory or ecological model of Urie Bronfenbrenner (1976, 1994/1993) provides a way of addressing college readiness by “focusing on multiple, integrated interactions of people, organizations, systems, culture, and time” (Arnold et al., 2012, p. ix). Bronfenbrenner’s model is founded on the belief that to understand human development, one must consider all the reciprocal interactions and interrelated ecological systems in which development occurs over time. According to Bronfenbrenner (1994/1993), the model is based on two underlying propositions and consists of five interrelated subsytems: microsystems, mesosystems, exosystems, macrosystems, and chronosystems. Arnold et al. (2012) have urged the adoption of an ecological conception of college readiness as the next step for the field.

For the purposes of this study, we used both the developmental lens suggested by Savitz-Romer and Bouffard (2012) and the ecological model of Bronfenbrenner (1976, 1994/1993). The two approaches are closely connected and aligned. The developmental frame is relevant because we are measuring not only academic performance but also noncognitive factors that are strongly tied to adolescent development and identity. The ecological systems model provides a comprehensive framework and “implies the need to coordinate efforts that affect students across the environments of their lives” (Arnold et al., 2012, p. ix). Reducing systematic inequities in college access and completion is a complex problem imbedded in a myriad of developmental, cultural, economic, social, and political issues.

Research Design and Method

The community-based program considered in this study ultimately measures its success by the number of students who complete high school on time, enroll in college, and eventually earn a college degree. The research methods described in this section were selected to determine if participation in a long-term college access program is associated with measurable intermediate differences in academic achievement, school attendance, and noncognitive factors after 2 years of participation. This is the first in a series of cohort studies that will examine these and additional questions every 2 years through high school and college.

The most rigorous method for examining the impact of participation in a long-term college access program would be to randomly assign participants to treatment or control conditions. Because of the numerous ethical and logistical challenges associated with this approach, we instead used a nonexperimental design in which participating students were compared with nonparticipating students both before and after programmatic intervention.

District personnel were critical partners in the data collection process. Central office administrators compiled all demographic, attendance, and achievement records from district databases and prepared confidential data files to share with the research team according to the terms of a memorandum of understanding and data sharing agreement. Survey data were collected in May. Homeroom teachers (at fifth grade in 2011) and advisory period teachers (at seventh grade in 2013) were asked to distribute opt-out forms to all students to ensure that parents were informed about the nature of the partnership between the district and the community-based program, the purpose of the study, and that their students would be asked to participate in a short questionnaire about their attitudes toward school and their long-term goals.

Although there was no incentive or reward for participation, very few families chose to opt out. Homeroom and advisory teachers were responsible for administering the questionnaires to their students and for delivering them to the central office. In 2013, seventh-grade advisory teachers also were asked to rate each of their students on a variety of noncognitive skills. Classroom participation was highly encouraged by the district’s Director of College Readiness. All completed student and teacher questionnaires were released to the research team for data entry, cleaning, and analysis. Questionnaire participation for program participants was 78% in 2011 and 2013. Questionnaire participation for nonparticipants was 31% in 2011 but increased to 45% in 2013. Teacher ratings of noncognitive skills were received for 64% of participants and 46% of nonparticipants.

Sample

The population for this study includes the 562 students who were enrolled as fifth graders in the district during the 2010-2011 school year. The program participants were the 55 students enrolled in the community-based college preparation program. To be eligible for the program, neither of a student’s parents may have graduated from a 4-year college or university. Students who chose to apply to the program were then selected for participation based on their grades, state assessment performance, regular school attendance, and teacher recommendations. Nonparticipants were the remaining 507 students who were enrolled in fifth grade in 2010-2011. To answer the preparticipation research questions for this study, and to conduct a selectivity analysis, we included all students for whom relevant data were available in fifth grade. To answer the additional research questions, students included in the analyses were restricted to those who were enrolled in the school district in both 2010-2011 and 2012-2013 and to those who had pertinent academic, attendance, and questionnaire data available.

Measures

The measurement of student demographic characteristics, student achievement, and school attendance was a straightforward process for the research team because that information is collected and maintained by the partner school district. However, school districts typically do not formally assess many of the noncognitive factors that have been identified in the literature as associated with success in school. For that reason, the research team obtained permission to use and adapt well-known questionnaires developed as part of the University of Michigan Childhood and Beyond Study (Eccles, 1986-1995), as well as a Brief Self-Control Scale developed by Tangney, Baumeister, and Boone (2004). The Childhood and Beyond Study was a large-scale, longitudinal study that examined children’s achievement and self-perceptions in various domains and the roles that parents and teachers play in socializing those beliefs. Items adapted from the study included those that asked students to report their intrinsic, extrinsic, and moral motivations for doing schoolwork, and the likelihood of achieving various long-term educational goals. The Brief Self-Control Scale focuses on factors related to impulse control that have been correlated with higher grade point averages and better adjustment. Although most items on the student questionnaire administered in 2011 were replicated on the questionnaire in 2013, there were some changes and additions. For example, there was a set of items regarding problem-solving abilities that was not included on the seventh-grade questionnaire because most students rated themselves very highly on those skills. The teacher questionnaire was developed for this study based on a review of the literature regarding noncognitive factors that are associated with success in school. Because the brevity of the questionnaires was important to the district partners, the research team used as few items as possible to assess the factors being studied.

Independent Variables

All independent variables were measured during the 2010-2011 school year.

Program participation;

Gender (male or female);

Ethnicity (African American, Hispanic, White, or Other, including multiple races);

Economic disadvantage (student was approved to receive free or reduced lunch);

State assessment scores in reading and math in fifth grade—scaled scores on Texas Assessment of Knowledge and Skills (TAKS);

Final course grades in reading and math in fifth grade (0%-100% scale);

School enrollment in fifth grade (days enrolled; maximum 175 days);

School attendance rate in fifth grade (days attended divided by days enrolled);

Special education designation in fifth grade;

Limited English proficient (LEP) designation in fifth grade;

First or second year monitored in fifth grade (student in first or second year after exit from LEP status);

Gifted and talented designation in fifth grade.

Dependent Variables

All dependent variables were measured during the 2012-2013 school year.

State assessment scores in reading, math, and writing in seventh grade—scaled scores on State of Texas Assessment of Academic Readiness (STAAR);

Final course grades in reading and math in seventh grade—(0%-100% scale);

School attendance rate in seventh grade (days attended divided by days enrolled);

Motivation for doing schoolwork in seventh grade—student questionnaire adapted from Childhood and Beyond Study (Eccles, 1986-1995). This questionnaire was also administered to fifth graders in 2011.

Educational goals in seventh grade—student questionnaire adapted from Childhood and Beyond Study (Eccles, 1986-1995). This questionnaire was also administered to fifth graders in 2011.

Self-perceptions of noncognitive skills in seventh grade—student questionnaire items developed for this project. This five-item self-report measure was designed to measure students’ own assessment of their level of tenacity with schoolwork, which has been identified in the literature as important for success in school.

Self-discipline in seventh grade—student questionnaire items adapted from Brief Self-Control Scale (Tangney et al., 2004);

Teacher ratings of students’ noncognitive skills in seventh grade—teacher questionnaire developed for this study;

Changes in motivation and educational goals from fifth grade to seventh grade as measured by comparing mean differences.

Analytic Approach

Descriptive statistics were used to compare the demographic, academic, and attendance characteristics of program participants with all other fifth-grade students in the district in 2010-2011. Significance tests were conducted using the chi-square or t tests as appropriate. These analyses allowed us to identify the ways in which program participants were similar to or different from nonparticipants. Because there was evidence of group differences, we confirmed which student characteristics were correlated with program participation and used those variables to construct a comparison group.

The comparison group was created by using the propensity score procedure, pscore, in STATA 12. Students were matched on the independent variables from their fifth-grade year, which were significantly different between the two groups of students before program participation: gender, LEP monitored status, gifted and talented, special education, the number of days enrolled in 2010-2011, and TAKS scores in reading and math. Students also differed on their reading and math grades; however, because those were highly correlated with TAKS performance, they were excluded from the matching procedure.

Intermediate effects were assessed by comparing program participants to matched nonparticipants on seventh-grade academic performance, school attendance, motivation for doing schoolwork, educational goals, self-perceptions, self-discipline, and teacher ratings of noncognitive skills. Most of these comparisons were conducted using t tests. However, because even after the propensity matching procedure there was still evidence of higher rates of state assessment performance among program participants in fifth grade, we used linear regression to examine the association between program participation and seventh-grade STAAR scores while controlling for the students’ own TAKS scores in fifth grade.

Scores on the student motivation and educational goals questions of the student questionnaire were available for a subset of program participants and matched nonparticipants in both fifth and seventh grades, which allowed us to calculate change scores for each item. Mean differences in change from fifth to seventh grade were examined using t tests.

Limitations

The results of this study should be considered in light of the following potential limitations. First, because students elected whether or not to participate in the program, it was not possible to conduct a controlled experiment. This created the potential for self-selection bias. Second, students in the comparison group were matched on demographic, academic, and attendance factors only and differed from program participants on some noncognitive factors. Third, although attempts were made to account for a variety of student factors in fifth grade, some important variables, such as parent, peer, and teacher support, or other program participation, were not measured. Fourth, although the questionnaires used in this study were adapted from other studies or based on a review of the literature, they were not subject to prior validation. Fifth, the size of the participant group was quite small in comparison with nonparticipants and only represented one cohort of the program. Sixth, because questionnaires were only received from a portion of the campuses and students, the possibility of systematic missing data is a limitation. Seventh, because the researchers had no way of accessing data for students who left the district between fifth and seventh grades, student attrition, mobility, and data quality are potential limitations. Finally, although the community-based college preparation program that is the focus of this study uses a variety of research-supported best practices, this study did not directly examine the quality or quantity of program features that students received. Although these limitations must be considered, they do not diminish the importance of the findings or value of the study.

Results

In 2011, data were collected to establish baseline measures of student characteristics on demographic, academic, attendance, and specific noncognitive measures and to determine ways in which program participants were similar to or different from their peers prior to program participation. Table 1 shows the comparison between program participants and all other fifth graders in the district on demographic, academic, and attendance measures. Table 2 shows the correlations among demographic, academic, and attendance measures and program participation.

Table 1.

Preparticipation Demographic and Academic Characteristics of Program Participants and All Other Fifth-Grade Students in the District.

	Program participants (n = 55)		Other students (n = 507)
Student characteristics	Frequency	%	Frequency	%	Significance tests
Demographics
African American	11	20	147	29	χ²(1, N = 562) = 1.99, p = .16
Hispanic	35	64	301	59	χ²(1, N = 562) = .38, p = .54
White	8	14	42	8	χ²(1, N = 562) = 2.40, p = .12
Other	1	2	17	3	χ²(1, N = 562) = .01, p = .94
Female	37	67	230	45	χ²(1, N = 559) = 9.31, p < .01
Male	18	33	274	54
Economic disadvantage	46	84	395	84	χ²(1, N = 524) = 0.01, p = .91
Academics
Gifted and talented	12	22	51	11	χ²(1, N = 522) = 5.51, p < .05
Special education	1	2	48	10	χ²(1, N = 524) = 4.11, p < .05
Limited English proficient	10	18	113	24	χ²(1, N = 524) = .96, p = .33
First or second year monitored	15	27	56	12	χ²(1, N = 524) = 9.88, p < .01
State assessment–TAKS	Scaled score
Reading	733		673		t(469) = 4.60, p < .001
Math	736		664		t(469) = 5.02, p < .001
Final course grades	0-100 scale
Reading	87.9		81.9		t(460) = 5.46, p < .001
Math	87.7		81.9		t(460) = 4.67, p < .001
Enrollment/attendance
Average days enrolled in 2010-2011 (maximum of 175)	173		155		t(303) = 7.25, p < .001
Attendance rate in 2010-2011	97.4		96.6		t(560) = 1.35, ns

Note. The frequencies represent the number of students with each characteristic and may not add up to the total “n” because of missing data. The denominator was adjusted to reflect valid percentages for each indicator. TAKS = Texas Assessment of Knowledge and Skills.

Table 2.

Correlations Between Program Participation and Demographic, Attendance, and Academic Variables in Fifth Grade.

	Program participation	White	Black	Hispanic	Other race/ethnicity	Female	Economic disadvantage	Limited English proficiency	Monitored	Gifted and talented	Special education	Days enrolled	Attendance rate	TAKS reading scale score	TAKS math Scale score	Final reading grade	Final math grade
Program participation	—
White	.066	—
Black	−.059	−.195**	—
Hispanic	.025	−.381**	−.763**	—
Other race/ethnicity	−.003	−.044	−.088*	−.172**	—
Female	.128**	.039	.051	−.072	−.007	—
Economic disadvantage	−.005	−.376**	.022	.225**	−.063	−.007	—
Limited English proficiency	−.044	−.174**	−.333**	.414**	.030	−.056	.180**	—
Monitored	.138**	−.124**	−.232**	.263**	.119**	−.061	.126**	−.220**	—
Gifted and talented	.103*	.113**	−.097*	.010	.087*	−.067	−.147**	−.179**	.196**	—
Special education	−.088*	−.009	.065	−.061	.008	−.141**	−.022	.022	−.108*	−.116**	—
Days enrolled	.125**	.028	−.053	.014	.027	.032	−.072	.044	.084	.064	−.008	—
Attendance rate	.057	−.113**	−.034	.084*	.030	−.010	−.035	.147**	.146**	.095*	−.034	.333**	—
TAKS reading scale score	.208**	.148**	−.076	−.049	.098*	.113*	−.134**	−.391**	.192**	.379**	−.042	.038	.040	—
TAKS math scale score	.227**	.198**	−.235**	.065	.109*	−.030	−.126**	−.219**	.269**	.466**	−.064	.136**	.229**	.597**	—
Final reading grade	.246**	.152**	−.151**	.024	.128**	.198**	−.066	−.229**	.220**	.393**	−.120*	.066	.222**	.636**	.621**	—
Final math grade	.213**	.122**	−.202**	.098*	.099*	.096*	−.014	−.091	.229**	.393**	−.130**	.008	.335**	.492**	.722**	.756**	—

Note. TAKS = Texas Assessment of Knowledge and Skills.

p < .05. **p < .01.

The results indicate that program participants were similar to other fifth-grade students with regard to ethnicity, economic disadvantage, attendance rate, and LEP status. The majority of students in both groups were Hispanic and the majority of students in both groups were economically disadvantaged. Program participants and nonparticipants were also different in many ways. Program participants were more likely to be female and designated as Gifted and Talented. Program participants were enrolled in more days of school and had higher state assessment scores and final course grades in reading and math. In addition, students enrolled in the program were less likely to be in special education and more likely to have exited from LEP status.

Tables 3 and 4 show the results of comparing program participants and nonparticipants in fifth grade, prior to program participation, on students’ motivation for doing schoolwork and future educational goals. Program participants and nonparticipants rated equally their likelihood of attending a 4-year college or attending graduate or professional school. Program participants were more likely than other students to say they do their schoolwork because they “feel bad if it’s not done” and less likely than other students to say they do their schoolwork because “my parents told me I have to.” In addition, program participants were less likely than other students to say they would “drop out of high school” or “work full-time right after high school.”

Table 3.

Average Responses in Fifth Grade Prior to Program Participation to the Question “Circle the Number That Shows WHY You Do Your Schoolwork . . . ?”

	Program participants (n = 44)	Other students (n = 157)
Why you do your schoolwork	M	M	Significance tests
I want to learn new things	5.727	5.819	t(197) = .38, ns
It’s fun and interesting	5.136	4.928	t(195) = .68, ns
I feel bad if it’s not done	5.614	4.707	t(192) = 3.04, p < .01
I think I should	5.977	5.667	t(188) = −1.11, ns
The teacher says I have to	5.095	5.527	t(188) = 1.24, ns
It makes my parents happy	5.791	6.285	t(54) = 1.70, ns
My parents told me I have to	4.227	5.273	t(196)) = 2.68, p < .01

Note. Response scale ranged from 1 (this is not like me at all) to 7 (this is a lot like me).

Table 4.

Average Responses in Fifth Grade Prior to Program Participation to the Question “We Can’t Always Do What We Most Want to Do. How Likely Do You Think Each of the Following Will Be?”

	Program participants (n = 44)	Other students (n = 157)
Outcome	M	M	Significance tests
You will drop out of high school before you finish	1.091	1.484	t(196) = 3.06, p < .01
You will go to work full-time right after high school	4.159	4.949	t(198) = 2.12, p < .05
You will go into the military right after high school	2.070	2.242	t(194) = .51, ns
You will graduate from a 4-year college	6.636	6.420	t(93) = 1.26, ns
You will attend a graduate or professional school after college	5.818	5.378	t(82) = 1.75, ns

Note. Response scale ranged from 1 (very unlikely) to 7 (very likely).

Table 5 shows the demographic, academic, and attendance characteristics of program participants and the comparison group of nonparticipants after the propensity match. The one program participant who was enrolled in special education was dropped from the participant group because it was interfering with the matching procedure. After the matching procedures, program participants still had significantly higher average scores than the comparison group on TAKS math and final course grades in reading and math. These differences were controlled for in subsequent tests of academic achievement by including students’ fifth-grade TAKS scores in reading or math as covariates in regression analyses.

Table 5.

Comparison of Program Participants and Propensity Matched Nonparticipants on Demographic, Academic, and Attendance Variables.

	Program participants (n = 54)	Matched nonparticipants (n = 265)
Student characteristics	Average or percentage	Average or percentage	Student characteristics
Demographics
African American	20%	24%	χ²(1, N = 319) = .43, p = .51
Hispanic	63%	60%	χ²(1, N = 319) = .17, p = .69
White	15%	11%	χ²(1, N = 319) = .66, p = .121
Other	2%	5%	χ²(1, N = 319) = .11, p = .42
Female	67%	63%	χ²(1, n = 319) = .26, p = .61
Male	33%	37%
Economic disadvantage	83%	81%	χ²(1, N = 319) = .14, p = .70
Academics
Gifted and talented	22%	18%	χ²(1, N = 319) = .56, p = .44
SPED	0%	0%	NA, no SPED students in analysis group
Limited English proficient	17%	17%	χ²(1, N = 319) = .04, p = .85
First or second year monitored	28%	20%	χ²(1, N = 319) = 1.62, p = .20
State assessment–TAKS
Reading	733	710	t(317) = 1.89, ns
Math	736	707	t(317) = 2.16, p < .05
Final course grades
Reading	88	85	t(98) = 3.90, p < .001
Math	88	85	t(297) = 2.60, p < .05
Enrollment/attendance
Average days enrolled in 2010-2011 (maximum of 175)	173	173	t(317) = .19, ns
Attendance rate in 2010-2011	97.3%	97.3%	t(317) = .15, ns

Note. SPED = special education; TAKS = Texas Assessment of Knowledge and Skills.

Table 6 shows how program participants and a matching group of nonparticipants compared on grades, test scores, and attendance in seventh grade, after 2 years of the program (Research Question 2). In comparison with the propensity-matched group, program participants had significantly higher state assessment (STAAR) scores in seventh-grade reading and math but no significant differences in final course grades in reading and math. No significant differences were found between the attendance rates of seventh-grade participants and nonparticipants.

Table 6.

Comparison of Seventh-Grade Program Participants and Matching Group of Nonparticipants on Test Scores, Grades, and Attendance.

Measure	Program participants (n = 49)	Matched nonparticipants (n = 190)	Significance tests
	M
Final course grade in reading	86.94	85.67	t(235) = 1.21, ns
Final course grade in math	81.90	82.25	t(236) = .32, ns
	Scaled score
STAAR reading	1,676	1,639	t(246) = 2.39, p < .05
STAAR math	1,670	1,630	t(245) = 2.45, p < .05
	Average daily attendance
Attendance rate	93.6%	93.4%	t(257) = .28, ns

Note. The number of matched nonparticipants varied slightly depending on the indicator. STAAR = State of Texas Assessment of Academic Readiness.

To account for baseline differences in academic achievement between the groups, regression analyses were used to examine the associations between program participation and seventh-grade state assessment scores while controlling for students’ state assessment performance in fifth grade. In this more rigorous analysis, program participation was significantly associated with higher state assessment (STAAR) scores in reading but not in math or writing in seventh grade (Table 7).

Table 7.

Summary of Regression Analyses for Seventh-Grade State Assessment Outcomes.

	Seventh-grade STAAR reading score			Seventh-grade STAAR math score			Seventh-grade STAAR writing score
Variable	B	SE B	β	B	SE B	β	B	SE B	β
Program participation	30.2	12.69	.124**	19.84	10.10	.077	71.37	52.52	.077
Fifth-grade TAKS reading score	.693	.063	.567***				2.125	.262	.460***
Fifth-grade TAKS math score				.861	.048	.740***
R ²	.586			.566			.220
F	F(2, 245) = 64.23			F(2, 244) = 159.15			F(2, 243) = 34.35
	p < .001			p < .001			p < .001

Note. TAKS = Texas Assessment of Knowledge and Skills; STAAR = State of Texas Assessment of Academic Readiness.

p < .01. ***p < .001.

Tables 8 to 11 show how participants and matched nonparticipants compare on self-reported noncognitive measures after matching for demographic characteristics, prior academic performance, and attendance (Research Question 4). Results indicate that seventh-grade program participants and matched nonparticipants reported similar motivation, educational goals, and self-discipline except for a few key areas. Program participants were significantly less likely to report doing their schoolwork because it makes their parents happy, an external motivation. Program participants also were significantly less likely to report they will go into the military right after high school. Although both groups had high average ratings of likely college graduation, program participants had significantly higher ratings on this goal. Surprisingly, program participants were significantly less likely to report that they completed their schoolwork on time. Also surprisingly, program participants were less likely than the comparison group to describe themselves as having a lot of self-discipline. No significant differences were found between participants and matched nonparticipants on the self-control scale.

Table 8.

Comparison of Seventh-Grade Program Participants and Matching Group of Nonparticipants to the Question “Circle the Number That Shows WHY You Do Your Schoolwork . . . ?”

	Program participants (n = 43)	Matched nonparticipants (n = 168)
Why you do your schoolwork	M	M	Significance tests
I want to learn new things	5.023	5.226	t(209) = .76, ns
It’s fun and interesting	3.744	3.982	t(208) = .77, ns
I feel bad if it’s not done	3.977	4.533	t(208) = 1.65, ns
I think I should	4.930	5.357	t(209) = 1.62, ns
The teacher says I have to	4.767	5.120	t(207) = 1.10, ns
It makes my parents happy	4.907	5.554	t(209) = 2.15 p < .05
My parents told me I have to	4.558	5.244	t(209) = 1.97, ns

Note. Response scale ranged from 1 (this is not like me at all) to 7 (this is a lot like me).

Table 9.

Comparison of Seventh-Grade Program Participants and Matching Group of Nonparticipants to the Question “We Can’t Always Do What We Most Want to Do. How Likely Do You Think Each of the Following Will Be?”

	Program participants (n = 44)	Matched nonparticipants (n = 157)
Outcome	M	M	Significance tests
You will drop out of high school before you finish	1.093	1.320	t(210) = 1.51, ns
You will go to work full-time right after high school	3.814	4.118	t(210) = .90, ns
You will go into the military right after high school	1.488	2.036	t(129) = 2.83 p < .05
You will graduate from a technical program or 2-year college	3.605	3.754	t(208) = .42, ns
You will graduate from a 4-year college	6.233	5.574	t(92) = 2.88 p < .05
You will attend a graduate or professional school after college	5.071	4.625	t(208) = 1.37, ns

Note. Response scale ranged from 1 (very unlikely) to 7 (very likely).

Table 10.

Comparison of Seventh-Grade Program Participants and Matching Group of Nonparticipants to Questions Related to Self-Perceptions of Noncognitive Skills.

	Program participants (n = 43)	Matched nonparticipants (n = 168)
Question	M	M	Significance tests
How often did you complete your schoolwork on time?	5.047	5.375	t(81) = 2.01 p < .05
How often were you excited about learning new things in at school?	4.302	4.574	t(210) = 1.08, ns
How often did you stick with your schoolwork even when it was difficult?	4.907	5.000	t(209) = .39, ns
How often did you get along with the other students at school?	5.256	5.349	t(210) = .41, ns
How often did you get along with the adults at school?	5.262	5.186	t(201) = .28, ns

Note. Response scale ranged from 1 (never) to 7 (always).

Table 11.

Comparison of Seventh-Grade Program Participants and Matching Group of Nonparticipants on Student Self-Reported Self-Discipline.

	Program participants (n = 433)	Matched nonparticipants (n = 168)
Question	M	M	Significance tests
I have a hard time breaking bad habits	3.024	3.163	t(206) = .79, ns
I say inappropriate things	3.349	3.554	t(209) = 1.09, ns
I refuse things that are bad for me	3.837	3.842	t(206) = .03, ns
I wish I had more self-discipline	3.349	3.452	t(20) = .52, ns
I am good at resisting temptation	3.674	3.436	t(204) = 1.33, ns
People would say that I have a lot of self-discipline	2.837	3.217	t(207) = 1.93, ns
Pleasure and fun sometimes keep me from getting work done	2.814	2.695	t(205) = .61, ns
I have trouble concentrating	2.930	3.137	t(209) = .99, ns
I am able to work effectively toward long-term goals	3.605	3.750	t(209) = .91, ns
Sometimes I can’t stop myself from doing something, even if I know it is wrong	3.628	3.405	t(209) = 1.15, ns
I often act without thinking through all the alternatives	3.372	3.149	t(209) = 1.08, ns

Note. Response scale ranged from 1 (not like me at all) to 5 (very like me).

Table 12 shows teacher ratings of program participants and matching nonparticipants in seventh grade in 2013 on noncognitive measures related to school success. No significant differences in teacher ratings were detected between the two groups.

Table 12.

Comparison of Teacher Ratings of Seventh-Grade Program Participants and Matching Group of Nonparticipants on Student Noncognitive Skills.

	Program participants (n = 34)	Matched nonparticipants (n = 157)
This student . . .	M	M	Significance tests
Is well organized	4.152	4.108	t(188) = .20, ns
Comes to class prepared for the day’s lesson	4.118	4.025	t(189) = .42, ns
Handles frustration well	4.088	3.949	t(190) = .61, ns
Seeks help when it’s needed	4.364	4.217	t(188) = .80, ns
Is curious about the world	4.258	4.275	t(171) = .10, ns
Is a creative problem solver	4.152	4.119	t(174) = .18, ns
Thinks before he or she acts (not impulsive)	4.212	4.095	t(174) = .54, ns
Shows empathy for others	4.333	4.184	t(183) = .82, ns
Has positive relationships with peers at school	4.471	4.386	t(190) = .57, ns
Has positive relationships with adults at school	4.471	4.323	t(190) = .87, ns

Note. Response scale ranged from 1 (rarely) to 5 (almost all of the time).

Tables 13 and 14 compare how program participants and matched nonparticipants changed between fifth and seventh grade on self-reported scores of student motivation and educational goals.

Table 13.

Changes in Self-Reported Motivation From Fifth Grade to Seventh Grade.

	Program participants (n = 36)	Other students (n = 55)
Why you do your schoolwork	M difference	M difference	Significance tests
I want to learn new things	−0.944	−0.782	t(89) = .44, ns
It’s fun and interesting	−1.694	−0.566	t(87) = 2.56, p < .05
I feel bad if it’s not done	−1.861	0.157	t(85) = 3.68, p < .001
I think I should	−1.229	−0.580	t(83) = 1.42, ns
The teacher says I have to	−0.412	−0.600	t(82) = .29, ns
It makes my parents happy	−0.914	−0.519	t(87) = .89, ns
My parents told me I have to	0.056	0.278	t(88) = .39, ns

Note. Response scale ranged from 1 (this is not like me at all) to 7 (this is a lot like me).

Table 14.

Changes in Self-Reported Educational Goals From Fifth Grade to Seventh Grade.

	Program participants (n = 36)	Other students (n = 55)
Outcome	M difference	M difference	Significance tests
You will drop out of high school before you finish	0.000	−0.278	t(88) = 1.08, ns
You will go to work full-time right after high school	−0.083	−0.963	t(88) = 1.52, ns
You will go into the military right after high school	−0.556	0.038	t(86) = 1.37, ns
You will graduate from a 4-year college	−0.472	−0.782	t(89) = .97, ns
You will attend a graduate or professional school after college	−0.971	−1.000	t(86) = .07, ns

Note. Response scale ranged from 1 (Very unlikely) to 7 (Very likely).

Discussion

The goal of this study was to investigate the intermediate effects of a community-based college preparation program on the academic achievement and noncognitive factors of participating middle school students after 2 years of participation. Results from this study indicate that participation in the program was associated with higher scores on the state assessment in reading and with several self-reported measures of motivation and educational goals. No associations were found related to state assessment scores in math, attendance rates, student self-reported self-discipline, or teacher ratings of noncognitive skills. These findings can be interpreted using both a developmental approach (Savitz-Romer & Bouffard, 2012) and an ecological systems model lens (Arnold et al., 2012).

The first set of results compared program participants and nonparticipants in fifth grade, prior to program participation. Although similar to their peers with regard to most demographic characteristics, a disproportionate number of females enrolled in the program, which supports previous research (Mau & Bikos, 2000) showing that females have higher educational aspirations than males among low-income youth. These findings may reflect important gender and developmental differences (Blackhurst & Auger, 2008) that are worthy of consideration during program recruitment and design.

It is not surprising that program participants were similar to their peers with regard to other demographic characteristics (e.g., ethnicity and economic status) but significantly different than nonparticipants on academic measures such as grades and test scores because those factors were part of the selection process for the program. This finding, though expected, highlights that access to early intervention programs is not available to large numbers of students who might benefit, and may actually weed out many students who are most in need of support. In Bronfenbrenner’s (1994/1993) model, this finding reflects a portion of the exosystem, an organization or structure that can exert influence over an individual student’s experience but that are not a part of a student’s immediate setting or control. For example, school and district administrators frequently face decisions regarding which community-based programs to partner with and which students should be served. At the same time, community-based organizations often have limited slots and must develop programs that demonstrate impact to secure and sustain partnerships as well as financial support. This situation creates an incentive to select students who already demonstrate characteristics associated with going to college, such as higher academic achievement. In this way, external groups play an important role in college preparation but fall short when it comes to serving all students, which is the ultimate responsibility of public schools (Lieber, 2009). These results and issues regarding equitable access were shared with program and district leaders, who responded by expanding opportunities for all fifth-grade students and families to be aware of, apply to, and participate in both school-based and community-based college readiness programs (M. Griffith, personal communication, January 29, 2014).

It is well accepted that intrinsic motivation, doing something because it is inherently interesting or enjoyable, is positively associated with both achievement and task persistence (Gottfried, 1985; Larson, 2000). On average, program participants did not differ from their fifth-grade peers with regard to their intrinsic motivation for doing schoolwork prior to program participation. Both groups reported being moderately motivated by the enjoyment of “learning new things” and because learning is “fun and interesting.”

Interestingly, prior to the program, in comparison with their peers, program participants were significantly more likely to report being motivated to do their schoolwork because they “feel bad if it is not done.” This finding changed after program participation, with participants less likely than nonparticipants to report feeling motivated because they feel bad if their schoolwork is not done. According to Deci and Ryan (1985), genuine enthusiasm for learning and accomplishment involves a sense of self-determination, as opposed to doing something to avoid feeling bad. Psychologist Reed Larson (2000) contends that young adolescents too often are not invested in a positive life trajectory that excites them or originates from within because they have too few opportunities for self-determination or to develop initiative. His research demonstrates how involvement in structured voluntary activities promotes the development of initiative, which he describes as a combination of intrinsic motivation, concerted engagement in the environment, and commitment over time. Because the program studied engages students in challenging and purposeful activities over time, the findings may signal a meaningful shift in participant motivation associated with the program.

Both program participants and nonparticipants reported less motivation to do schoolwork because “it is fun and interesting” in seventh grade compared with fifth grade. These results support prior research that intrinsic motivation for academic work typically declines between Grades 3 and 9 (Gottfried, Fleming, & Gottfried, 2001; Harter, 1981). Reasons for this decline may reflect developmental changes and stages, as well as structural differences between elementary and secondary schools, including increasingly controlling and competitive school environments.

Somewhat surprisingly, when measuring changes in motivation between program participants and a matched group of nonparticipants between fifth and seventh grades, participants had a significantly steeper decline in reporting doing schoolwork because “it is fun and interesting.” Explanations for this comparative decline in intrinsic motivation may reflect something unmeasured about program participants in fifth grade or that program participants now have more externally motivated, long-term goals. In addition, as a part of the program, participants have been exposed to a hands-on summer and Saturday curriculum taught by young near peers. In comparison, the school year program in the district may actually be less “fun and interesting.” From an ecological systems perspective, the stark differences between the summer program and the typical school environment may reflect an incongruent mesosystem, where expectations and experiences across settings are fragmented and unaligned. These findings illustrate yet again the importance of coordinating community-based and school-based programs (Leonard, 2011).

Goals are an important measure because they are associated with the way children approach, engage in, and respond to academic tasks (Hidi & Harackiewicz, 2000). It is noteworthy that among the goals included on the questionnaire given in fifth grade prior to program participation, both groups of students gave the highest likelihood rating to graduating from a 4-year college, with no significant differences between the groups. Interestingly, participants were significantly less likely to report that they would drop out of high school or go to work full-time after high school. These results may reflect a difference between students’ aspirations, referring to their hopes and dreams, and expectations, indicating beliefs about the future (Savitz-Romer & Bouffard, 2012).

Though both program participants and nonparticipants showed a slight drop between fifth and seventh grade in the self-reported likelihood of graduating from a 4-year college, program participants were significantly more likely than nonparticipants to report in seventh grade that they expected to graduate from a 4-year college. This finding is meaningful and reflects the program’s focus on increasing college expectations and knowledge. Some researchers have posited that although nearly all young children aspire to attend college, a decreasing percentage actually expect to attend as they get older (Elliott, 2007; Mickelson, 1990), and expectations can affect goal setting and future performance (Bandura, Barbaranelli, Caprara, & Pastorelli, 2001; Cunningham, Corprew, & Becker, 2009; Eccles & Wigfield, 2002). In addition, “although people’s beliefs about themselves and their future possibilities are internal processes, they are strongly influenced by social contexts and experiences” (Savitz-Romer & Bouffard, 2012, p. 104). Family, peers, teachers, and counselors, as well as the culture of schools, programs, and communities, all play a role in shaping a young person’s view of themselves and their future (Savitz-Romer & Bouffard, 2012).

Importantly, results from this study indicate that after 2 years of program participation, and after controlling for prior test results, program participants scored significantly higher than nonparticipants on state achievement tests in reading, though not in math or writing. These differences may reflect that many of the program’s project-based activities require more reading than math or writing. Because middle school academic achievement and behaviors (Conley, 2007; Wimberly & Noeth, 2005) are important indicators of ultimate success in school, this finding has implications for the types of complementary programs and resources in which schools and districts invest, and the specific interventions and services provided. For example, when program and district leaders reviewed the preliminary results of this study, they decided to adjust a portion of the summer curriculum to provide additional time and support for math and writing (M. Griffith, personal communication, January 29, 2014). From a different perspective, the program’s lack of emphasis on short-term gains on high stakes state tests may be a factor in its long-term positive outcomes (Nichols & Valenzuela, 2013).

It is notable that no significant differences were found on the self-discipline scale or the questionnaires administered to advisory teachers in seventh grade. These null results may reflect issues with the measures, bring into question the role of teacher awareness and expectations, and highlight the importance of the chronosystem, or the role of time, in college preparation and readiness, especially for low-income youth. Because the program being studied has demonstrated positive long-term outcomes, it is possible that some program effects are cumulative and not measurable until later. Although it has been documented that the most successful programs provide a continuum of supports over long periods, the appropriate timing of specific interventions and program activities have received little research attention. The findings may also reveal the growing consensus across several fields that social, emotional, and cognitive skills influence one another and cannot be understood in isolation (Arnold et al., 2012; McKillip, Godfrey, & Rawls, 2012).

Finally, the results of this study point to a larger context. As the rate of suburban poverty grows, so does the need for increased attention and additional assistance to help all students be ready for postsecondary education. Unlike many large urban districts, who frequently have established partnerships with a major city and community-based organizations, suburban and exurban districts generally have fewer partners and a patchy safety net. At the same time, college access organizations have traditionally focused their efforts in large urban centers, where there is a ready supply of labor, public transportation, and a concentration of philanthropic support. Although these challenges are daunting, the smaller size and reduced bureaucracy of some suburban districts may provide opportunities for innovation, focused attention, flexibility, and partnership development that could reap long-term rewards. Community-based organizations may view these districts as opportunities for growth and collaboration where their services are now needed and wanted.

Conclusion

Several important conclusions and implications can be drawn from this study. The results provide evidence of an association between 2 years of program participation and higher levels of achievement in reading and higher expectations of going to college. These findings suggest that the program has a meaningful and measurable influence on the development of students’ academic resources and college-going identity, both of which are essential to make a successful transition to and through college. The null results related to other student achievement variables and noncognitive factors do not explain the previously demonstrated effectiveness of the program and may reflect the challenge of measuring the impact of comprehensive, long-term, and personalized programs by examining discreet program elements. Indeed, these findings may serve as a critique to some of the short-term interventions that are prevalent in many schools today where the desire for immediate and quantifiable positive results may actually discourage the delivery of some programs and services our underserved students and families need the most.

This study also provides an excellent example of how community-based organizations and school districts can use evaluation research to inform decision making. When district and program leaders discussed the results, they took specific steps to enhance coordination, increase access, improve effectiveness, and sustain the partnership. They agreed to continue the longitudinal study of the program, and expressed interest in questions related to program delivery, such as dosage, time, curriculum, instruction, and case management (M. Griffith, personal communication, January 29, 2014). An ongoing question with widespread implications concerns the measurement of the program’s ripple effects on students not in the program. Is there an optimal number or percentage of students in each grade or school who should be served by the program to maximize its overall positive impact and cost-effectiveness?

Finally, results from this study illustrate the importance of examining the topic of college access and readiness through a developmental and ecological framework. Continuing and expanding this longitudinal study to include more systemic and contextual factors, such as interviews with students, parents, teachers, and program staff, may yield explanations for trends and patterns in the data, which can help inform the decisions of all those who share the goal of increasing students’ educational attainment.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Author Biographies

Barry Aidman is an assistant professor of educational leadership at Texas State University. Prior to joining the faculty, he served as a teacher, principal, assistant superintendent, and non-profit executive director. His research interests include leadership preparation and development, multi-sector partnerships, and college access and completion.

Catherine Malerba earned her doctorate in Human Development and Family Sciences at the University of Texas at Austin and currently serves as Director of Research and Evaluation in the Round Rock Independent School District. She has conducted a wide range of research and evaluation projects in PK-12 public school settings. er current interests include out-of-school programs, district technology initiatives, and empowering teachers to evaluate their own work in the classroom.

References

Alderman

M. K.

(2013). Motivation for achievement: Possibilities for teaching and learning (3rd ed.). Florence, KY: Taylor & Francis.

Arnold

K. D.

E. C.

Armstrong

K. J.

(2012). The ecology of college readiness. ASHE Higher Education Report, 38(5), 1-138.

Aud

Hussar

Kena

Bianco

Frohlich

Kemp

Tahan

(2011). The condition of education 2011 (NCES 2011-033). Washington, DC: U.S. Department of Education, National Center for Education Statistics, U.S. Government Printing Office.

Bandura

Barbaranelli

Caprara

G. V.

Pastorelli

(2001). Self-efficacy beliefs as shapers of children’s aspirations and career trajectories. Child Development, 72, 187-206.

Beckett

Borman

Capizzano

Parsley

Ross

Schirm

Taylor

(2009, July). Structuring out-of-school time to improve academic achievement: A practice guide (NCEE 2009-012). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education.

Bedsworth

Colby

Doctor

(2006). Reclaiming the American dream. Boston, MA: The Bridgespan Group.

Berube

Singer

Wilson

J. H.

Frey

W. H.

(2006). Finding exurbia: America’s fast-growing communities at the metropolitan fringe. Washington, DC: The Brookings Institution.

Blackhurst

A. E.

Auger

R. W.

(2008). Precursors to the gender gap in college enrollment: Children’s aspirations and expectations for their futures. Professional School Counseling, 11(3), 149-158.

Bronfenbrenner

(1976). The experimental ecology of education. Educational Researcher, 5(9), 5-15.

10.

Bronfenbrenner

(1994). Ecological models of human development. In International encyclopedia of education (Vol. 3, 2nd ed.). Oxford, UK: Elsevier. (Reprinted in Reading on the development of children, pp. 37-43, by M. Gauvain & M. Cole, Eds., 1993, 2nd ed., New York, NY: Freeman)

11.

Bryan

Holcomb-McCoy

Moore-Thomas

Day-Vines

N. L.

(2009). Who sees the school counselor for college information? A national study. Professional School Counseling, 12, 280-291.

12.

Bureau of Labor Statistics. (2014, April 22). College enrollment and work activity of 2013 high school graduates. Economic News Release. Retrieved from http://www.bls.gov/news.release/hsgec.nr0.htm

13.

Coles

(2012, October). The role of community-based organizations in the college access and success movement. Washington, DC: Institute for Higher Education Policy.

14.

College Board National Office for School Counselor Advocacy. (2010). Eight components of college and career readiness counseling. New York, NY: College Board Advocacy and Policy Center.

15.

Conley

(2007). Toward a more comprehensive conception of college readiness. Eugene, OR: Educational Policy Improvement Center.

16.

Cunningham

Corprew

C. S.

III Becker

J. E.

(2009). Associations of future expectations, negative friends, and academic achievement in high-achieving African American adolescents. Urban Education, 44, 280-296.

17.

Deci

E. L.

Ryan

R. M.

(1985). Intrinsic motivation and self-determination in human behavior. New York, NY: Plenum.

18.

Dynarski

Clarke

Cobb

Finn

Rumberger

Smink

(2008, August). Dropout prevention: A practice guide (NCEE 2008-4025). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education.

19.

Eccles

J. S.

(1986–1995). Childhood and beyond (Murray Research Archive). Retrieved from https://hdl-handle-net.web.bisu.edu.cn/1902.1/01930

20.

Eccles

J. S.

(2009). Who am I and what am I going to do with my life? Personal and collective identities as motivators of action. Educational Psychologist, 44, 78-89.

21.

Eccles

J. S.

Wigfield

(2001). Development of achievement motivation. San Diego, CA: Academic Press.

22.

Eccles

J. S.

Wigfield

(2002). Motivational beliefs, values and goals. Annual Review of Psychology, 53, 109-132.

23.

Elliott

(2007). Specifying children’s educational expectations: The potential impact of institutions (CSD Working Papers. No. 07-17). St. Louis, MO: Center for Social Development, Washington University in St. Louis.

24.

Engle

Bermeo

O’Brien

(2006). Straight from the source: What works for first-generation college students. Washington, DC: The Pell Institute for the Study of Opportunity in Higher Education.

25.

Engle

Yeado

Brusi

Cruz

J. L.

(2012). Replenishing opportunity in America: The 2012 midterm report of public higher education systems in the access to success initiative. Washington, DC: The Education Trust.

26.

Executive Office of the President. (2014). Increasing college opportunity for low-income students: Promising models and a call to action. Washington, DC: The White House.

27.

Farrington

C. A.

Roderick

Allensworth

Nagaoka

Keyes

T. S.

Johnson

D. W.

Beechum

N. O.

(2012). Teaching adolescents to become learners: The role of noncognitive factors in shaping school performance: A critical literature review. Chicago, IL: University of Chicago Consortium on Chicago School Research.

28.

Gottfried

A. E.

(1985). Academic intrinsic motivation in elementary and junior high school students. Journal of Educational Psychology, 77, 631-645.

29.

Gottfried

A. E.

Fleming

J. S.

Gottfried

A. W.

(2001). Continuity of academic intrinsic motivation from childhood through late adolescence: A longitudinal study. Journal of Educational Psychology, 93, 3-13.

30.

Gutman

L. M.

Schoon

(2013). The impact of non-cognitive skills on outcomes for young people: Literature review. London, England: Education Endowment Foundation.

31.

Harter

(1981). A new self-report scale of intrinsic versus extrinsic orientation in the classroom: Motivational and informational components. Developmental Psychology, 17, 300-312.

32.

Harvey

D. W.

Slate

J. R.

Moore

G. W.

Barnes

Martinez-Garcia

(2013). College readiness gaps: A review of the literature. Journal of Education Research, 7, 181-204.

33.

Hidi

Harackiewicz

J. M.

(2000). Motivating the academically unmotivated: A critical issue for the 21st Century. Review of Educational Research, 70, 151-179.

34.

Hooker

Brand

(2009). Success at every step: How 23 programs support youth on the path to college and beyond. Washington, DC: American Youth Policy Forum.

35.

Hurwitz

Howell

(2013, February). Measuring the impact of high school counselors on college enrollment (Research Brief). New York, NY: College Board Advocacy and Policy Center.

36.

Kena

Aud

Johnson

Wang

Zhang

Rathbun

. . . Kristapovich

(2014). The condition of education 2014 (NCES 2014-083). Washington, DC: U.S. Department of Education, National Center for Education Statistics. Retrieved from http://nces.ed.gov/pubs2014/2014083.pdf

37.

Kneebone

Berube

(2013). Confronting suburban poverty in America. Washington, DC: Brookings Institution Press.

38.

Larson

R. W.

(2000). Toward a psychology of positive youth development. American Psychologist, 55, 170-183.

39.

Leonard

(2011). Using Bronfenbrenner’s ecological theory to understand community partnerships: A historical case study of one urban high school. Urban Education, 46, 987-1010.

40.

Lieber

C. M.

(2009). Increasing college access through school based models of postsecondary preparation, planning and support. Cambridge, MA: Educators for Social Responsibility.

41.

Malerba

C. A.

Caplovitz

A. G.

(2011). Breakthrough Austin impact study. Unpublished report. Austin, TX: Technology & Education Consulting Associates.

42.

Mau

W.-C.

Bikos

L. H.

(2000). Educational and vocational aspirations of minority and female students: A longitudinal study. Journal of Counseling & Development, 78, 186-194.

43.

McDonough

(2006). Effective counseling in schools increases college access. Arlington, VA: National Association for College Admission Counseling.

44.

McKillip

M. E.

Godfrey

K. E.

Rawls

(2012). Rules of engagement: Building a college-going culture in an urban school. Urban Education, 48, 529-556.

45.

McKillip

M. E.

Rawls

Barry

(2012). Improving college access: A review of research on the role of high school counselors. Professional School Counseling, 16(1), 49-58.

46.

Mickelson

R. A.

(1990). The attitude-achievement paradox among Black adolescents. Sociology of Education, 63, 44-61.

47.

Milner

H. R.

IV . (2012). But what is urban education? Urban Education, 47, 556-561.

48.

Mishook

(2012). College readiness and smart education systems. Voices in Urban Education, 35, 42-50.

49.

Nagaoka

Roderick

Coca

(2009, January 27). Barriers to college attainment: Lessons from Chicago (The Consortium on Chicago School Research at the University of Chicago). Chicago, IL: Center for American Progress.

50.

National Center for Education Statistics. (2012). Digest of education statistics: 2012. Washington, DC. Retrieved from http://nces.ed.gov/programs/digest/d12/index.asp

51.

National College Access Program Directory. (2013). Retrieved from http://www.collegeaccess.org/accessprogramdirectory/rep_state.aspx

52.

Nichols

S. L.

Valenzuela

(2013). Education policy and youth: Effects of policy on practice. Theory Into Practice, 52, 152-159.

53.

Organization for Economic Co-Operation and Development. (2011). Education at a glance 2011: OECD indicators. OECD Publishing. Retrieved from https://dx-doi-org.web.bisu.edu.cn/10.1787/eag-2011-en

54.

Ramsey

(2008, June). Creating a high school culture of college-going: The case of Washington state achievers (Issue Brief). Washington, DC: Institute for Higher Education Policy.

55.

Root Cause. (2010, September). College access and success (Social Issue Report). Boston, MA: Author. Retrieved from http://www.rootcause.org/resources2/college-access-and-success

56.

Savitz-Romer

Bouffard

S. M.

(2012). Ready, willing, and able: A developmental approach to college access and success. Cambridge, MA: Harvard Education Press.

57.

Sokatch

(2006). Peer influences on the college-going decisions of low socioeconomic status urban youth. Education and Urban Society, 39, 128-146.

58.

Swail

W. S.

Quinn

Landis

Fung

(2012). A blueprint for success: Case studies of successful pre-college outreach programs. Virginia Beach, VA: Educational Policy Institute.

59.

Tangney

J. P.

Baumeister

R. F.

Boone

A. L.

(2004). High self-control predicts good adjustment, less pathology, better grades, and interpersonal success. Journal of Personality, 72, 271-324.

60.

Tierney

W. G.

Hagedorn

L. S.

(2007). Making the grade in college prep: A guide for improving college preparation programs. Los Angeles: The Center for Higher Educational Policy Analysis, Rossier School of Education, University of Southern California.

61.

U.S. Department of Education, Office of Planning, Evaluation, and Policy Development. (2010). ESEA blueprint for reform. Washington, DC: U.S. Department of Education.

62.

Weiner

(1992). Human motivation: Metaphors, theories and research. Newberry Park, CA: SAGE.

63.

Wimberly

G. L.

Noeth

R. J.

(2005). College readiness begins in middle school. Washington, DC: ACT.