The Potential of Advanced Placement to Improve College Outcomes and Narrow Racial/Ethnic and Socioeconomic Disparities

Abstract

The U.S. economy requires a highly educated workforce, yet too few black, Latino, and low-income students attend, persist, and graduate from college. The present study examines the college outcomes of participants in a model Advanced Placement^® (AP) intervention to shed light on its effectiveness and determine whether improving AP participation and performance is a promising strategy for closing persistent racial/ethnic and socioeconomic disparities in college outcomes. Findings suggest the college outcomes of program participants are better than those of similar students statewide while also highlighting variation within and across subgroups. At the same time, they confirm that AP participation and performance predict college outcomes and suggest that improving AP participation and performance among low-income white, black, and Latino students could be a useful strategy for closing persistent racial/ethnic and socioeconomic disparities in college outcomes.

Keywords

advanced placement college outcomes race/ethnicity socioeconomic status

The U.S. economy requires a highly educated workforce. A total of 99% of all new jobs created between January 2010 and January 2016 went to workers with at least some college education (Carnevale et al., 2016). In 2019, the U.S Bureau of Labor Statistics found that 37.8% of all occupations required at least some college for entry. That number is expected to grow by 39.0% by 2029 the U.S. government projects that of jobs will require at least some college by 202 (Employment Projections, 2020). Yet deep and persistent racial/ethnic and socioeconomic disparities in college outcomes threaten the country’s ability to meet its workforce needs.

Educators and policy-makers have argued that more equitably distributing access to and success in the College Board’s Advanced Placement^® (AP) program could help address disparities in college outcomes (Challenge Success, 2013; Lichten, 2010; U.S. Department of Education, 2000). The AP program enables students to take college-level classes and earn college credit and advanced standing while still in high school (College Board, 2020c). The claim that this program could help address disparities in college outcomes is based on evidence suggesting that AP participation and performance are associated with increased rates of college matriculation, persistence, and/or graduation (Ackerman et al., 2013; Geiser & Santelices, 2004; Klopfenstein, 2010; Klopfenstein & Thomas, 2009; Kolluri, 2018; Mattern et al., 2009; Morgan & Klaric, 2007; Sadler & Sonnert, 2010; Speroni, 2011).

What remains unclear is (a) how best to improve black, Latino, and low-income students’ participation and performance in AP and (b) whether expanding black, Latino, and low-income students’ AP participation and performance is a high-leverage strategy to mitigate racial/ethnic and socioeconomic disparities in college outcomes. To shed light on these issues, the present study examines the college outcomes of participants in a model AP program designed to improve college outcomes by improving AP participation and performance, particularly among black, Latino, and low-income students.

Racial/Ethnic and Socioeconomic Disparities in College Outcomes

Deep and persistent racial/ethnic and socioeconomic disparities in college matriculation, persistence, and graduation mean that many students leave school without the credentials they need to succeed in the U.S. economy. Immediate college enrollment rates for the U.S. class of 2016 (the percent of high school completers who are enrolled in a 2- or 4-year institution by the October immediately following their high school graduation) indicate that racial/ethnic matriculation disparities are greatest between Asian and black students, 87% compared with 56%, respectively. Immediate college enrollment disparities also exist between Asian and white and Asian and Latino students, 87% compared with 71% for both groups (McFarland et al., 2018). Data from the 2012/17 Beginning Postsecondary Students Study suggest similar racial/ethnic disparities exist in college persistence and graduation (U.S. Department of Education, 2018). Among students in the class of 2012 who were enrolled in a 4-year institution by October of the same year, 84.0% of Asian students were still enrolled 3 years later compared with 79.6% of white students, 75.1% of Latino students, and 67.5% of black students. Similarly, 76.5% of Asian students enrolling in a 4-year institution received a degree or certificate within 6 years compared with 72.1% of white students, 62.2% of Latino students, and 50.0% of black students.

Disparities among students from different socioeconomic backgrounds are similarly pervasive. Whereas 83% of high-income students in the U.S. class of 2016 enrolled in a 2- or 5-year institution by October 2016, just 56% of low-income students enrolled during the same time period (McFarland et al., 2018). Among students in the class of 2012 who participated in the 2012/17 Beginning Postsecondary Students Study, 90% of high-income students were still enrolled 3 years later compared with 69.6% of low-income students. Similarly, 84.2% of high-income students enrolling in a 4-year institution received a degree or certificate within 6 years compared with just 54.4% of low-income students (U.S. Department of Education, 2018).

In all instances, these differences are statistically significant. Their practical significance is perhaps best evidenced by differences in life outcomes among individuals with varying levels of postsecondary education. According to the U.S. Bureau of Labor Statistics (Torpey, 2018), an individual with a bachelor’s or associate’s degree earned, respectively, $23,972 and $6,448 more in 2017 than an individual with a high school diploma but no college experience. In addition to earning more, individuals with more postsecondary education live longer. In 2017, the average life expectancy of white and black adults with a bachelor’s degree or higher or some college but no 4-year degree was, respectively, 7.97 and 2.71 years greater than the average life expectancy of white and black adults with a high school diploma or less (Sasson & Hayward, 2019). Although these are just two examples of variation in life outcomes associated with varying levels of postsecondary education, they underscore the practical significance of closing racial/ethnic and socioeconomic disparities in college outcomes.

The Promise of Advanced Placement

Policy-makers and practitioners have argued that the College Board’s AP program can help close racial/ethnic and socioeconomic disparities in college outcomes (Challenge Success, 2013; Lichten, 2010; U.S. Department of Education, 2000). But empirical evidence suggests the promise of AP has not yet been fully realized. Although enrolling in an AP course is a clear signal of a student’s intent to attend college (Klopfenstein, 2010), AP course-taking is generally considered a weak predictor of college matriculation, persistence, and grade point average (GPA) once other measures of academic achievement and motivation are controlled (Ackerman et al., 2013; Geiser & Santelices, 2004; Klopfenstein, 2010; Klopfenstein & Thomas, 2009; Kolluri, 2018). However, new evidence from The College Board suggests the number of AP exams a student takes may be a better measure of AP participation than enrollment in an AP course. In one recent study, first-year college GPA and 4-year college completion rates were much higher for students who took one to two AP exams compared with students who took no exams and increased slightly with each successive exam taken up to five (Beard et al., 2019). This finding is generally consistent with prior research examining the relationship between AP exams taken and first-year GPA (Ackerman et al., 2013).

In the literature, AP performance, as measured by a student’s AP exam score, tends to be a much stronger predictor of college outcomes than AP participation. AP exam scores range from 1 to 5, and the College Board has found that scores of 3, 4, or 5 are most closely correlated with the performance of college students in a comparable college course (College Board, 2020a). Students with scores in this range—that is, “qualifying scores”—may earn college credit upon enrollment in an institution of higher education or place out of an introductory-level course (College Board, 2020b). Quasi-experimental research has found that earning a qualifying score is causally related to college matriculation (Speroni, 2011), persistence (Mattern et al., 2009), grades (Mattern et al., 2009; Morgan & Klaric, 2007; Sadler & Sonnert, 2010), and graduation (Ackerman et al., 2013; Burns et al., 2019; Klopfenstein, 2010; Morgan & Klaric, 2007; Speroni, 2011).

Together, these findings suggest that interventions aimed at expanding the number of black, Latino, and low-income students who take multiple AP exams and receive at least one qualifying score could be a fruitful mechanism for closing racial/ethnic and socioeconomic disparities in college outcomes. No studies testing this hypothesis could be located for this review. However, a wealth of evidence indicates that significant racial/ethnic and socioeconomic disparities exist with respect to AP exam participation and the distribution of qualifying scores (College Board, 2014; Malkus, 2016a).

Some of these disparities are related to the process by which students are sorted into classes. In general, high-achieving black, Latino, and low-income students are less likely than white, non-low-income students to be placed in higher level classes (Card & Giuliano, 2015; Gamoran, 2009). Using data from the Council on Great City Schools’ class of 2014, for example, the College Board found that 52.47% of students of color with the potential to succeed on an AP exam, as measured by Preliminary Scholastic Assessment Test (PSAT) scores, did not take an exam in that subject (College Board, 2015).

But racial/ethnic and socioeconomic disparities in the distribution of qualifying scores are also likely to be associated with structural differences between schools that result in inadequate academic preparation for many black, Latino, and low-income students. Consistent with Renbarger and Long (2019), Kolluri (2018) has suggested that poor performance in AP may be the result of poorly delivered content, particularly in schools in low-income neighborhoods, which often struggle to attract and retain high-quality teachers (Clotfelter et al., 2006; Lankford et al., 2002; U.S. Department of Education, Office for Civil Rights, 2014). Course availability also plays a role. A study by the U.S. Department of Education, Office for Civil Rights (2014) found that one quarter of high schools with the highest percentage of black, Latino, Native American, and Alaskan Native students did not offer Algebra II in 2014 and one third did not offer Chemistry—key prerequisites for many AP courses. At the same time, Malkus (2016b) found that high-poverty schools were among the group of schools least likely to offer AP classes between 2000 and 2008.

Examining the Promise of Advanced Placement Through an Intersectional Lens

Regardless of their source, persistent racial/ethnic and socioeconomic disparities in AP participation and performance suggest that AP interventions will be most successful at closing corresponding disparities in college outcomes if they start by closing racial/ethnic and socioeconomic gaps in AP. However, little is known about within-group variation. None of the studies citied in the previous section examined the extent to which socioeconomic status (SES) moderates the relationship between race/ethnicity on one hand and AP participation and performance or college outcomes on the other. Furthermore, most publicly available information on AP exam participation and performance cannot be disaggregated by race/ethnicity and SES simultaneously. Yet, intersectional theorists remind us that we cannot understand an individual’s sociopolitical experience by examining various dimensions of their identity in isolation. We must look at their identities in concert (Crenshaw, 1991) or risk drawing incomplete or incorrect conclusions about within- and across-group similarities and differences (Ovadia, 2001).

The present study aims to address this gap in the literature by using an intersectional lens to examine the long-term impact of a model AP program and evaluate AP’s promise as a strategy for closing persistent, racial/ethnic, and socioeconomic disparities in college outcomes. As described more fully in the “Method” section, analyses use interaction terms to examine multiplicative effects. This is a well-established method for quantitative intersectional analyses (Bowleg, 2008; Dubrow, 2008; Else-Quest & Hyde, 2016; McCall, 2005; Ovadia, 2001). As McCall (2005) explains:

In the language of statistics, the analysis of intersectionality usually requires the use of “interaction effects.” . . . Such models ask not simply about the effect of race on income but how that effect differs for men versus women or for highly educated men versus poorly educated men, and so forth. (pp. 1787–1788)

However, as Cole (2009) makes clear, including statistical interactions among social categories is not, in and of itself, sufficient for an approach to be considered intersectional. Researchers must also consider the meaning and underlying processes of the interactions in the particular social context in which they occur. This issue is addressed more fully in the “Discussion” section; a brief overview of the context for the present study is offered below.

A Model AP Program

This study uses data from Mass Insight Education & Research (Mass Insight)’s AP STEM & English program (STEM = Science, Technology, Engineering, and Mathematics). This program has been heralded by the College Board and others for helping Massachusetts lead the nation in the percent of graduating seniors receiving a qualifying score on at least one AP exam (Massachusetts Department of Elementary and Secondary Education, 2020; M. Williams, 2018). Established through a grant from the National Math and Science Initiative, the AP STEM and English program is based on the Advanced Placement Training and Incentive Program (APIP) but has been modified over time. It is a school-level intervention designed to improve college matriculation, persistence, and graduation, particularly among black, Latino, and low-income students. Since its inception in 2008, the AP STEM & English program has been implemented in nearly 140 schools across Massachusetts, serving more than 45,000 students. Recruitment efforts target schools with large populations of black, Latino, and low-income students or schools with large racial/ethnic and/or SES disparities in AP participation and performance.

Theoretical Framework

Figure 1 illustrates the theoretical framework underpinning the AP STEM & English program. Drawing on research demonstrating a relationship between AP performance on one hand and college outcomes on the other (Ackerman et al., 2013; Geiser & Santelices, 2004; Klopfenstein, 2010; Klopfenstein & Thomas, 2009; Kolluri, 2018; Mattern et al., 2009; Morgan & Klaric, 2007; Sadler & Sonnert, 2010; Speroni, 2011), program activities are designed to improve AP participation and performance, particularly among black, Latino, and low-income students. Inputs fall into three categories: school supports, teacher supports, and student supports.

Figure 1.

Logic model: Mass Insight AP STEM & English program.

School supports

School supports draw on research demonstrating an association between academic achievement and data-driven school improvement planning (Chrispeels et al., 2000; Earl & Katz, 2002). Mass Insight staff share best practices for improving AP participation, including prioritizing black, Latino, and low-income students, establishing open access policies, and setting equity goals (College Board, 2015). They help school administrators set goals, make plans, and track progress over time. Mass Insight also funds equipment and supply purchases, which prior research suggests should improve the fidelity with which schools implement the AP curriculum (Penuel et al., 2007).

Together, school supports—best practices, data-driven school improvement planning, equipment, and supplies—are hypothesized to increase the number of AP sections and courses offered in participating schools. They are also hypothesized to increase their availability to black, Latino, and low-income students. Drawing on College Board (2015) research, these changes are hypothesized to improve AP participation among black, Latino, and low-income students.

Teacher supports

Teacher supports combine high-quality stand-alone workshops with job-embedded professional development and modeling. Drawing on best practices in professional development (Darling-Hammond et al., 2017; Garet et al., 2001; Penuel et al., 2007), teachers new to the AP STEM & English program participate in a 1-week summer institute. At the same time, all teachers are invited to participate in a 2-day fall training designed to facilitate curricular coherence and pedagogical content knowledge through active learning.

Because research suggests isolated professional development workshops are, at best, only modestly associated with improvements in teaching and learning (Hill et al., 2013), teachers also receive periodic, job-embedded professional development. Outstanding former AP teachers observe current AP teachers in action, offer feedback, model effective instruction, and facilitate reflective dialogue. These behaviors are consistent with, although substantially less intensive than, the concept of instructional coaching (Knight, 2006; Neufeld & Roper, 2002; Taylor, 2008), which has been associated with improvements in teacher practice (Kraft et al., 2018; Reinke et al., 2008). Each teacher is expected to receive at least three job-embedded sessions annually.

Finally, teachers are invited to observe Saturday Study Sessions. Described in more detail below, Saturday Study Sessions are led by outstanding AP teachers. While their primary focus is student support, they allow current teachers to see what highly effective AP instruction looks like. This is important because modeling has been shown to improve teachers’ implementation of new strategies in the classroom (Joyce & Showers, 2002; Kinnucan-Welsch et al., 2006).

Together, teacher supports—modeling, job-embedded professional development, and stand-alone workshops—are hypothesized to improve the effectiveness of AP teachers. Because teacher effectiveness strongly predicts student achievement (Chetty et al., 2014; Rivkin et al., 2005; Rockoff, 2004), improvements in teacher effectiveness are hypothesized to predict corresponding improvements in AP performance. This is hypothesized to be particularly true among black, Latino, and low-income students.

Student supports

Drawing on research demonstrating that time-on-task (Cattaneo et al., 2017; Dobbie & Fryer, 2013; Jerrim et al., 2017; Lavy, 2015; Rivkin & Schiman, 2015) and teaching effectiveness (Chetty et al., 2014; Rivkin et al., 2005; Rockoff, 2004) predict student achievement, student supports include two Saturday Study Sessions or nine additional hours of AP instruction facilitated by outstanding educators. Because practice tests can improve both learning and test performance (Kulik et al., 1984; Rawson & Dunlosky, 2012), student supports also include a 4.5-hr mock exam that is scored and returned to students and teachers midway through the year. Finally, low-income students receive exam-fee subsidies, which have been shown to improve AP exam participation in this population (Jeong, 2009). Together, student supports—Saturday study sessions, mock exams, and exam fee subsidies—are hypothesized to improve the participation and performance of black, Latino, and low-income students in AP.

Duration

The typical school receives the full array of school, teacher, and student supports for 3 years. Some schools elect to continue these supports for a fourth, fifth, or even a sixth year, while most transition to a less intensive program where they choose which supports to receive. The vast majority of schools choose to continue teacher professional development workshops and student supports and participate in the less intensive program for 2 to 4 additional years.

Effectiveness

Observational research suggests that Mass Insight’s AP STEM & English program is reaching schools with large populations of black, Latino, and low-income students and is able to improve the quality of AP programs at participating schools (Challenge Success, 2013). To date, the program has been implemented in approximately 70% of Massachusetts schools with the most black, Latino, and “high needs”¹ students, and the most recent quasi-experimental, third-party evaluation concludes that participating schools experience a statistically significant increase in the proportion of students participating in the AP program, taking an AP exam, and earning a qualifying score (UMass Donahue Institute, 2017a, 2017b). After 2 years, the proportion of students taking an AP course is 4.7 percentage points higher in intervention schools than comparison group schools. Similarly, after 1 year, the proportion of students taking an AP exam and receiving a qualifying score is, respectively, 6.6 and 2.0 percentage points higher in intervention schools than comparison group schools.

The Present Study

The present study aims to examine the long-term outcomes of Mass Insight’s AP STEM & English program and evaluate AP’s promise as a strategy for closing persistent racial/ethnic and SES disparities in college outcomes. It addresses two gaps in the literature. First, it adds to the field’s understanding of racial/ethnic and SES disparities in college outcomes by examining these disparities from an intersectional, rather than an independent or additive perspective. Second, it offers the field new information about the potential impact of closing racial/ethnic and SES disparities in AP participation and performance on corresponding disparities in college outcomes.

As noted above, Mass Insight’s AP STEM & English program was designed to improve college attendance, persistence, and graduation, particularly among black, Latino, and low-income students. It was not designed to close gaps. Yet, program data can be used to simulate the impact of closing disparities in AP participation and performance on corresponding disparities in college outcomes. Research questions include the following:

Research Question 1 (RQ1): What are the college matriculation, persistence, and graduation rates of program participants by race/ethnicity and SES?

Research Question 2 (RQ2): How do these rates compare with statewide rates by race/ethnicity and SES?

Research Question 3 (RQ3): How do race/ethnicity and SES interact to predict college matriculation, persistence, and graduation among program participants graduating from the same high school?

Research Question 4 (RQ4): If programs like the AP STEM & English program were able to eliminate racial/ethnic and SES disparities in AP participation and performance, how would corresponding disparities in college outcomes be affected among students graduating from the same high school?

Method

Participants

The sampling universe included all students taking an AP exam and graduating from a high school participating in the AP STEM & English program between 2009 and 2013 (N = 14,171). Students who could not be correctly linked across data sets (n = 1,570) were omitted along with students who met inclusion criteria but were missing SES data or identified as a race/ethnicity other than white, black, Latino, or Asian (n = 1,303). Students who identified as American Indian/Alaskan Native as well as students who identified as Native Hawaiian/Other Pacific Islander were omitted because they comprised a very small proportion of the sample (n = 48 and 652 students, respectively). Similarly, multiracial students were omitted because the sample size was very small (n = 375) and a full analysis of these students’ experience would require even further segmentation (e.g., students who identified as black and white compared with students who identified as black and Asian). An additional 264 students who did not report their race/ethnicity were also omitted from analyses along with 105 students whose high school did not provide their SES.

Of the 11,190 unique students meeting inclusion criteria, 67.73% identified as white, 11.72% identified as black, 10.20% identified as Latino, and 10.35% identified as Asian. Slightly more than one third (34.38%) were identified as low-income, just over 40% (41.23%) identified as male, and 58.77% identified as female. These students were drawn from 64 high schools that had been partnering with the AP STEM & English program for 2.82 years by the time study participants graduated from high school (SD = 1.27 years, range = 1–5).

Measures

Analyses merged AP STEM & English administrative records with data from The College Board and The National Student Clearinghouse. Dependent variables included college matriculation, persistence, and graduation. These variables were dichotomous and calculated from National Student Clearinghouse data:

Matriculation – A student was defined as matriculating if the student enrolled in a postsecondary institution by the October after graduating from high school (i.e., enrolled in college immediately).

Persistence—A student was defined as persisting if the student enrolled in a postsecondary institution for at least 2 years. A student did not have to persist at the same institution at which they initially enrolled to be classified as persisting.

Graduation—A student was defined as graduating if the student received a degree from a postsecondary institution within 150% of normal time—that is, 6 years for a student matriculating at a 4-year institution. A student did not have to graduate from the same institution at which they initially enrolled to be classified as graduating.

Independent variables captured student race/ethnicity, SES, AP participation, and AP performance:

Race/Ethnicity—Race/ethnicity was self-reported by students to the College Board. A student was identified as Latino if the student identified as Hispanic or Latino. A student was identified as black, Asian, or white if the student did not identify as Hispanic or Latino and identified as solely black, Asian, or white, respectively.

SES—Eligibility for Free or Reduced Price Lunch was used as a proxy for SES. It was drawn from AP STEM & English administrative records and originally provided by school partners. Students who qualified were classified as low-income; students who did not qualify were classified as non-low-income. Although eligibility for Free or Reduced Price Lunch is a time-varying covariate and schools provided updated information on participating students annually, only the most recent student record was retained in AP STEM & English administrative records.

AP Participation—AP participation was calculated from College Board data. It re-coded the number of AP exams a student took over the course of their high school career into three categories—one, two, and three or more exams. AP participation was treated as a categorical variable in analyses.

AP Performance—AP performance was also calculated from College Board data. It was a students’ average AP exam score, rounded to the nearest integer. Like AP participation, it was treated as a categorical variable in analyses.

Analyses

College outcomes among program participants (RQs 1 and 2)

Descriptive and bivariate analyses examined college matriculation, persistence, and graduation rates for the entire sample as well as a subsample of participants graduating from high school in 2011, the last year for which statewide data are publicly available. Whereas the first set of analyses examined the proportion of students who enrolled in a 4-year institution and, of those students, the proportion who ultimately persisted and graduated, the second set of analyses examined the proportion of students who enrolled in a 2- or 4-year institution and, of those students, the proportion who ultimately persisted and graduated. This is because statewide results cannot be disaggregated by postsecondary institution type, but the broader literature on AP has focused almost exclusively on the experience of AP students in 4-year institutions. In all analyses, the denominator for matriculation rates was the total number of students in the sample. For persistence and graduation rates, the denominator was the total number of students who matriculated at a 2- and/or 4-year institution.

The interaction of race/ethnicity and SES (RQ 3)

Descriptive analyses examined the relationship between race/ethnicity and SES. Multivariate analyses used Equation 1 to partial out the relationship among race/ethnicity, SES, and college outcomes, although as Gillborn and colleagues (2018, p. 20) note, factors such as SES and prior attainment can never be “entirely independent of racist influences”:

\begin{array}{l} l o g (Y_{j t}) = β_{0} + β_{1} B l a c k_{j t} + β_{2} L a t i n o_{j t} + β_{3} A s i a n_{j t} + β_{4} L o w - I n c o m e_{j t} \\ \begin{array}{l} + β_{5} B l a c k X L o w - I n c o m e_{j t} + β_{6} L a t i n o X L o w - I n c o m e_{j t} \\ + β_{7} A s i a n X L o w - I n c o m e_{j t} + μ_{t} + e_{j} \end{array} \end{array}

(1)

where,

Y is the log odds that student j who attended high school t matriculated at, persisted in, or graduated from a 4-year institution.

$β_{0}$ is the log odds of matriculating at, persisting in, or graduating from a 4-year institution when all other variables in the model were zero—for example, for a non-low-income white student.

$β_{1 - 3}$ is the additional variation in the log odds of Y among non-low-income black, Latino, and Asian students.

$β_{4}$ is the additional variation in the log odds of Y among low-income white students.

$β_{5 - 7}$ is the additional variation in the log odds of Y among black, Latino, and Asian students who are also low income.

µ_t is a school-level fixed effect, included to account for the nonindependence of observations from students attending the same high school.

$e$ is the error term, capturing everything omitted from the model that predicts the log odds of Y.

Models predicting matriculation at a 4-year institution included all students in the sample. Models predicting persistence and graduation were limited to students who enrolled at a 4-year institution. This included students who enrolled in a 4-year institution by the October after their high school graduation (i.e., students who enrolled immediately) as well as students who enrolled later.

Preliminary analyses (not shown) evaluated the extent to which proposed models met the assumptions of logistic regression. In all cases, a specification link test for single-equation models was not significant, p > .10, suggesting that the true conditional probabilities were a logistic function of the independent variables and the independent variables were specified correctly (Pregibon, 1980). There were no variance inflation factors (VIF) above 5 (Hair et al., 2010), suggesting multicollinearity was not problematic. Because multilevel null models indicated that 13.47% and 7.84% of the variation in 4-year enrollment and graduation, respectively, occurred between-schools, a school-level fixed effect was included in Equation 1 to account for nonindependence among observations from students attending the same high school (Bliese, 2000).² Analyses of Pearson residuals, deviance residuals, and Pregibon leverage suggested influential observations were not biasing results (Sarkar et al., 2011). Finally, McFadden’s R² statistics were in the range of .01–.05 in models omitting school fixed effects, suggesting that the independent variables included in Equation 1 were not strong predictors of matriculation, persistence, or graduation rates. Notably, McFadden’s R² statistics were larger in models including school fixed effects, .04–.14, but still far below the accepted threshold of .20–.40 (McFadden, 1977).

Controlling for AP participation and success (RQ4)

To examine the effect of eliminating racial/ethnic and SES disparities in AP participation and performance on college outcomes, descriptive analyses examined the extent of these disparities among study participants. Next, variables capturing AP participation and performance were added to Equation 1 and models predicting college matriculation, persistence, and graduation were re-estimated. Y-standardized results were used to examine how the coefficients associated with race/ethnicity, SES, and their interaction changed in re-estimated models. Because the variance of the log odds of the dependent variable changes when additional variables are added to a model, y-standardizing independent variables is one way to ensure coefficients in nested models are measured using the same metric—in this case, the standard deviation change in the log odds of the dependent variable associated with a 1-unit increase in a given independent variable (Long & Freese, 2006; R. Williams & Wang, 2019).

Using the y-standardized coefficients, the log odds of matriculating at, persisting in, and graduating from a 4-year institution for low-income and non-low-income black, Latino, and Asian students and low-income white students were calculated and compared with those for non-low-income white students. Non-low-income white students were used as the reference group for these analyses because it is generally agreed that these students occupy the most privileged social position in the United States. Analyses related to RQ3 also indicated that, where differences were statistically significant, this group experienced the best outcomes.

Results

College Outcomes Among Program Participants Compared With Statewide Results

Table 1 presents the college outcomes of AP STEM & English participants. It indicates that nearly three quarters of participants (73.15%) enrolled in a 4-year institution by the October after they graduated from high school. Nearly all participants who enrolled in a 4-year institution (98.09%) persisted for at least 2 years. The vast majority (77.78%) of participants who enrolled in a 4-year institution received a degree within 150% of normal time.

Table 1.

Outcomes for AP STEM & English Participants by Race/Ethnicity and SES.

Key variable of interest		Matriculate (%)	Persist (%)	Graduate (%)
All		73.15	98.09	77.78
Race/Ethnicity	White	75.64	98.01	80.76
	Black	68.06	98.32	67.97
	Latino	61.09	97.27	67.00
	Asian	74.53	99.07	76.83
SES	Non-low-income	76.94	97.95	81.22
SES	Low-Income	65.92	98.16	70.11

Note. The denominator for persistence and graduation rates is the number of students who matriculated at a 4-year institution by the October following their high school graduation. AP = Advanced Placement; STEM = Science, Technology, Engineering, and Mathematics; SES = socioeconomic status.

Racial/ethnic and SES disparities in matriculation and graduation were large, with white, Asian, and non-low-income participants matriculating and graduating at higher rates than black, Latino, and low-income participants. Chi-square tests confirmed the statistical significance of distributional difference by race and ethnicity for matriculation and graduation but not persistence, matriculation: χ²(3, N = 11,190) =126.93, p < .001, graduation, χ²(3, N = 8,186) = 126.47, p < .001, and persistence: χ²(3, N = 8,186) = 7.38, p > .05. Trends were similar for SES: matriculation, SES, χ²(1, N = 11,190) = 156.18, p < .001, graduation, SES, χ²(1, N = 8,186) = 125.02, p < .001, and persistence, χ²(1, N = 8,186) = 0.41, p > .05.

Analyses restricting the sample to students from the class of 2011 and including students who matriculated at both 2- and 4-year institutions revealed similar trends (Table 2) and suggested that the college outcomes of AP STEM & English program participants were much better than the college outcomes of Massachusetts students statewide. Differences were particularly large among black, Latino, and low-income students. For these groups, rates of matriculation, persistence, and graduation were 18.37 to 29.56 percentage points higher among AP STEM & English participants than students from similar racial/ethnic and SES backgrounds statewide.

Table 2.

Outcomes for the High School Class of 2011 by Race/Ethnicity, SES, and AP Performance.

Key variable of interest		AP STEM & English participants			All Massachusetts students
Key variable of interest		Matriculate (%)	Persist (%)	Graduate (%)	Matriculate (%)	Persist (%)	Graduate (%)
Race/Ethnicity	White	83.67	96.56	81.05	71.92	87.68	74.59
	Black	81.58	96.37	65.73	62.26	78.00	45.32
	Latino	74.43	96.32	71.78	53.06	72.03	42.22
	Asian	80.89	98.90	77.47	76.84	91.28	73.74
SES	Non-low-income	83.76	96.79	81.82	—	—	—
SES	Low-Income	79.49	96.62	70.35	55.65	75.06	46.24

Note. The denominator for persistence and graduation rates is the number of students who matriculated at a 2- or 4-year institution by the October following their high school graduation. Statewide averages are calculated from District Analysis and Review Tools (DART) detail: Success After High School, a publicly available report from the Massachusetts Department of Elementary and Secondary Education (2019). Data on non-low-income students in Massachusetts are not available. Because statewide averages include students matriculating at both 2- and 4-year institutions, AP STEM & English averages do the same. SES = socioeconomic status; AP = Advanced Placement; STEM = Science, Technology, Engineering, and Mathematics.

The Interaction of Race, Ethnicity, and SES

As illustrated in Table 3, there was a strong and highly significant relationship between race/ethnicity and SES among study participants, χ²(3, N = 11,190) = .003, p < .001. Whereas 82.09% of white participants came from non-low-income backgrounds, more than 65.00% of black, Latino, and Asian participants came from low-income backgrounds. This strong bivariate relationship is one reason multivariate models were required to tease out the independent relationship between race/ethnicity, SES, and college outcomes.

Table 3.

Race/Ethnicity by SES Among AP STEM & English Participants.

Race/ethnicity	SES
Race/ethnicity	Low-income (%)	Non-low-income (%)
White	17.91	82.09
Black	65.53	34.47
Latino	70.26	29.74
Asian	66.71	33.29

Note. SES = socioeconomic status; AP = Advanced Placement; STEM = Science, Technology, Engineering, and Mathematics.

Tables 4 –6 present y-standardized multivariate results as well as odds ratios from Equation 1. Because odds ratios may not be intuitive for some readers, STATA’s postestimation margins command was also used to predict the probability of each outcome for the average student from each racial/ethnic and SES combination using the coefficients from Tables 4 –6, Model 1. These probabilities are presented in Figure 2 and referenced throughout the remainder of the results section alongside odds ratios from Tables 4 –6. The former are not comparable with the descriptive results presented previously.

Table 4.

Predicting College Enrollment by Race/Ethnicity, SES, AP Participation, and AP Performance.

	(1)					(2)
Covariates	β (StdY)	z	ρ	OR	SE (OR)	β (StdY)	z	ρ	OR	SE (OR)
Black	−0.25	−4.00	***	0.62	0.07	−0.20	−3.27	**	0.67	0.08
Latino	−0.29	−4.40	***	0.58	0.07	−0.23	−3.62	***	0.63	0.08
Asian	−0.19	−2.94	**	0.69	0.09	−0.25	−3.82	***	0.61	0.08
Low-income	−0.33	−8.85	***	0.53	0.04	−0.29	−7.85	***	0.56	0.04
Black × Low-income	0.35	4.49	***	1.96	0.29	0.37	4.81	***	2.08	0.32
Latino × Low-income	0.15	1.93		1.35	0.21	0.15	1.86		1.34	0.21
Asian × Low-income	0.43	5.14	***	2.28	0.37	0.45	5.43	***	2.44	0.40
Average Exam Score = 2						0.14	4.78	***	1.33	0.08
Average Exam Score = 3						0.24	6.57	***	1.60	0.11
Average Exam Score = 4						0.27	5.79	***	1.70	0.15
Average Exam Score = 5						0.19	2.65	**	1.46	0.21
Two Exams						0.22	7.28	***	1.53	0.09
Three + Exams Taken						0.42	13.48	***	2.30	0.14
McFadden’s R²	.07					.10
N	11,190					11,190

Note. The omitted group is white students from non-low-income backgrounds. In Model 2, this group also has an average AP score of 1 and took one AP exam. School-level fixed effects are omitted for brevity but are available upon request. SES = socioeconomic status; AP = Advanced Placement; OR = odds ratio.

*ρ

< .05. **ρ < .01. ***ρ < .001.

Table 5.

Predicting College Persistence by Race/Ethnicity, SES, AP Participation, and AP Performance.

	(1)					(2)
Covariates	β (StdY)	z	ρ	OR	SE (OR)	β (StdY)	z	ρ	OR	SE (OR)
Black	0.06	0.28		1.13	0.48	0.10	0.46		1.22	0.53
Latino	−0.36	−2.03	*	0.50	0.17	−0.33	−1.92		0.52	0.18
Asian	0.30	1.07		1.76	0.93	0.25	0.91		1.62	0.86
Low-income	−0.16	−1.34		0.74	0.17	−0.15	−1.27		0.75	0.17
Black × Low-income	0.08	0.30		1.16	0.58	0.09	0.34		1.18	0.60
Latino × Low-income	0.45	1.85		2.35	1.09	0.45	1.88		2.40	1.11
Asian × Low-income	0.07	0.22		1.15	0.73	0.10	0.29		1.20	0.77
Average Exam Score = 2						0.16	1.64		1.36	0.25
Average Exam Score = 3						0.22	1.90		1.52	0.33
Average Exam Score = 4						0.30	2.04	*	1.81	0.53
Average Exam Score = 5						0.09	0.42		1.19	0.48
Two Exams						0.10	1.01		1.20	0.22
Three + Exams Taken						0.23	2.26	*	1.55	0.30
McFadden’s R²	.04					.05
n	8,468					8,468

ρ < .05. **ρ < .01. ***ρ < .001.

and *** are statistical significance stars.

Table 6.

Predicting College Graduation by Race/Ethnicity, SES, AP Participation, and AP Performance.

	(1)					(2)
Covariates	β (StdY)	z	ρ	OR	SE (OR)	β (StdY)	z	ρ	OR	SE (OR)
Black	−0.29	−3.80	***	0.58	0.08	−0.23	−3.06	**	0.64	0.09
Latino	−0.37	−4.75	***	0.50	0.07	−0.33	−4.32	***	0.53	0.08
Asian	−0.17	−2.15	*	0.73	0.11	−0.20	−2.65		0.68**	0.10
Low-income	−0.27	−5.85	***	0.60	0.05	−0.25	−5.37	***	0.62	0.06
Black × Low-income	0.23	2.51	*	1.56	0.27	0.25	2.69	**	1.62	0.29
Latino × Low-income	0.29	2.97	**	1.74	0.32	0.30	3.12	**	1.80	0.34
Asian × Low-income	0.26	2.63	**	1.63	0.30	0.28	2.89	**	1.72	0.32
Average Exam Score = 2						0.22	5.91	***	1.52	0.11
Average Exam Score = 3						0.34	7.93	***	1.91	0.16
Average Exam Score = 4						0.42	7.85	***	2.27	0.24
Average Exam Score = 5						0.56	5.93	***	2.95	0.54
Two Exams						0.08	2.31	*	1.17	0.08
Three + Exams Taken						0.21	5.67	***	1.49	0.11
McFadden’s R²	.05					.07
N	8,835					8,835

*ρ

< .05. **nb ρ < .01. ***ρ < .001.

Figure 2.

Predicted probability of college enrollment, persistence, and graduation among AP STEM & English participants by race/ethnicity and SES.

Analyses demonstrate that, among study participants, the relationship between race/ethnicity on one hand and college matriculation and graduation on the other varied significantly by SES. In general, white and Asian non-low-income participants tended to have better outcomes than non-low-income black and Latino students. The reverse was true among low-income participants where Asian and black students often had better outcomes than white students. Furthermore, whereas SES was often associated with large, statistically significant differences in college outcomes among white participants, there appeared to be much less within-group SES variation in college outcomes among black, Asian, and, to a lesser extent, Latino participants. The following paragraphs offer more detail on these trends.

Matriculation

Significant interaction terms in Table 4, Model 1 are an indication that the relationship between race/ethnicity and matriculation at a 4-year institution varied by SES among black and Asian participants compared with white. Among non-low-income participants, the odds of matriculating at a 4-year institution were significantly higher among white participants than black, Latino, or Asian students. As illustrated in Figure 2, the predicted probability that a non-low-income white participant would matriculate at a 4-year institution was 77.75% compared with 68.87% and 67.49% among non-low-income black and Latino participants, respectively. In contrast, the predicted probability of matriculating at a 4-year institution was higher among low-income Asian and black participants than among low-income white students—74.68% and 69.65% compared with 65.78%, respectively. This is because the association between SES and matriculation was significantly more positive among low-income Asian and black participants than among low-income white students, as illustrated by statistically significant odds ratios greater than 1 for these race/ethnicity by low-income interaction terms. There was no significant difference in the relationship between SES and matriculation among low-income Latino participants compared with low-income white participants.

Looking within, rather than across, racial/ethnic groups, Table 4, Model 1 demonstrates that the odds a low-income white participant would matriculate at a 4-year institution were 47% lower than the odds a non-low-income white participant would do the same, SE = 0.04, p < .001. Because white students are the omitted racial/ethnic group in Equation 1, Table 4, Model 1 cannot examine the statistical significance of within-group SES differences for students from other racial or ethnic backgrounds. However, the predicted probabilities presented in Figure 2 suggest that SES is much less strongly associated with matriculation among black and Asian participants than white and, to a lesser extent, Latino participants. The predicted probability that a non-low-income white participant would matriculate at a 4-year institution was 11.97 percentage points higher than the predicted probability a low-income white participant would do the same. Among black, Asian, and Latino participants, within-group SES differences were −0.78, –3.58, and 7.22 percentage points, respectively.

Persistence

The relationship between race/ethnicity and persistence did not vary significantly by SES (Table 5, Model 1). In addition, there were few significant differences by race/ethnicity or SES. Indeed, the only statistically significant difference was among non-low-income Latino participants, who had 50% lower odds of persisting at a 4-year institution than non-low-income white participants, SE = 0.17, p < .05.

Graduation

Consistent with results for matriculation, the relationship between race/ethnicity and graduation varied significantly by SES (Table 6, Model 1). Among non-low-income participants, the odds of graduating from a 4-year institution within 150% of normal time were significantly higher among white students than black, Latino, or Asian participants. As illustrated in Figure 2, the predicted probability of graduating from a 4-year institution within 6 years was 80.58% for non-low-income white participants compared with 75.36%, 70.98%, and 67.89% for non-low-income Asian, black, and Latino participants, respectively. Among low-income students, the relationship between SES and graduation rates was significantly more positive among black, Latino, and Asian participants compared with white, as illustrated by statistically significant odds ratios greater than 1 for these race/ethnicity × low-income interaction terms. Put another way (Figure 2), the predicted probability of graduating from a 4-year institution within 6 years was 71.65% for low-income white participants compared with 74.93%, 69.59%, and 68.79% for low-income Asian, black, and Latino participants, respectively.

Looking within, rather than across, racial/ethnic, Table 6, Model 1 demonstrates that the odds a low-income white participant would graduate from a 4-year institution within 150% of normal time were 40% lower than the odds a non-low-income white participant would do the same, SE = 0.05, p < .001. Figure 2 presents these differences in terms of predicted probabilities. In Figure 2, the within-group difference in among white participants—8.93 percentage points—was much larger than the same difference among Asian, black, and Latino participants—0.43, 1.39, and −0.90 percentage points, respectively.

Controlling for AP Participation and Performance

Table 7 illustrates the extent of racial/ethnic and SES disparities in AP participation and performance among study participants, while Tables 4 –6, Model 2 simulate the impact of eliminating these disparities on corresponding disparities in college matriculation, persistence, and graduation. Together, they suggest that equalizing participation and performance may shrink but will not entirely eliminate racial/ethnic disparities in college outcomes. The following paragraphs describe these results in more detail.

Table 7.

AP Participation and Performance by Race, Ethnicity, and SES Among AP STEM & English Participants.

	AP participation						AP performance
	Low-income			Non-low-income			Low-income		Non-low-income
	1 exams (%)	2 exams (%)	3 + exams (%)	1 exams (%)	2 exams (%)	3 + exams (%)	M	SD	M	SD
White	42.90	27.67	29.43	39.58	27.73	32.69	2.17	1.01	2.60	1.11
Black	45.58	26.33	28.10	36.03	31.37	32.60	1.80	0.83	2.02	0.96
Latino	45.92	26.47	27.60	43.15	29.07	28.78	1.91	0.91	2.17	1.04
Asian	28.72	26.41	44.87	25.13	21.43	53.44	2.20	1.10	2.50	1.17

Note. AP = Advanced Placement; SES = socioeconomic status.

Racial/ethnic and SES disparities in AP participation and performance

Deep disparities in AP participation and performance were observed among study participants from different racial/ethnic and SES backgrounds. Whereas more than 40.00% of low-income white, black, and Latino participants took just one AP exam (Table 7), 44.87% of low-income Asian students took three or more exams. Similarly, while 53.44% of non-low-income Asian participants took three or more exams, slightly less than one third of non-low-income white, black, and Latino participants did the same—32.69%, 32.60%, and 28.78%, respectively. Bivariate analyses confirmed the statistical significance of racial/ethnic differences in AP participation among low-income and non-low-income participants, low-income: χ²(6, N = 3,847) = 91.24, p < .001 and non-low-income: χ²(6, N = 7,343) = 76.94, p < .001.

In terms of AP performance, white and Asian participants tended to receive higher average AP exam scores than black and Latino participants from similar SES backgrounds (Table 7). Treating AP exam scores as continuous rather than categorical variables indicates that, among non-low-income participants, mean scores were highest among Asian and white participants—2.50 and 2.60, respectively—and lowest among black and Latino participants—2.02 and 2.17, respectively. Among low-income students, performance gaps were smaller. The mean score of low-income Asian and white participants was 2.20 and 2.17, respectively, compared with 1.80 and 1.91 among low-income black and Latino participants, respectively. One-way ANOVAs confirmed the statistical significance of racial/ethnic differences among low-income and non-low-income participants: low-income: F(3, 3,843) = 28.05, p < .001 and non-low-income: F(3, 7,339) = 50.55, p < .001.

The potential of eliminating racial/ethnic and SES disparities by equalizing AP participation and performance

Tables 4 –6, Model 2 add controls for AP participation and performance to Model 1. Comparing Y-standardized coefficients across Models 1 and 2 suggests that, with few exceptions, eliminating racial/ethnic and SES disparities in AP participation and performance will reduce but not entirely eliminate most corresponding disparities in college outcomes between non-low-income white participants and students from other racial/ethnic and SES backgrounds. However, in some instances, equalizing AP participation and performance may have the opposite effect.

Non-low-income participants

Holding AP participation and performance constant reduced disparities in the log odds of college matriculation and graduation among non-low-income black and Latino participants compared with non-low-income white participants by 0.04–0.06 standard deviations, a change of 10.81%–20.69%. In Table 4, Model 1, the log odds that a non-low-income black or Latino participant would matriculate at a 4-year institution were −0.25 and −0.29 standard deviations lower than the log odds a non-low-income white participant would do the same. In Model 2, the same odds were −0.20 and −0.23, respectively. Similarly, in Table 6, Model 1, the log odds that a non-low-income black or Latino participant would graduate from a 4-year institution were −0.29 and −0.37 standard deviations lower than the log odds a non-low-income white participant would do the same. In Model 2, the same log odds were −0.23 and −0.33, respectively.

Holding AP participation and performance constant also decreased persistence disparities between non-low-income white and Latino participants by 0.03 standard deviations or 8.33%. In Table 5, Model 1, the log odds that a non-low-income Latino participant would persist in a 4-year institution for at least 2 years were −0.36 standard deviations lower than the same odds for a non-low-income white participant. In Table 5, Model 2, the difference in log odds was −0.33 standard deviations and no longer a statistically significant, p > .05.

In contrast, holding AP participation and performance constant increased matriculation and graduation disparities between non-low-income Asian participants compared with non-low-income white participants by 0.06 and 0.03 standard deviations, respectively, a change of 31.58% and 17.65%. In Tables 4 and 6, Model 1, the log odds that a non-low-income Asian participant would matriculate at and graduate from a 4-year institution were −0.19 and −0.17 standard deviations lower than the log odds a non-low-income white student would do the same. In Tables 4 and 6, Model 2, the same log odds were −0.25 and −0.20, respectively.

Low-income participants

Holding AP participation and performance constant decreased SES disparities in matriculation and graduation between low-income black, Latino, and white participants compared with non-low-income white participants by −0.02 to −0.11 standard deviations, a change of 7.41% to 47.83%. In Table 4, Model 1, the log odds that a low-income black, Latino, and white participant would matriculate at a 4-year institution were respectively −0.23, −0.47, and −0.33 standard deviations lower than the same odds for a non-low-income white student. In Table 4, Model 2, the same log odds were −0.12, −0.37, and −0.29, respectively. Similarly, in Table 6, Model 1, the log odds that a low-income black, Latino, or white participant would graduate from a 4-year institution were, respectively, −0.33, −0.35, and −0.27 standard deviations lower than the odds a non-low-income white participant would do the same. In Table 6, Model 2, the same log odds were −0.23, −0.28, and −0.25, respectively.

In contrast, matriculation and graduation disparities between low-income Asian students and non-low-income white students remained largely unchanged when controls for AP participation and performance were added to Equation 1. In Table 4, Model 2, the log odds a low-income Asian participant would matriculate at a 4-year institution are the same as the log odds in Model 1, −0.09 standard deviations lower than the log odds a non-low-income white participant would do the same. In Table 6, Model 2, they were just 0.01 standard deviations lower than the same log odds in Model 1, −0.17 standard deviations lower than the log odds a non-low-income white participant would do the same.

The association between AP participation and performance and college outcomes

Although controlling for AP participation and performance affected racial/ethnic and SES disparities in matriculation, graduation, and, to a lesser extent, persistence, AP participation and performance were relatively weak predictors of all three college outcomes. Notably, McFadden’s R² values in Tables 4 –6, Model 2 were just 0.02–0.03 greater than corresponding values in Model 1, and they remained well below commonly accepted thresholds of model fit (McFadden, 1977).

Nonetheless, several statistically significant relationships were observed. Holding all else constant, participants taking two and three or more exams had significantly greater odds of matriculating at and graduating from a 4-year institution than students taking just one exam (Tables 4 and 6, Model 2). Similarly, participants taking three or more exams had significantly greater odds of persisting in college than participants taking just one exam. There was no significant difference is persistence among participants taking two versus one AP exam (Table 4, Model 2).

Like AP participation, AP performance had a positive and statistically significant association with most outcomes. Holding all else constant, participants with an average AP exam score of 2, 3, 4, or 5 had significantly greater odds of matriculating at and graduating from a 4-year institution compared with participants with an average AP exam score of 1, and, with one exception, odds ratios generally increased among participants with higher average scores (Tables 4 and 6, Model 2). In contrast, AP performance was not significantly related to persistence except that the odds a participant with an average AP exam score of 4 would persist for at least 2 years were significantly higher than the same odds for a participant with an average AP exam score of 1 (Table 5, Model 2).

Summary

In general, AP STEM & English participants who took more AP exams and received higher scores on those exams had greater odds of matriculating at, graduating from, and, to a lesser extent, persisting at a 4-year institution. However, there was some variation by outcome and predictor. Although AP participation and performance were only weakly associated with college matriculation, persistence, and graduation, holding AP participation and performance constant narrowed non-low-income white participants’ matriculation and graduation advantage relative to low-income black, white, and Latino participants and non-low-income black and Latino participants by 7.41%–47.83%. Controlling for AP participation and performance also shrunk persistence disparities between non-low-income Latino and white students by 8.33%. The reverse was true for matriculation and graduation disparities between non-low-income white and Asian participants’ AP participation, which grew by 31.58% and 17.65%, respectively. Disparities between low-income Asian participants and non-low-income white students were also not substantially impacted in models controlling for AP participation and performance.

Discussion

Study findings suggest the AP STEM & English program is having its intended effect. Matriculation, persistence, and graduation rates were much higher among program participants than similar students statewide. Differences favoring program participants were also largest among black, Latino, and low-income students. Synthesizing these findings with prior evaluation results suggests the AP STEM & English program is operating as theorized. Program participation leads to significant improvements in AP participation and performance (UMass Donahue Institute, 2017a, 2017b). AP participation and performance, in turn, predict college outcomes among program participants.

While multivariate analyses suggest the relationship between AP participation and performance on one hand and college outcomes on the other is weak, findings are consistent with prior research on the relationship between AP performance and college matriculation (Speroni, 2011), persistence (Mattern et al., 2009), and graduation (Ackerman et al., 2013; Burns et al., 2019; Klopfenstein, 2010; Morgan & Klaric, 2007; Speroni, 2011). Findings also extend Beard et al.’s (2019) work on AP participation, demonstrating a significant association between exams taken on one hand and all three college outcomes on the other. They also confirm prior research finding that black, Latino, and low-income students, on average, participate in AP at lower rates than non-low-income white students and score lower on AP exams (College Board, 2014; Malkus, 2016a).

At the same time, findings regarding the college outcomes of AP STEM & English participants compared with similar students statewide are consistent with quasi-experimental studies of the National Math and Science Initiative’s APIP program—the program on which the AP STEM & English program was modeled. Prior research has demonstrated improved college matriculation, persistence, and graduation among students enrolled in APIP schools (Jackson, 2010, 2014). Like the present study, these studies have shown that Latino, and, to a lesser extent black, students in participating schools experience the greatest improvement in college outcomes. However, it is not yet clear why this is the case, and additional research is needed to understand the program components most strongly associated with improving college outcomes among black, Latino, and low-income students.

Study findings are also significant because they demonstrate the value of looking within as well as across race/ethnicity. We are aware of no prior studies that have examined AP processes or outcomes at the intersection of race/ethnicity and SES, yet the results of the present study demonstrate that outcomes can vary (or fail to vary) in predictable ways within particular subgroups of participants. For example, while evidence suggests that, on average, low-SES students enroll in, persist at, and graduate from college at lower rates than non-low-SES students (McFarland et al., 2018; U.S. Department of Education, 2018), the findings of the present study indicate that there is very little difference in college outcomes between low- and non-low-income black and Latino AP STEM & English participants. Indeed, in two instances in Figure 2, low-income black and Latino students had better outcomes than non-low-income peers from the same racial/ethnic background.

These findings are consistent with the “intersectional paradox” cited frequently in health research (Bowleg, 2008; Cole, 2009) and often observed but not labeled as such in education research (Battle & Lewis, 2002; Battle & Pastrana, 2007; Becares & Priest, 2015). As Cole (2009) explains, “Although higher socioeconomic status is generally associated with better health outcomes, on many health indices, highly educated Blacks fare no better than Whites with the lowest education” (p. 137). While a more fully intersectional study is needed to understand the meaning of this paradox among black and Latino AP students in AP STEM & English schools, findings suggest practitioners may need to develop different strategies to support black and Latino students from low-income and non-low-income backgrounds.

Finally, study findings shed light on AP’s promise as a high-leverage strategy for closing persistent racial/ethnic and SES disparities in college outcomes. With the exception of Asian students, statistically significant racial/ethnic and SES disparities in college outcomes shrunk by 7.41%–47.83% when AP participation and performance were held constant. While this suggests that equalizing AP participation and performance among black, white, and Latino students from different SES backgrounds could be a fruitful strategy for narrowing persistent racial/ethnic and SES disparities in college matriculation and graduation, caution is warranted. As noted below, without controls for academic achievement, AP participation and performance are likely acting, at least in part, as proxies for academic achievement.

Limitations

Without an independent measure of student achievement—for example, GPAs or standardized test scores—it is difficult to determine if AP participation and performance are merely proxies for achievement. As a result, it is impossible to determine if college outcomes are actually associated with AP participation and performance or with unmeasured characteristics of students who take AP (Klopfenstein & Thomas, 2009). Similarly, findings likely overestimate AP’s promise for closing persistent racial/ethnic and SES disparities in college outcomes.

Second, without a comparison group, findings related to the impact of the AP STEM & English program on college outcomes should be viewed as preliminary. Comparisons to statewide averages are promising. However, a quasi-experimental study is required to calculate the true effect of the AP STEM & English program on college outcomes.

Third, persistence is operationalized somewhat differently in the present study than in most AP literature. Due to data limitations, persistence is defined as whether a participant attended any college for at least 2 years. In the literature, persistence is often defined as whether a student was continuously enrolled in the same college for 2 years (Klopfenstein & Thomas, 2009; Massachusetts Department of Elementary and Secondary Education, 2019; Mattern et al., 2009). This may explain why the persistence rates of AP STEM & English program participants are so much higher than statewide results for the class of 2011.

Fourth, during the study period, the state of Massachusetts used eligibility for Free and Reduced Price Lunch (FRPL) as a proxy for SES. Scholars generally agree that SES should be a composite variable including measures of education, income, and occupation (Broer et al., 2019), and consistently find that FRPL enrollment is a relatively weak proxy for household income (Domina et al., 2018). Furthermore, while eligibility for FRPL is a time-varying covariate, AP STEM & English program records keep only the most recent data on each student.

Finally, analyses omitted students who identified as Native American/Alaskan Native, Native Hawaiian/Other Pacific Islander, or Multiracial. This is common practice in AP literature. Indeed, just three of the studies cited above included Native American/Alaskan Native students in their sample (Burns et al., 2019; College Board, 2015; Mattern et al., 2009), and none included students who identified as Native Hawaiian/Other Pacific Islander or Multiracial. But a more inclusive understanding of the relationship between AP participation and performance on one hand and college outcomes on the other is needed, and future studies would do well to test whether the findings of the present study hold in these populations.

Conclusion

Validating key elements of the AP STEM & English program’s theory of change, the present study offers practitioners and policy-makers new details about an evidence-based college readiness program they may wish to replicate in their own communities and demonstrates how an intersectional perspective can surface the needs of otherwise hidden subgroups of participants. At the same time, study findings add to the accumulating body of evidence regarding the relationship between AP participation and performance on one hand and college outcomes on the other. They also suggest AP may be a fruitful strategy for narrowing racial/ethnic and SES disparities in college outcomes, particularly among low-income black and Latino students compared with non-low-income white students.

Footnotes

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors received no external financial support for the research, authorship, and/or publication of this article.

ORCID iD

Sarah Fierberg Phillips

Notes

About the Authors

Sarah Fierberg Phillips is currently the director of research and impact at Mass Insight Education and Research, where she leads studies evaluating the effectiveness of Mass Insight’s programs and initiatives, including the Advanced Placement (AP) Science, Technology, Engineering, and Mathematics (STEM) & English program examined here. Her research interests center on issues of racial and socioeconomic equity, and she has presented and published widely on issues such as school climate, student engagement, and teacher effectiveness. Her research is informed by nearly a decade of experience as an educator and social worker, and she was recently elected to the School Committee in Somerville, MA. She holds a doctorate in Social Policy from Brandeis University, a master’s in Social Work from the University of Michigan, and BA in U.S. History from Brown University.

Brett Lane is president and executive director of INSTLL, LLC, an education consulting firm in Baltimore, MD. He has more than 15 years of experience working with state education agencies, districts, and schools on a wide range of research, consultative, and policy-oriented educational initiatives. He has worked extensively with Mass Insight’s AP STEM & English program since its inception in 2008.

References

Ackerman

P. L.

Kanfer

Calderwood

(2013). High school advanced placement and student performance in college: STEM majors, non-STEM majors, and gender differences. Teachers College Record, 115(10), 1–43.

Battle

Lewis

(2002). The increasing significance of class: The relative effects of race and socioeconomic status on academic achievement. Journal of Poverty, 6(2), 21–35. https://doi.org/10.1300/J134v06n02_02

Battle

Pastrana

Jr. (2007). The relative importance of race and socioeconomic status among Hispanic and White students. Hispanic Journal of Behavioral Sciences, 29(1), 35–49. https://doi.org/10.1177/0739986306294783

Beard

J. J.

Hsu

Ewing

Godfrey

K. E.

(2019). Studying the relationships between the number of APs, AP performance, and college outcomes. Educational Measurement: Issues and Practice, 38(4), 42–54. https://doi.org/10.1111/emip.12295

Becares

Priest

(2015). Understanding the influence of race/ethnicity, gender, and class on inequalities in academic and non-academic outcomes among eighth-grade students: Findings form an intersectionality approach. PLOS ONE, 10(10), Article e0141363. https://doi.org/10.1371/joinral.pone.0141363

Bliese

P. D.

(2000). Within-group agreement, non-independence, and reliability: Implications for data aggregation and analysis. In Klein

K. J.

Koziowski

W. J.

(Eds.), Multilevel theory, research, and methods in organizations: Foundations, extensions, and new directions (pp. 349–381). Jossey-Bass.

Bowleg

(2008). When Black + lesbian + woman ≠ Black lesbian woman: The methodological challenges of qualitative and quantitative intersectionality research. Sex Roles, 59(5–6), 312–325. https://doi.org/10.1007/s11199-008-9400-z

Broer

Bai

Fonseca

(2019). Socioeconomic inequality and educational outcomes: Evidence from twenty years of TIMSS. IEA research for education (A series of in-depth analyses based on data of the International Association for the Evaluation of Educational Achievement (IEA)) (Vol. 5). Springer.

Bureau of Labor Statistics, U.S. Department of Labor, Employment Projections. Table 5.2 Employment, wages, and projected change in employment by typical entry-level education. Retrieved May 21, 2021. https://www.bls.gov/emp/tables/education-summary.htm

10.

Burns

Ellegood

W. A.

Bernard Bracy

J. M.

Duncan

Sweeney

D. C.

(2019). Early college credit programs positively impact student success. Journal of Advanced Academics, 30(1), 27–49. https://doi.org/10.1177/1932202X18801274

11.

Card

Giuliano

(2015). Can universal screening increase the representation of low income and minority students in gifted education. (NBER Working Paper, No. 21519). National Bureau of Economic Research. Retrieved October 13, 2020. https://www.nber.org/papers/w21519

12.

Carnevale

A. P.

Jayasundera

Gulish

(2016). America’s divided recovery: College haves and have-nots. Georgetown University Center on Education and the Workforce. https://1gyhoq479ufd3yna29x7ubjn-wpengine.netdna-ssl.com/wp-content/uploads/Americas-Divided-Recovery-web.pdf

13.

Cattaneo

M. A.

Oggenfuss

Wolter

S. C.

(2017). The more, the better? The impact of instructional time on student performance. Education Economics, 25(5), 433–445. https://doi.org/10.1080/09645292.2017.1315055

14.

Challenge Success. (2013). The Advanced Placement program: Living up to its promise?

15.

Chetty

Friedman

J. N.

Rockoff

J. E.

(2014). Measuring the impacts of teachers II: Teacher value-added and student outcomes in adulthood. American Economic Review, 104(9), 2633–2679. https://doi.org/10.3386/w19424

16.

Chrispeels

J. H.

Brown

J. H.

Castillo

(2000). School leadership teams: Factors that influence their development and effectiveness. Advances in Research and Theories of School Management and Educational Policy, 4, 39–73.

17.

Clotfelter

C. T.

Ladd

H. F.

Vigdor

J. L.

(2006). Teacher-student matching and the assessment of teacher effectiveness. The Journal of Human Resources, 41(4), 778–820. https://doi.org/10.3368/jhr.xli.4.778

18.

Cole

E. R.

(2009). Intersectionality and research in psychology. American Psychologist, 64(3), 170–180. https://doi.org/10.1037/a0014564

19.

College Board. (2014). The 10th annual AP report to the nation.

20.

College Board. (2015). District leadership playbook: Expanding access to advanced placement for students of color.

21.

College Board. (2020a). About AP scores. https://apstudents.collegeboard.org/about-ap-scores

22.

College Board. (2020b). AP credit policy search. https://apstudents.collegeboard.org/getting-credit-placement/search-policies

23.

College Board. (2020c). AP program. https://ap.collegeboard.org

24.

Crenshaw

(1991). Mapping the margins: Intersectionality, identity politics, and violence against women of color. Stanford Law Review, 43(6), 1241–1299. https://doi.org/10.2307/1229039

25.

Darling-Hammond

Hyler

M. E.

Gardner

Espinoza

(2017). Effective teacher professional development. Learning Policy Institute.

26.

Dobbie

Fryer

R. G.

Jr. (2013). Getting beneath the veil of effective schools: Evidence from New York City. American Economic Journal: Applied Economics, 5(4), 28–60. https://doi.org/10.3386/w17632

27.

Domina

Pharris-Ciurej

Penner

A. M.

Penner

E. K.

Brummet

Porter

S. R.

Sanabria

(2018). Is free and reduced-price lunch a valid measure of educational disadvantage? Educational Researcher, 47(9), 539–555. https://doi.org/10.3102/0013189x18797609

28.

Dubrow

J. K.

(2008). How can we account for intersectionality in quantitative analysis of survey data? Empirical illustration for Central and Eastern Europe. ASK: Research and Methods, 17, 85–100.

29.

Earl

Katz

(2002). Leading schools in a data-rich world. In Leithwood

Hallinger

(Eds.), Second international handbook of educational leadership and administration (pp. 1003–1022). Kluwer Academic.

30.

Else-Quest

N. M.

Hyde

J. S.

(2016). Intersectionality in quantitative psychological research: II. Methods and techniques. Psychology of Women Quarterly, 40(3), 319–336. https://doi.org/10.1177/0361684316647953

31.

Gamoran

(2009). Tracking and inequality: New directions for research and practice. (WCER Working Paper, No. 2009-06). Wisconsin Center for Education Research. Retrieved October 13, 2020. https://wcer.wisc.edu/docs/working-papers/Working_Paper_No_2009_06.pdf

32.

Garet

M. S.

Porter

A. C.

Desimone

Birman

B. F.

Yoon

K. S.

(2001). What makes professional development effective? Results from a national sample of teachers. American Educational Research Journal, 38(4), 915–945. https://doi.org/10.3102/00028312038004915

33.

Geiser

Santelices

(2004). The role of advanced placement and honors courses in college admissions (No. CSHE.4.04). Center for Studies in Higher Education, University of California, Berkeley.

34.

Gillborn

Warmington

Demack

(2018). QuantCrit: Education, policy, “Big Data” and principles for a critical race theory of statistics. Race Ethnicity and Education, 21(2), 158–179. https://doi.org/10.1080/13613324.2017.1377417

35.

Hair

J. F.

Jr. Black

W. C.

Babin

B. J.

Anderson

R. E.

(2010). Multivariate data analysis: A global perspective (7th ed.). Pearson Education.

36.

Hill

H. C.

Beisiegel

Jacob

(2013). Professional development research: Consensus, crossroads, and challenges. Educational Researcher, 42(9), 476–487. https://doi.org/10.3102/0013189x13512674

37.

Jackson

C. K.

(2010). The effects of an incentive-based high-school intervention on college outcomes. National Bureau of Economic Research.

38.

Jackson

C. K.

(2014). Do college-preparatory programs improve long-term outcomes? Economic Inquiry, 52(1), 72–99. https://doi.org/10.1111/ecin.12040

39.

Jeong

D. W.

(2009). Student participation and performance on Advanced Placement exams: Do state-sponsored incentives make a difference? Educational Evaluation and Policy Analysis, 31(4), 346–366. https://doi.org/10.3102/0162373709342466

40.

Jerrim

Lopez-Agudo

L. A.

Marcenaro-Gutierrez

O. D.

Shure

(2017). What happens when econometrics and psychometrics collide? An example using the PISA data. Economics of Education Review, 61, 51–58. https://doi.org/10.1016/j.econedurev.2017.09.007

41.

Joyce

Showers

(2002). Student achievement through staff development. In Designing training and peer coaching: Our needs for learning. Association for Supervision and Curriculum Development. https://www.unrwa.org/sites/default/files/joyce_and_showers_coaching_as_cpd.pdf

42.

Kinnucan-Welsch

Rosemary

C. A.

Grogan

P. R.

(2006). Accountability by design in literacy professional development. The Reading Teacher, 59(5), 426–435. https://doi.org/10.1598/rt.59.5.2

43.

Klopfenstein

(2010). Does the Advanced Placement program save taxpayers money? The effect of AP participation on time to college graduation. In Sadler

P. M.

Sonnert

Tai

R. H.

Klopfenstein

(Eds.), AP: A critical examination of the Advanced Placement program (pp. 189–218). Harvard Education Press.

44.

Klopfenstein

Thomas

(2009). The link between Advanced Placement experience and early college success. Southern Economic Journal, 75, 873–891.

45.

Knight

(2006). Instructional coaching. School Administrator, 63(4), 36–40.

46.

Kolluri

(2018). Advanced Placement: The dual challenge of equal access and effectiveness. Review of Educational Research, 88(5), 671–711. https://doi.org/10.3102/0034654318787268

47.

Kraft

M. A.

Blazar

Hogan

(2018). The effect of teacher coaching on instruction and achievement: A meta-analysis of the causal evidence. Review of Educational Research, 88(4), 547–588. https://doi.org/10.3102/0034654318759268

48.

Kulik

J. A.

Kulik

C.-L. C.

Bangert

R. L.

(1984). Effects of practice on aptitude and achievement test scores. American Educational Research Journal, 21(2), 435–447. https://doi.org/10.3102/00028312021002435

49.

Lankford

Loeb

Wyckoff

(2002). Teacher sorting and the plight of urban schools: A descriptive analysis. Educational Evaluation and Policy Analysis, 24(1), 37–62. https://doi.org/10.3102/01623737024001037

50.

Lavy

(2015). Do differences in schools’ instruction time explain international achievement gaps? Evidence from developed and developing countries. The Economic Journal, 125(588), F397–F424. https://doi.org/10.3386/w16227

51.

Lichten

(2010). Whither Advanced Placement—Now? In Sadler

P. M.

Sonnert

Tai

Klopfenstein

(Eds.), AP: A critical examination of the Advanced Placement program (pp. 233–243). Harvard Education Press.

52.

Long

J. S.

Freese

(2006). Regression models for categorical dependent variables using Stata. Stata Press.

53.

Malkus

(2016a). AP at scale: Public school students in Advanced Placement, 1990–2013. American Enterprise Institute.

54.

Malkus

(2016b). The AP peak: Public schools offering Advanced Placement, 2000–12. American Enterprise Institute.

55.

Massachusetts Department of Elementary and Secondary Education. (2017). Low-income student calculation study. https://www.doe.mass.edu/bese/docs/fy2017/2017-02/item9-study.pdf

56.

Massachusetts Department of Elementary and Secondary Education. (2019). DART detail: Success after high school. http://www.doe.mass.edu/dart/

57.

Massachusetts Department of Elementary and Secondary Education. (2020). Massachusetts students lead nation in Advanced Placement success for fourth year [Press release].

58.

Mattern

K. D.

Shaw

E. J.

Xiong

(2009). The relationship between AP® exam performance and college outcomes (Research Report No. 2009-4). College Board.

59.

McCall

(2005). The complexity of intersectionality. Signs: Journal of Women in Culture and Society, 30(3), 1771–1800.

60.

McFadden

(1977). Quantitative methods for analyzing travel behaviour of individuals: Some recent developments. Cowles Foundation for Research in Economics, Yale University. https://EconPapers.repec.org/RePEc:cwl:cwldpp:474

61.

McFarland

Hussar

Wang

Zhang

Wang

Rathbun

Barmer

Cataldi

E. F.

Nachazel

Smith

Ossolinski

Mann

(2018). The conditions of education 2018: Postsecondary education, postsecondary students. National Center for Education Statistics, Institute of Education Sciences, U.S. Department of Education.

62.

Morgan

Klaric

(2007). AP® students in college: An analysis of five-year academic careers (Research Report No. 2007-4). College Board.

63.

Neufeld

Roper

(2002). Off to a good start: Year I of collaborative coaching and learning in the effective practice schools. Education Matters.

64.

Ovadia

(2001). Race, class, and gender differences in high school seniors’ values: Applying intersection theory in empirical analysis. Social Science Quarterly, 82(2), 340–356. https://doi.org/10.1111/0038-4941.00027

65.

Penuel

W. R.

Fishman

B. J.

Yamaguchi

Gallagher

L. P.

(2007). What makes professional development effective? Strategies that foster curriculum implementation. American Educational Research Journal, 44(4), 921–958. https://doi.org/10.3102/0002831207308221

66.

Pregibon

(1980). Goodness of link tests for generalized linear models. Journal of the Royal Statistical Society: Series C (Applied Statistics), 29(1), 15–24. https://doi.org/10.2307/2346405

67.

Rawson

K. A.

Dunlosky

(2012). When is practice testing most effective for improving the durability and efficiency of student learning? Educational Psychology Review, 24(3), 419–435. https://doi.org/10.1007/s10648-012-9203-1

68.

Reinke

W. M.

Lewis-Palmer

Merrell

(2008). The classroom check-up: A classwide teacher consultation model for increasing praise and decreasing disruptive behavior. School Psychology Review, 37(3), 315–332. https://doi.org/10.1080/02796015.2008.12087879

69.

Renbarger

Long

(2019). Interventions for postsecondary success for low-income and high-potential students: A systematic review. Journal of Advanced Academics, 30(2), 178–202. https://doi.org/10.1177/1932202x19828744

70.

Rivkin

S. G.

Hanushek

E. A.

Kain

J. F.

(2005). Teachers, schools, and academic achievement. Econometrica, 73(2), 417–458. https://doi.org/10.1111/j.1468-0262.2005.00584.x

71.

Rivkin

S. G.

Schiman

J. C.

(2015). Instruction time, classroom quality, and academic achievement. The Economic Journal, 125(588), F425–F448. https://doi.org/10.3386/w19464

72.

Rockoff

J. E.

(2004). The impact of individual teachers on student achievement: Evidence from panel data. American Economic Review, 94(2), 247–252. https://doi.org/10.1257/0002828041302244

73.

Sadler

P. M.

Sonnert

(2010). High school Advanced Placement and success in college coursework in the sciences. In Sadler

P. M.

Sonnert

Tai

R. H.

Klopfenstein

(Eds.), AP: A critical examination of the Advanced Placement program (pp. 119–137). Harvard Education Press.

74.

Sarkar

S. K.

Midi

Rana

(2011). Detection of outliers and influential observations in binary logistic regression: An empirical study. Journal of Applied Sciences, 11(1), 26–35. https://doi.org/10.3923/jas.2011.26.35

75.

Sasson

Hayward

M. D.

(2019). Association between educational attainment and causes of death among White and Black U.S. adults, 2010–2017. Journal of the American Medical Association, 322(8), 756–763. https://doi.org/10.1001/jama.2019.11330

76.

Speroni

(2011). Determinants of students’ success: The role of Advanced Placement and dual enrollment programs. National Center for Postsecondary Research.

77.

Taylor

J. E.

(2008). Instructional coaching: The state of the art. In Mangin

M. M.

Stoelinga

(Eds.), Effective teacher leadership: Using research to inform and reform (pp. 10–35). Teachers College Press.

78.

Torpey

(2018). Measuring the value of education. U.S. Bureau of Labor Statistics. https://www.bls.gov/careeroutlook/2018/data-on-display/education-pays.htm

79.

UMass Donahue Institute. (2017a). Evaluation of the statewide STEM Advanced Placement program: AP course taking and passing rates. MA Department of Elementary and Secondary Education.

80.

UMass Donahue Institute. (2017b). Evaluation of the statewide STEM Advanced Placement Program: AP exam taking and passing rates. MA Department of Elementary and Secondary Education.

81.

U.S. Department of Education. (2000). A forum to expand Advanced Placement opportunities: Increasing access and improving preparation in high schools. Transcript of proceedings. https://eric.ed.gov/?id=ED448521

82.

U.S. Department of Education. (2018). 2012/17 beginning postsecondary students longitudinal study (Computation by NCED QuickStats on 1/9/2020 No. BPS: 12/17). https://nces.ed.gov/surveys/bps/

83.

U.S. Department of Education, Office for Civil Rights. (2014). Civil rights data collection: Data snapshot (college and career readiness) (No. Issue Brief No. 3). https://files.eric.ed.gov/fulltext/ED577784.pdf

84.

Williams

(2018, February 21). Massachusetts leads nation in highest scores on Advanced Placement exams. MassLive. https://www.masslive.com/news/2018/02/massachusetts_leads_nation_in.html

85.

Williams

Wang

(2019, March). Comparing logit and profit coefficients between models [Paper presentation]. August 2012 Annual Meetings of the American Sociological Association, Department of Sociology, University of Notre Dame. https://www3.nd.edu/~rwilliam/stats3/Nested02.pdf