Abstract
This article examines the role of socioeconomic status (SES) in the relationships among college admissions-test scores, secondary school grades, and subsequent academic performance. Scores on the SAT (a test widely used in the admissions process in the United States), secondary school grades, college grades, and SES measures from 143,606 students at 110 colleges and universities were examined, and results of these analyses were compared with results obtained using a 41-school data set including scores from the prior version of the SAT and using University of California data from prior research on the role of SES. In all the data sets, the SAT showed incremental validity over secondary school grades in predicting subsequent academic performance, and this incremental relationship was not substantially affected by controlling for SES. The SES of enrolled students was very similar to that of specific schools’ applicant pools, which suggests that the barrier to college for low-SES students in the United States is a lower rate of entering the college admissions process, rather than exclusion on the part of colleges.
Access to a college or university education is a scarce resource. The college admissions systems of different countries fall into three main categories. First, some systems use a test focusing on a common set of topics for all test takers. The use of the SAT (with Critical Reading, Mathematics, and Writing subtests) and ACT (with English, Reading, Mathematics, and Science subtests) in the United States illustrates this model, as do testing systems in countries such as Israel, Pakistan, Russia, Turkey, and South Korea. Second, some countries have test-based systems that focus on achievement in specific topics, with students choosing the topics on which to focus. The British A-levels and the French baccalauréat exemplify this approach. Finally, other systems do not use tests, but rely primarily on performance in secondary school (e.g., Australian Tertiary Admissions Record system).
The U.S.-based tests are administered worldwide, as many international students wish to pursue postsecondary education in the United States. Thus, there is broad interest in tests of the type used in the United States. Admissions tests are commonly used in conjunction with a variety of other sources of information, with secondary school performance the most prominent. In the U.S. educational system, secondary school is termed high school and typically involves 4 years of study between the ages of 14 and 18. The relative emphasis on test scores, high school performance, and other factors varies by college or university.
Concern about access to postsecondary education for differing segments of the population is widespread. Given societal goals of broad access to educational opportunities, understanding the roles played by family socioeconomic status (SES) and standardized tests in the admissions process is of interest both as a scientific issue and as an issue of social policy. Whereas college admissions tests have extensive research support as predictors of subsequent academic performance (e.g., Hezlett et al., 2001; Noble & Sawyer, 2002; Sackett, Kuncel, Arneson, Cooper, & Waters, 2009), the role of SES in college admissions remains the focus of considerable attention. Indeed, prominent critics of the SAT have asserted that University of California data show that the incremental power of the SAT in predicting college performance, over and above the predictive power of high school grade point average (GPA), is “decisively diminished” (Atkinson, 2009) or “significantly reduced” (Atkinson & Geiser, 2009, p. 673) once SES is taken into consideration; prompted by such concern, the University of California system has questioned its continued use of the SAT.
Here, we examine two critical questions about the role of SES in the college admissions process. The first is whether the incremental validity of the SAT in predicting subsequent academic performance above and beyond the level of predictive accuracy obtained using high school GPA is substantially reduced once SES is controlled. We present results using the current SAT in a data set of 110 colleges and universities and then compare these results with findings from two historical data sets, one including scores on the prior version of the SAT and the other comprising University of California data.
Other studies have examined related issues. Rothstein (2004) reported that the contribution of SAT scores to the prediction of academic performance was reduced when a broad set of demographic variables, including race and demographic characteristics of the high school a student attended, were controlled. However, Rothstein did not include measures of SES at the individual student level. Bowen, Chingos, and McPherson (2009) reported that the small relationship between SAT scores and college graduation is reduced further when SES is controlled. Geiser and Studley (2002) simultaneously added scores on another test (the SAT II), as well as SES, to models using SAT scores and high school GPA to predict performance, so it is not possible to determine whether the observed effects of adding these predictors were due to SAT II scores or to SES. Our study differs from these in focusing on the possible role of the SES of the individual student in influencing the contribution of SAT scores to the prediction of subsequent college grades.
The second question concerns the effect of admissions systems on the SES distribution of college enrollees. There are multiple mechanisms that might limit college access for low-SES students. First, students with lower SES are less likely to enter into the college admissions process (e.g., Cabrera & La Nasa, 2000; Hossler, Schmit, & Vesper, 1999; Walpole, 2003). Second, colleges may have an implicit or explicit bias against low-SES students, and thus use SES as part of the admissions process. Third, factors commonly considered in the admissions process, such as admissions-test scores, are correlated with SES, and heavy reliance on these factors may result in limiting access for low-SES students. Sackett et al. (2009) reported a correlation of .42 between SAT scores and SES among the population of SAT takers; exclusive reliance on SAT scores in admissions would limit access for low-SES students.
To shed light on the role of SES in the admissions process, we compared the variability of SES in the applicant pool for each school with the SES variability in the group of enrolled students. The extent to which SES variability was reduced in the enrolled group relative to the applicant group was our index of the degree to which the admissions process restricts access on the basis of SES.
Method
2006 revised-SAT data set
In collaboration with 110 U.S. colleges and universities, the College Board collected information on revised-SAT scores, high school GPA, SES, and freshman college grades from the students entering each institution in 2006. The schools were selected, on the basis of a sampling plan, to be geographically diverse, to include large and small schools, to include public and private institutions, and to cover a broad range of school selectivity. The 110 schools included 63 private and 47 public institutions. Freshman class size averaged 1,305.5 (SD = 1,286.5) and ranged from 106 to 6,462. The mean high school GPA (the traditional scale from 0 to 4, with extra credit for Advanced Placement and honors courses) among entering students across the 110 schools was 3.56 (SD = 0.24), and the mean school-specific high school GPA ranged from 2.86 to 4.05. The mean SAT total score (i.e., Critical Reading + Math + Writing) across schools was 1676.5 (SD = 181.8), and the mean school-specific SAT score ranged from 1281.2 to 2157.3 (scores can range from 600 to 2400). Our primary analyses are based on data from those students for whom the data set included an SES measure, in addition to SAT scores and high school and college GPAs; this subset consisted of 143,606 cases. These primary analyses relied on students’ self-reported high school GPAs. We also conducted a secondary set of analyses based on high school GPAs provided by the colleges and universities. School-reported GPAs were available from 49 of the 110 colleges and universities, with the number of students totaling 60,361.
Test scores
Critical Reading (CR), Math (M), and Writing (W) scores on the revised SAT were obtained from College Board records. We combined these scores into a unit-weighted composite: CR + M + W.
SES
Three SES variables were obtained from questionnaires completed by students at the time they took the SAT: father’s education, mother’s education, and family income. The natural log of family income was used. In the entire national population of SAT takers, the correlation between the two parent’s education variables was .60; father’s and mother’s education correlated .42 and .39, respectively, with family income. Following other research on this topic (e.g., Sackett et al., 2009), we created an equally weighted composite of these three variables.
Freshman GPA
Freshman (i.e., 1st-year) GPA was provided by the colleges and universities.
Self-reported high school GPA
Self-reported high school GPA was obtained from the students’ SAT questionnaires.
College-reported high school GPA
A subset of individual colleges and universities provided students’ high school GPAs from their own records. Self- and school-reported high school GPA can differ for multiple reasons. For example, students may forget or misreport their GPAs. Also, postsecondary schools may recompute enrollees’ high school GPAs, using students’ transcripts and including only grades for a specified set of classes (e.g., core academic subjects). In our data, self-reported and school-reported high school GPA correlated .75; a meta-analysis of self- and school-reported high school GPA by Kuncel, Crede, and Thomas (2005) reported a mean r of .82.
1995–1997 SAT data set
As already mentioned, we compared results obtained using the 2006 data set with those obtained using an earlier data set in which the SAT scores were from the version of the test in use prior to the 2005 revision. The earlier data set, including 136,725 students from cohorts entering 41 schools in 1995 through 1997, is structurally similar to the 2006 data set in terms of the variables included. In the 1995–1997 data set, the SAT total score was a combination of Verbal (renamed Critical Reading in the 2005 revision) and Math scores, as the Writing test was added in the 2005 revision. Details about the data set (e.g., means, standard deviations, and intercorrelations) can be found in Sackett et al. (2009).
Analyses
For the 2006 data set, we sought to obtain data on the applicant population in order to correct correlations among SAT scores, high school GPA, freshman GPA, and SES for range restriction. Restricted variance in enrolled students, due to the use of the SAT and high school GPA in admissions decisions, results in a lower test-grade correlation than would be found if the relationship were examined in applicant samples (Sackett & Yang, 2000).
The additional data needed to correct for multivariate range restriction are the unrestricted means, standard deviations, and correlations among the variables. We used two separate sources of information for this purpose. First, we obtained means, standard deviations, and correlations for SAT scores, self-reported high school GPA, and SES among the entire population of individuals who took the SAT and completed a questionnaire reporting SES in 2006 (more than 1.25 million students). Second, we estimated the means, standard deviations, and correlations for these same variables in the applicant pool for each specific college or university. To do this, we needed a reasonable proxy for each school-specific applicant pool. When students take the SAT, they indicate the schools to which they wish their scores be sent; we used the set of students who asked that their scores be sent to a given school as our estimate of the applicant pool for that school (a strategy also used by Sackett et al., 2009).
Correcting for range restriction using each college’s applicant pool estimated how well SAT score and high school GPA predicted freshman GPA across that college’s applicant pool. Correcting for range restriction using the entire population of SAT takers estimated how well SAT score and high school GPA predicted freshman GPA in the population of test takers. We present both types of correction for comparison purposes.
To examine the SAT composite as a predictor of freshman GPA when used in conjunction with high school GPA and SES in the 2006 data set, we looked at a series of regression models. In Model 1, SAT score was the only predictor, and in Model 2, high school GPA was the only predictor. Model 3 included both SAT score and high school GPA, and Model 4 added SES. These models were examined separately for each of the 110 schools, and sample-size-weighted mean standardized regression coefficients and squared multiple correlations were computed. These four models were also examined in the subset of 49 schools for which school-reported high school GPA was available and in the 1995–1997 data set with scores from the previous version of the SAT.
To investigate the degree to which the admissions process restricts access for low-SES students, we examined SAT, high school GPA, and SES means and variances among applicants and enrolled students separately at each school. If a school preferentially selected students with higher SES, the SES variance would be substantially reduced in the enrolled group.
Results
Table 1 presents sample-size-weighted mean correlations between study variables across the schools in the 2006 and 1995–1997 data sets. Table 2 presents results for the regression models for the 2006 revised-SAT data set; Figure 1 presents the results graphically. We first discuss observed relationships, prior to correction for restriction of range. In Model 1, SAT score alone had a mean regression weight of 0.35. In Model 2, high school GPA alone had a mean regression weight of 0.37. When SAT and high school GPA were both included, in Model 3, the resulting regression weights were 0.30 for high school GPA and 0.27 for SAT score. The squared multiple correlation of .21 for Model 3 was higher than was found using either SAT score alone (.13) or high school GPA alone (.14).
Sample-Size-Weighted Correlations From the 2006 and 1995–1997 Data Sets
Note: Correlations above the diagonal are from the 1995–1997 data set (K = 41, N = 136,725), and correlations below the diagonal are from the 2006 data set (K = 110, N = 143,606). SAT score was the total score from the Verbal and Math subtests for the 1995–1997 data set and from the Critical Reading, Math, and Writing subtests for the 2006 data set. GPA = grade point average; SES (socioeconomic status) composite = unit-weighted composite of father’s education, mother’s education, and log parental income.
SAT Score, Self-Reported High School Grade Point Average (GPA), and Socioeconomic Status (SES) as Predictors of Freshman GPA: 2006 Data
Note: The values in the table are means across 110 schools (N = 143,606).
SAT score, high school grade point average (GPA), and socioeconomic status (SES) as predictors of freshman GPA in the 2006 data set. Results are shown for Model 1, in which SAT score was the only predictor; Model 2, in which high school GPA was the only predictor; Model 3, which included both SAT score and high school GPA; and Model 4, which added SES. The lower graph presents standardized regression coefficients, and the upper graph presents the R2 values for all four models. Both observed results and results corrected for school-specific range restriction are shown.
These findings are consistent with prior literature in three respects. First, secondary school performance is generally found to be a slightly better predictor of academic performance than are admissions-test scores (Zwick, 2002). Second, when SAT score and high school GPA are used in conjunction, the regression coefficient for each is smaller than when each is used alone, a result reflecting the shared variance between the two. Third, using the two predictors together produces a meaningful increment in predictive power over using either individually.
Finally, Model 4 included SAT score, high school GPA, and SES simultaneously. The addition of SES to the model reduced the SAT coefficient only slightly: from 0.27 to 0.25. In other words, SAT score retained the vast majority of its predictive power when SES was added to the model. In addition, adding SES increased the overall predictive power of the model by about 1% of the variance. This is consistent with the comparatively weak relationship between SES and freshman grades (r = .13 in the present data). Thus, SAT score and high school GPA together remained useful predictors of academic performance when SES was controlled.
The observed values represent relationships in the samples of enrolled students, whereas the question of interest concerns the relationships in the school-specific applicant pools and the national SAT-taking population. On average, the standard deviation for SAT score among enrolled students at a given school was 18% smaller than the standard deviation among applicants to that school, a reduction reflecting the selection process used by the school, as well as any SAT-related self-selection in accepting admission offers. Table 2 includes regression coefficients and squared multiple correlations corrected for range restriction at both the school-specific applicant-pool and national applicant-pool levels; Figure 1 shows the school-specific corrected values only (in the interest of ease of presentation). Although the regression coefficients and squared multiple correlations were, as expected, higher after correction for range restriction, the pattern of results paralleled that for the observed data.
The same four models were examined in the 49-school subset for which school-reported high school GPA was available. As school-reported high school GPA was available only for admitted students, correction for restriction of range was not possible. These results are presented in Table 3. Several key findings obtained using self-reported high school GPA continued to hold: Using high school GPA and SAT score in conjunction produced a higher squared multiple correlation than using either one alone, and adding SES had minimal effects on the regression coefficients for SAT score and high school GPA. Note, though, that the regression weights for high school GPA were higher using school reports than self-reports, and that the predictive advantage of high school GPA over SAT score was greater using the school-reported data.
SAT Score, School-Reported High School Grade Point Average (GPA), and Socioeconomic Status (SES) as Predictors of Freshman GPA: 2006 Data
Note: The values in the table are means across 49 schools (N = 60,361).
These four models were also examined in the 1995–1997 data set with scores from the previous version of the SAT. The results (see Table 4), based on observed data without correction for range restriction, closely parallel those obtained with the revised SAT. Thus, neither the 2006 data nor the 1995–1997 data offer evidence that controlling for SES will substantially reduce the incremental predictive validity of SAT score above and beyond high school GPA.
SAT Score, Self-Reported High School Grade Point Average (GPA), and Socioeconomic Status (SES) as Predictors of Freshman GPA: 1995–1997 Data
Note: The values in the table are means across 41 schools (N = 136,725).
We now turn to the University of California data set, which includes about 78,000 entering students at eight of the University of California campuses between 1996 and 1999. We drew from supporting materials (Geiser & Studley, 2001) from a study of these data to create close analogues to the four models we examined for the previous data sets. The approach taken by Geiser and Studley (2001, 2002) in their analyses of the University of California data differs from our approach with the College Board data in two respects. First, Geiser and Studley (2002) aggregated data across the eight University of California campuses, whereas we analyzed each school’s data separately and then averaged the results. Second, Geiser and Studley included parental income and parental education as two separate variables, whereas we used a composite of parental income, mother’s education, and father’s education in the College Board data (though keeping the three separate did not discernibly affect our findings). In our analyses of the College Board data, adding SES to the model involved adding this composite; in Geiser and Studley’s analyses of the University of California data, adding SES involved adding the two SES indicators. The high school GPA measure in the University of California data was school reported and used a weighting formula that gave extra points for honors courses; thus, it was more comparable to the school-reported than to the self-reported high school GPA measure we used in analyzing the 2006 College Board data.
Table 5 includes a summary of results for the University of California data (Models 1–4 plus two additional models). Findings closely followed those for the 2006 and 1995–1997 data. For comparison here, we focus on the 2006 data using school-reported high school GPA. In Model 1, SAT I (the traditional SAT) score alone had a coefficient of 0.37 (vs. 0.35 in the 2006 data set). Including both SAT score and high school GPA reduced the coefficients for both, with the SAT coefficient taking a value of 0.25 (vs. 0.23 in the 2006 data set). Adding SES to the model reduced the SAT coefficient to 0.22 (vs. 0.21 in the 2006 data set). Thus, in both the University of California data and the 2006 data, SAT score retained the vast majority of its predictive power above and beyond high school GPA even after SES was added to the model.
SAT I Score, SAT II Score, High School Grade Point Average (GPA), and Socioeconomic Status as Predictors of Freshman GPA: University of California data
Atkinson (2009) and Atkinson and Geiser (2009) relied on Geiser and Studley’s (2001) work in asserting that adding SES substantially reduces the SAT coefficient. However, it is important to consider that Geiser and Studley did not add SES by itself to the model including SAT and high school GPA; rather, they also added composite score on the SAT II (scores on three achievement scales: writing, mathematics, and a third subject of the student’s choosing). Thus, they focused on what we include in Table 5 as Model 5. In this model, the SAT I coefficient did indeed drop to near zero (i.e., 0.02). But it would be incorrect to infer from these results that it was the addition of SES that caused the SAT I coefficient to drop. As Model 4 shows, adding SES alone did not reduce the SAT I coefficient. For added clarity, we also include in Table 5 the results for a Model 6, which ignored SES completely and added SAT II score to a model including SAT I score and high school GPA. This model showed that adding SAT II score reduced the SAT I coefficient from 0.25 to 0.07. So it was the addition of SAT II score, not SES, that drove the reduction in the SAT I coefficient in Geiser and Studley’s study.
Turning to the effects of the admissions process on the distribution of SES in enrolled students, we compared the SES of applicants and enrolled students. In the 2006 data on 110 schools, the mean SES among enrolled students was only 0.126 standard deviations higher than the mean SES in the school-specific applicant pools. Similarly, the standard deviation of SES among enrolled students was, on average, only 3% smaller than the standard deviation among applicants. Both of these findings are contrary to what would be found if proxies for SES were the sole or primary determinant of admissions, or if schools had a per se preference for high-SES students. If either were the case, mean SES among enrollees would be much higher than the mean in the applicant pool, and the standard deviation for enrollees would be much smaller than the standard deviation in the applicant pool. It is useful to contrast this analysis with similar analyses for SAT scores and high school grades. Although the standard deviation of SES among enrolled students averaged only 3% less than the standard deviation in the applicant pool, the enrolled students’ standard deviations for SAT score and high school grades averaged 20% and 18% less than the standard deviations in the applicant pool. Note that there was variability from school to school in the difference in mean SES between applicants and enrolled students, with some schools showing greater differences than others. These differences tended to be larger in more selective schools: There was a correlation of –.39 between the percentage of applicants admitted and the difference in mean SES between applicants and enrolled students.
Discussion
The finding that SAT scores provide incremental validity in predicting freshman grades, beyond the predictive validity contributed by high school grades, and that this is true even when controlling for SES supports the usefulness of the SAT for predicting first-year academic performance. Although it will take time for relationships between the revised SAT and longer-term academic performance to be observed, it is useful to note that, contrary to the popular assertion that tests like the SAT predict nothing but freshman grades, extensive research documents the relationship between test scores and academic performance throughout the curriculum (Berry & Sackett, 2009; Hezlett et al., 2001).
Although it is not the case that test validity is an artifact of SES, SES is linked to the development of abilities that are predictive of academic performance. In the 2006 national population of test takers, the correlation between SES and composite SAT score was .46. Therefore, 21.2% of variance in SAT scores is shared with SES, as measured here as a composite of mother’s education, father’s education, and parental income. Thus, SAT scores are by no means isomorphic with SES, although the source of the SES-SAT relationship is likely due to some combination of educational opportunity, school quality, peer effects, and other social factors.
The finding that SES variability was only 3% smaller among enrolled students than in the applicant pool has important consequences that challenge the popular notion that U.S. college selection systems actively screen out low-SES students. An admissions policy that relied heavily or exclusively on the SAT would indeed screen out low-SES students at a higher rate than high-SES students. However, the data from the 110 schools suggest that in the typical U.S. school, SES does not play a primary exclusionary role in the admissions process. Thus, once students are in the applicant pool for a given school, the school typically does not substantially restrict entry on the basis of SES. Although low-SES students are less likely to enroll in college, this appears to be a function of differences in the rates at which high- and low-SES students choose to enter the college application process, rather than of exclusion on the part of colleges.
The present study focused on one widely used U.S-based admissions test. Although our study did include a broad range of colleges, it was nonetheless limited to U.S. colleges. Each of the major findings merits investigation in other settings. The strength of the relationship between admissions-test scores and SES, for example, could vary from country to country as a function of the degree to which the quality of available education is linked to SES. Similarly, the finding that SES affects the decision to enter the application process to a greater degree than it affects admissions decisions by colleges may not hold in all countries. Thus, we present these findings as providing a basis for research comparing the role of SES in other settings.
In sum, large multi-institution data sets show that (a) the SAT retains the vast majority of its weight in predicting subsequent college grades when high school GPA and SES are controlled, (b) the bivariate relationship between SAT score and freshman GPA is not substantially reduced when SES is controlled, and (c) implicitly, given these two findings, it is not the case that the SAT is nothing more than a proxy for SES. An additional important finding is that the SES distribution of enrolled students at a college closely resembles the distribution of applicants to that college, which indicates that the barrier to college for low-SES students in the United States is far more a matter of factors that affect the decision to enter the college application process than of exclusionary admissions decisions on the part of colleges.
Footnotes
Acknowledgements
Declaration of Conflicting Interests
Paul R. Sackett serves as a research adviser to the College Board, and this research was supported by a grant from the College Board to Paul R. Sackett and Nathan R. Kuncel. Beyond this, the authors declared that they had no conflicts of interest with respect to their authorship or the publication of this article.