The Concept of Academic Mobility: Normative and Methodological Considerations

Abstract

Most of the literature on the development of educational inequality has operated under the achievement gaps paradigm, often assuming that the underlying normative and methodological foundations related to equality and justice in education are a settled matter. In this article, we argue that important normative dimensions are overlooked with traditional mean-based measures, and that metrics that capture students’ academic mobility as they progress through school can provide the informational base needed to describe and evaluate these policy and socially relevant aspects. We discuss some key normative principles and methodological dimensions related to academic mobility and provide an empirical example of the mobility metrics presented using a nationally representative data set.

Keywords

academic mobility educational inequality achievement gap

The idea that all individuals should have equality of opportunity regardless of their social origin is a fundamental principle in modern democratic societies. In addition, it is commonly believed that education plays a key role in achieving this goal, as schooling processes should provide equal opportunities to all individuals, allowing them to move up or down the social ladder based on their efforts and accomplishments, rather than their family background and other factors that lie beyond the individuals’ control. Academic and policy discussions on education regularly examine educational inequalities, often assuming that these normative¹ foundations, as well as the metrics appropriate to measure educational systems on these dimensions, are a settled matter. In particular, within-nation analyses on educational inequality have been to a large extent dedicated to measuring the so-called “achievement gaps,” which refer to the differences in mean academic performance between different groups of students (defined, e.g., by race, gender, or socioeconomic status). The implicit normative implication seems straightforward: The larger the gap, the less equality of opportunity among groups of individuals. Discussing or making explicit these normative assumptions can be regarded as stating the obvious, and as a consequence many researchers focus instead on methodological variations of measuring achievement gaps, which are assumed to be the main metric of educational inequality.

However, the normative implications related to the concept of equality of opportunity in general, and the concept of equality of educational opportunity in particular, are far from settled. For a long time, researchers working on issues of justice and inequality have called attention to the tensions and ambiguities associated with egalitarian ideals. These discussions have been held primarily by philosophers (e.g., Anderson, 1999; Cohen, 1989; Rawls, 1971) and economists working on issues of inequality (e.g., Roemer, 1998; Roemer & Trannoy, 2015; Sen, 2009). Researchers in the field of education (e.g., Coleman, 1968; Jencks, 1988) have also distinguished different meanings associated with the concept of educational opportunity and noted some prominent dilemmas that arise from this ideal. This concept can be understood in different ways, implying different ideals of equality and opportunity, and therefore different metrics to examine the distribution of educational outcomes. These underlying conceptual disagreements, as well as their methodological and practical implications, might not be immediately evident to many researchers and policy makers in the field of education. As Jencks (1988, p. 518) puts it, “the enduring popularity of equal educational opportunity probably derives from the fact that we can all define it in different ways without realizing how profound our differences really are.”

A central idea underlying the previous discussion is that, as many researchers in the field of economic inequality have recognized, measurement practices are inevitably entangled with normative considerations. Several scholars have argued that, when dealing with social science theories (which normally involve thick ethical concepts such as fairness and inequality), one cannot separate a purely descriptive part and a purely evaluative part, as the researcher simultaneously describes and evaluates (e.g., Anderson, 2004; Putnam, 2002; Schumpeter, 1949).² Furthermore, as Putnam (2002, p. 112) notes, there are facts “which only come into view through the lenses of an evaluative outlook.” What this implies, then, is that social science theories should be regarded as an inextricable combination of a normative framework and a measurement model, and that both of these components should be explicitly stated and justified (e.g., Anderson, 2004; Feuer, 2015). In the case of educational inequality, this implies answering a series of related questions, notably: What normative principles should we adopt to describe and evaluate the state of our educational systems? And, in view of these principles, what features of the distribution of educational outcomes become important to measure, and how can we measure them?

In this article, we argue that the notion of achievement gaps provides a limited informational base to assess educational inequality, and that the concept of academic mobility, which refers to changes in individuals’ rank position over time, can help us overcome some of these limitations. Policy and socially relevant aspects that are not captured by average-based differences include (1) achievement patterns at particular segments of the achievement distribution (e.g., the upper and lower ends), (2) existing gaps and distinct mobility patterns within groups, and (3) differences and similarities in patterns of within-person variation. Additional limitations include a restricted generalizability (as they are only interpretable by reference to another group), and susceptibility to problems of scale. We will explain these points in detail and show how academic mobility metrics can help us overcome these limitations.

The objective of the article is to discuss appropriate ways of describing and evaluating inequalities in an education system. The purpose then is not to explain why educational inequalities occur, or to identify factors that predict differences in educational outcomes. In other words, the aim of the present study is entirely descriptive (rather than, e.g., explanatory or confirmatory). In addition, it is worth noting that the “education system” we refer to in our theoretical and empirical sections is the national education system. The academic mobility metrics that we present can belong then to what Reeves (2015) calls the “measure of a nation.” At the same time, these metrics can be used to describe and evaluate other systems (e.g., particular schools), and they can be adapted to account for hierarchical structures (e.g., patterns of educational inequalities within and between classroom, schools, districts, etc.). However, for the purpose of simplification we focus on the national education system monolithically conceived.

Finally, our objective is to examine educational inequalities in skill development, and our empirical application considers reading achievement. Most of the empirical research on educational inequality has focused on access and completion (e.g., Lucas, 2001; Bailey & Dynarski, 2011), school and college transitions (e.g., Breen & Jonsson, 2000; Page & Scott-Clayton, 2016), track or curriculum stratification (e.g., Lucas & Berends, 2002; Van de Werfhorst & Mijs, 2010), and intergenerational mobility (e.g., Breen & Jonsson, 2005). Researchers have also investigated inequalities related to learning outcomes (generally recognized as an important—or even the main—outcome of educational processes), mostly by conducting achievement gap analyses. In this article, we reflect on the appropriateness and limitations of achievement gaps and other measures commonly used to examine learning outcomes to describe and evaluate educational inequalities in skill development.

The article is structured as follows. First, we argue that academic mobility provides an adequate normative framework for describing and evaluating educational inequalities. Second, we present methodological considerations along with different metrics that are appropriate for measuring academic mobility. Finally, using a nationally representative sample, we provide an empirical example of the metrics by conducting a comparative analysis of academic mobility based on racial groups.

The Normative Dimensions of Academic Mobility

Defining Equality of Opportunity

Egalitarian concerns have been at the center of modern social and political debates since John Rawls published his Theory of Justice in 1971. In his groundbreaking book, Rawls challenged the dominant ethical view of utilitarianism and argued that, in a just society, individuals should be treated as equals in some specific respects: Everyone should have equal basic liberties and fair equality of opportunity (Rawls, 1971). The precise meaning and acceptability of these principles of justice have been widely discussed in the literature (see, e.g., Anderson, 1999; Cohen, 1989; Roemer, 1998). However, most researchers working in the egalitarian tradition agree on a fundamental point, namely that one should distinguish between morally acceptable and unacceptable forms of inequality (Roemer & Trannoy, 2015). In particular, many authors have moved from expecting equality of outcomes to expecting equality of opportunities, generally defined as “chance[s] of getting a good if one seeks it” (Arneson, 1989, p. 85). The rationale for this idea is that while achieving equality of outcomes would presumably imply severe limitations on individual liberties, equality of opportunity serves the egalitarian ideal and at the same time preserves individual freedom and responsibility (Cohen, 1989).

Theories of equality of opportunity have generally tried to specify which are the tolerable and the intolerable sources of inequality. In an influential account, Roemer (1998) differentiates between “circumstances” (i.e., exogenous factors that lie beyond the individual’s responsibility such as race or gender) and “accountable effort” (i.e., endogenous factors deriving from the individual’s choices), and argues that only the inequalities deriving from accountable effort should be tolerated. Even though this approach provides a powerful framework for defining and measuring educational inequalities, it also faces considerable challenges (e.g., Kanbur & Wagstaff, 2016). For example, it is unlikely that researchers can identify and measure all relevant circumstances, and it is not clear at what point we should begin attributing disparities in outcomes to differences in accountable effort (as differences among infants and young children mainly reflect the parents’ circumstances and effort; see, e.g., Bodovski & Farkas, 2008; Lareau, 2003).

These complications point to the difficulty of articulating a positive theory of justice in general, and of equality of opportunity in particular (see, e.g., Jencks, 1998). Providing a normative framework that establishes in a clear-cut fashion the ideal distribution of resources based on individuals’ circumstances and personal choices (or other exogenous and endogenous factors) may be both theoretically and practically unattainable. In order to overcome this conundrum, Sen (2009) argues in favor of a “relational” approach to justice (in opposition to the “transcendental” view espoused by many egalitarians), according to which the aims of social justice should consist in removing the manifest injustices rather than in conceiving an abstract ideal of justice. That is, rather than advancing a positive theory of justice, we should focus on the negative aims of egalitarian justice (Anderson, 1999). By assigning epistemological priority to social inequalities, the central goal becomes to bring attention to injustices which any reasonable egalitarian theory would condemn. As Sen explains, instead of speculating on what a perfectly just society would look like, a relational approach produces comparative assessments intended to identify clear cases of social asymmetry and discrimination. One advantage of this approach is that, in contrast to the distant ideals offered by the transcendental tradition, the identification of clear cases of manifest injustice can lead to more targeted policy interventions, as well as a clearer diagnosis of the current situation.

The goal of this article is to reflect on the comparative assessments that are appropriate for describing and evaluating educational inequalities, and which might allow us to identify cases of manifest injustice. The literature on educational inequality has mostly operated under the achievement gaps paradigm, comparing averaged-based measures between groups at different stages of development and using a variety of metrics and research designs (see Reardon, Robinson-Cimpian, & Weathers, 2015, for a survey of the achievement gap literature). Frequently, however, researchers do not explicitly state their normative assumptions and implications for their analyses. Value judgments regarding features of the educational system that can be considered fair or unfair are often embedded in the methodological sections and are not explicitly recognized and justified. Thus, it is not clear how to interpret—from an ethical standpoint—achievement differences between groups, and what other ethically relevant aspects are not captured by analyses within this paradigm.

In order to shed light on these issues, we discuss key normative principles that can help us distinguish morally acceptable and unacceptable forms of educational inequality. Similar to other theories of equality of opportunity, we start from the premise that a completely unequal society is one in which circumstances (e.g., race or socioeconomic status) completely determine individuals’ outcomes. Unlike many of those theories, however, we espouse neither a particular theory of justice nor a particular distributional ideal. Instead, we adopt Sen’s relational view of justice, and focus on comparative assessments to help us identify manifest disadvantages. Identifying these disadvantages requires moving beyond average-based differences between groups and implementing fine-grained comparative analyses of how students traverse through the achievement distribution over time. We argue, then, that the concept of academic mobility can provide an appropriate normative framework for describing and evaluating educational inequalities, as well as offering a suitable operationalization of our intuitive ideal of “equality of opportunity.”³

Measuring Educational Inequality: Normative Considerations

In this section, we argue that the concept of academic mobility, which has not been utilized in educational research (with very few exceptions, notably Feinstein, 2003; Kerckhoff, 1993; McDonough, 2015; and Sohn, 2012), provides a valuable normative framework for evaluating and describing potential inequalities within education systems. We support this claim by showing how the concept of academic mobility is sensitive to particular forms of inequality that can be regarded as more concerning and unacceptable than others. In particular, we discuss the normative implications of two well-established dichotomies in the literature: between versus within-person variation, and relative versus absolute change.

Between Versus Within-Person Variation

Many researchers and practitioners in the field of education recognize that educational processes are better described by metrics that capture students’ performance over time rather than “status” or cross-sectional measures. That is, instead of focusing on the differences in performance of an individual (or group of individuals) at a single point in time (commonly referred to in the education literature as “gaps”), one needs to consider how these differences change (or do not change) over time. An important normative reason behind this perspective is that, in order to examine educational inequality, one needs to understand to what extent inequalities increase or decrease throughout the schooling process, above and beyond preexisting differences. Cross-sectional or “status” measures are, then, inherently biased, as they reflect both schooling effects (or, more generally, inequalities that increase or decrease throughout schooling) as well as preexisting inequalities.

For purposes of describing and evaluating educational inequalities, it is important to examine, then, the magnitude and direction of individuals’ deviation from their starting position. The average amplitude of these changes in the population would provide a measure of the overall degree of mobility (or stratification) in the education process (or, put differently, the degree to which individuals’ final outcome is determined by their initial position). For example, a system where all individuals remain in the position that they started in throughout the schooling process would be completely stratified, whereas a system in which the individuals’ initial and final positions are fully uncorrelated will have total mobility. Furthermore, given that these measures control for the individuals’ initial status, differential patterns of mobility (e.g., groups of individuals who tend to move downward) can be particularly concerning and could indicate clear cases of social asymmetry and discrimination.

The variability describing individuals’ deviation from their own baseline levels, regardless of the baseline values per se, is generally referred to in the literature as within-person (or intraindividual) variation (e.g., Hoffman, 2015). On the other hand, the variability describing differences between individuals, independently of the degree to which the individuals themselves vary over time, is generally referred to as between-person (or interindividual) variation. Researchers have long argued that we should clearly distinguish between these two kinds of variation, as they represent different phenomena, and inferences from one level of analysis to the other are seldom warranted (e.g., Hamaker, 2012; Molenaar, 2004). In particular, group generalization can obscure genuine individual differences (Fisher, Medaglia, & Jeronimus, 2018), and group-derived estimates can misrepresent individual-level processes (commonly referred to as the “ecological fallacy”; e.g., Curran & Bauer, 2011). For example, based on six studies with a repeated-measure design, Fisher et al. (2018) found that the variance within individuals was two to four times larger than the variance within groups. This suggests that we might fail to identify important differences in individuals’ variation if we rely exclusively on group-level estimates.

Many researchers have defended the priority of within-person variation for adequately describing processes of change (e.g., Hamaker, 2012; Nesselroade & Molenaar, 2010). As explained above, examining differences in intraindividual variability (i.e., the degree to which individuals deviate from their initial position), is also essential for descriptive efforts related to educational inequality. The average-based differences commonly used in the achievement gap literature represent useful metrics for summarizing achievement disparities between groups. However, mean-based measures do not capture patterns of intraindividual variation, and as a consequence can obscure important individual-level processes. A comprehensive description of the development of educational inequalities should address within- and between-level phenomena and explain how processes that take place at the within-person level differ across individuals (and groups of individuals). Some of the academic mobility metrics that we discuss below are sensitive to within-person variation, while others to between-person variation; a comprehensive application of different metrics can yield, then, a better understanding of the development of individual and group differences in academic achievement.

Relative Versus Absolute Measures of Change

In the context of educational processes, the dichotomy between absolute and relative change is based on a fundamental difference between the properties that are being measured: While relative measures refer to relational properties, for example, percentile scores, absolute measures refer to intrinsic properties, for example, a particular ability $θ$ . This distinction mirrors the difference in the psychometric literature between norm-referenced reporting, in which student’s performance is described relative to a reference group (i.e., in terms of their position with respect to the group), and criterion-referenced reporting, in which student’s performance is described according to some fixed level of performance, independently of their position in the group (Hamilton, 2003). Since Glaser (1963) argued in favor of criterion-referenced reporting for describing students’ achievement, this kind of reporting has become increasingly common in education, and performance standards have been the cornerstone of educational reform (Hamilton, 2003; Shepard, Hannaway, & Baker, 2009).

“Pure” measures of absolute change are provided by studies assessing growth trends in particular competencies using vertically scaled assessments. For example, using nationally representative assessments, Hill, Bloom, Black, and Lipsey (2008) computed the difference of mean scale scores in adjacent grades, in order to characterize the natural developmental progress in achievement made by students from one year to the next. According to the authors, these differences can serve as normative expectations and benchmarks for achievement effect sizes in educational interventions. This goal is justified, inasmuch as one is interested in providing a measure of pure absolute change, that is, a description of “natural” (in the sense of “shared” or “structural”) growth, with normative implications that emerge from these common stages of development.

For many purposes it is useful to use absolute measures, as researchers and practitioners are frequently interested in the knowledge, skills, and dispositions that children have acquired at particular moments in their education (see Pallas, 2000 for a review on the many valuable nonpositional dimensions of educational outcomes). It might also be important to attach normative implications to absolute measures, for example, by defining proficiency cut scores. However, even though absolute measures can provide the foundation for certain normative claims, in order to measure educational inequality it is essential to consider the relational or positional aspects of academic achievement.

A good has a positional dimension if an individual’s relative position in the distribution of the good affects the value of the good (Brighouse & Swift, 2006). Education is often considered a paradigmatic case of a positional good, as the value of one’s education partly depends on how well-educated other people are (Anderson, 2007; Brighouse & Swift, 2006; Nikolaev, 2016; Solnick & Hemenway, 1998). Health outcomes, in contrast, are often considered nonpositional goods, as the conception of one’s health is largely independent of the health of others. As Brighouse and Swift (2006) explain, the positional aspect of education is derived from its competitive component: our K–12 education system is instrumental in accessing a range of valuable goods (e.g., a good higher education, an interesting job, etc.), and our competitive success in these markets depends to a large extent on our relative position in the overall distribution. Furthermore, schooling processes play a critical role in sorting individuals into different positions, exacerbating or compensating social inequalities, and shaping students’ identities as well as their future educational and noneducational experiences (see, e.g., Domina, Penner, & Penner, 2017; Downey & Condron, 2016).

Acknowledging the positional aspect of academic achievement has important implications for the study of educational inequality. Primarily, it implies that the value of one’s education is not only determined by the specific skills and knowledge one has acquired (or other nonpositional aspects), but also by one’s relative position in the overall distribution. In addition, different normative concerns apply to different parts of the distribution. Following Rawls’s (1971) theory of justice, priority should be given to promoting the opportunity of the worst-off group. That is, analyses of educational inequality should focus on the lowest segments of the distribution, and examine to what extent some groups face particular barriers that prevent them from moving up, and persist in disadvantage. At the same time, other important normative concerns apply at the top of the distribution. Following Anderson (2007), one can argue that democratic elites should be drawn from all sectors of society, including the less advantaged. Analyses of educational inequality should examine, then, to what extent some groups are underrepresented at the top of the distribution (which comprises the individuals who have higher chances of getting accepted to the best higher education schools and the highest positions).

Defining Academic Mobility

We have argued that in order to more precisely examine educational inequality, researchers should consider metrics that fulfill three key conditions. First, the measures used need to reflect how the initial differences between individuals (or groups of individuals) change over time. Second, these metrics should capture individual and group differences in intraindividual variability. Third, these metrics need to be sensitive to the individuals’ relative position in the overall distribution, rather than to absolute properties reflecting natural or shared growth. We will refer to the processes described by these metrics as “academic mobility.” In a nutshell, these metrics describe the change in individuals’ relative position in the overall achievement distribution over time.⁴

Based on the previous discussion, one can identify additional desirable features of metrics describing academic mobility. First, one should be able to identify mobility patterns at different segments of the academic distribution, and in particular at the extremes of the distribution—this is due to the importance of monitoring the composition of democratic elites (i.e., the top of the distribution) as well as the worst-off group (i.e., the bottom of the distribution). Second, one should be able to make comparisons across time and between different groups and populations. This is supported by the relational theory of justice and opportunity presented above, in which investigations of inequality should be regarded as an essentially comparative exercise intended to identify cases of social asymmetry and discrimination. Third, given that the ultimate goal is to provide a clear diagnostic of a relevant issue, as well as helping policy makers make informed decisions for improving educational equality, the metrics should be transparent and easy to understand. This is related to the intended use of measurement procedures, and the key role of consequential validity (see, e.g., Shepard, 1997).⁵ Finally, it is worth noting that given that academic mobility is multidimensional, one should consider different metrics that are sensitive to different aspects of mobility. In the next section, we will describe the methodological aspects that allow mobility metrics to fulfill these conditions.

Metrics of Academic Mobility

Measuring Educational Inequality: Methodological Considerations

Using Growth Models to Measure Relative Change

Even though in theory one can differentiate between pure measures of relative and absolute change, in practice many measures combine in intricate ways the two dimensions. It is useful, however, to keep in mind this dichotomy, as it represents different kinds of processes with distinct normative implications, which require different measures and modelling strategies. The objective of absolute measures is to quantify the magnitude of change of a particular construct over time, generally referred to in the educational literature as “growth.” The construct of interest is assumed to follow a common developmental trend, but some individuals “grow” faster or slower than others. Given that all individuals follow a common developmental pattern, it is sensible to define a “normal” growth process, as well as “abnormal” departures from the common trend. A classic example of absolute measures of change are the height and weight charts used by pediatricians to monitor infant health and development. As this example illustrates, absolute measures have the following properties: (1) they assume that most individuals grow, albeit at different rates (i.e., there is a “main effect” of time); (2) they support normative claims related to what is “normal” or “abnormal” at a particular developmental stage (e.g., a normal height for a 5-year-old might be abnormal for a 10-year-old); and (3) they require an underlying score scale with interval properties (e.g., the difference between 30 and 40 inches is the same as the difference between 40 and 50 inches).

The purpose of relative measures, on the other hand, is to describe the change in individuals’ position in some particular distribution over time. Contrary to absolute measures, relative measures (1) do not describe common developmental patterns (or growth), and are thus unconcerned about any “main effects” of time; (2) do not support normative claims regarding what is “normal” or “abnormal,” but rather about the advance or decline of equality and justice; and (3) do not have to rely on interval scales, as the purpose is not to quantify the magnitude of growth but to describe changes in the individuals’ relative position (which only requires an ordinal scale).

Shared or structural development processes can be adequately described by traditional growth models (e.g., hierarchical models or latent growth curve models), as in this case one can assume that all individuals grow according to the same function but their growth varies in magnitude (Raudenbush, 2001). In other words, these models are appropriate for measuring absolute change, as they assume that all individuals in the population follow a similar functional form of development, and that the variance of the growth factors are sufficient to capture interindividual differences in change across time (Nagin, 2005).

Traditional growth models can also be used to measure relative change, for example by examining the intercept-slope covariance (see, e.g., Pfost, Hattie, Dörfler, & Artelt, 2014). However, traditional growth models are better suited for describing absolute processes of change, and the inferences one can make regarding mobility are limited. The reason for this is that relative change does not represent an aggregate process that can be described by a common function, but rather a mixture of movements in the rank ordering of individuals, net of any common developmental process (i.e., of any main effect of time). Relative processes imply that if someone moves up in the rank order, then another person needs to move down; consequently, one cannot describe relative changes by a common developmental function, as these changes involve opposite developmental patterns, and the common function describes the average change. In other words, absolute measures are not necessarily sensitive to mobility patterns in the population, as the average gains can remain the same independently if there is high or low mobility. Furthermore, mobility studies are often interested in obtaining descriptions of the movement occurring in the edges of the distribution, whereas traditional growth models are well suited for describing average trends.

Traditional growth models quantify the magnitude of change by relying on a measurement scale with interval properties (i.e., they require vertically scaled assessments). However, in the context of skill development this assumption is often untenable (Briggs & Betebenner, 2009; Ho & Haertel, 2006). In addition, as some studies have demonstrated (Bond & Lang, 2013; Ho & Haertel, 2006), the estimated statistics can change dramatically and even change sign under some monotonic (i.e., order-preserving) transformations—which could represent the “true” underlying scale. In order to deal with this issue, various modelling strategies have been proposed that do not require vertically scaled assessments (see, e.g., Castellano & Ho, 2013).⁶

Using Achievement Gaps to Measure Relative Change

Educational research and policy have focused primarily on student growth—which is demonstrated by the emphasis on criterion-referencing reporting and the extensive use of growth models of different kinds. On the other hand, the most common measures employed in the literature to assess relative change (i.e., the change in individuals’ position over time) are the average-based differences coming from the achievement gap literature (e.g., Fryer & Levitt, 2004; Reardon & Galindo, 2009).⁷ These studies normally use standardized score differences, obtained by dividing the difference in mean achievement scores between two groups by the pooled standard deviation of those groups. These measures have the advantage that they are easily computed and interpretable.

Average-based measures, however, have the following limitations. First, by standardizing the mean differences, one confounds the difference in means with the variation within groups (Reardon & Galindo, 2009). Second, in most applications average-based measures depend on the interval properties of the scale for the original score (which, as indicated above, is often an untenable assumption). Third, average-based measures do not take into account the individuals’ position in the overall distribution (see Figure 1, Patterns A and B). However, as noted above, it is not only important to consider the mean difference in achievement between two groups but also the position of the individuals or group of individuals in the overall distribution. In particular, one should be able to conduct fine-grained analyses at different segments of the achievement distribution, and examine the extent to which some subgroups are more or less mobile (e.g., at the extremes of the distribution; see Figure 1, Patterns D and E).

Figure 1.

Six patterns representing the development of inter and intraindividual differences among four individuals belonging to two different groups.

The fourth limitation of average-based measures is that, given that they are mainly sensitive to between-person variation, they provide very little information regarding differences in within-person variability (see Figure 1, Pattern C). We know, however, that there is considerable heterogeneity within racial groups (see, e.g., Davis-Kean & Jager, 2014). Thus, it is important to consider more fine-grained analyses of change within groups. For this purpose, it is beneficial to move from the “changes-in-gaps” paradigm, based on changes in differences in group means, to a within-person study of academic mobility, which describes who moves, by how much, from where, and when does the movement occur.

Finally, average-based differences have a limited generalizability, as they are only interpretable by reference to another group. However, one might be interested in knowing the degree of academic mobility across all groups or at the population level (e.g., in order to conduct historical comparisons within countries, or between-country comparisons).

Mobility Metrics and Their Advantages for Studying Relative Change

The previous discussion indicates that the majority of studies in education have focused on absolute measures of change (or, more generally, measures describing student growth), limiting their ability to identify patterns of relative change. On the other hand, studies more explicitly interested in relative change have used measures that have important disadvantages or limitations. In the next section, we present several mobility metrics that have been predominantly used in other domains (particularly in studies on income mobility and personality psychology), and which can be readily adopted to measure academic mobility. In contrast to the measures of change described above, these metrics exhibit important advantages. First, they are only sensitive to changes in the rank ordering of individuals, and are therefore unaffected by structural growth processes. This implies that the metrics are comparable across time (i.e., they are “intertemporally scale invariant”). This also implies that the results do not rely on—or, depending on the metric, are less conditioned by—functional form assumptions, which have become a hindrance for modelling developmental trajectories, as researchers try to find the best shape (e.g., complex polynomial or nonlinear trajectories; see Cameron, Grimm, Steele, Castro-Schilo, & Grissmer, 2015; Hoffman, 2015) that fit the observed patterns of change. Second, some of the measures are “metric-free” or “scale-invariant,” meaning that they are not altered by any monotonic transformation in the values of achievement, as well as changes in the marginal distributions. Thus, the metrics are comparable across assessments. Third, some metrics are sensitive to within-person variation and others to between-person variation, offering researchers different metrics that provide a comprehensive picture of the development of individual differences in achievement. Fourth, the metrics are easily computable and interpretable, which are key aspects for consequential validity.

Dimensions of Academic Mobility

Academic mobility is multifaceted, and the adequate measure depends on one’s normative objective. Based on the normative considerations presented in the first section, some fundamental questions are the following: (1) What is the overall degree of mobility of a particular education system? (2) How mobile or persistent are the individuals at the bottom and at the top of the distribution? (3) Are there differential mobility rates across groups (e.g., racial groups)? (4) How do mobility rates change throughout schooling? (5) To what extent does schooling serve as an equalizer?⁸ Below we present several metrics that are adequate to measure each of these dimensions.

Data

In order to provide an empirical example of the mobility metrics presented below, we use the Early Childhood Longitudinal Study kindergarten cohort (ECLS-K), which is a nationally representative sample of 21,409 American children who entered kindergarten in 1998 (see, Tourangeau et al., 2009, for more information regarding this study). The ECLS-K study followed these children from fall of kindergarten to spring of eighth grade in 2007, providing a comprehensive picture of children’s academic development until secondary school.

The choice of this data set is driven by two considerations. First, it is to date the nationally representative sample that covers the longest time period in student’s schooling experience since kindergarten. Second, there have been a large number of studies using this data set, so one can identify the consistencies with previous results as well as inferences regarding the development of inequalities that could not have been made using traditional measures (including both measures of growth as well as those attempting to examine relative change).

In the present study, we used students’ reading achievement as the main dependent variable. In particular, we used student’s percentile rank in the overall distribution in six waves: fall-kindergarten; spring-kindergarten; spring–first grade; spring–third grade; spring–fifth grade; and spring–eighth grade. The appropriate longitudinal weight provided by ECLS was used in all calculations. The use of this weight allows us to (1) provide population estimates, (2) adjust for differential selection (e.g., oversampling), and (3) reduce bias associated with missing data. The percentile ranks were created using only the analytic subset (i.e., considering individuals with a nonzero and nonmissing weight). Table A1 in the Appendix presents descriptive statistics of our main outcome disaggregated by race. In order to obtain adequate standard errors for the estimates, we employed in each computation the paired jackknife method utilizing appropriate replicate weights provided by ECLS.

We use students’ percentile rank scores in order to stress the importance of considering individuals’ relative position in the overall distribution (e.g., the top or bottom 25%). In addition, given that these scores are purely relational, they are not affected by any structural growth processes (i.e., the distribution is stationary). Importantly, however, even if some of the metrics presented do not rely on parametric assumptions (in particular transition probabilities), the other metrics can assume linearity or can be affected by the distributional form. Consequently, studies using these metrics should test their results using other scores (e.g., T or theta-scores). A full exploration of the sensitivity of these metrics to various parametric assumptions is beyond the scope of this study.

Linear rank-rank measures

A commonly used measure of mobility is obtained using a linear rank-based approach (see Chetty, Hendren, Jones, & Porter, 2018; Chetty, Hendren, Kline, & Saez, 2014), which provides a parsimonious summary of the overall degree of mobility at the population and subpopulation levels. In this approach, mobility estimates are obtained by regressing the child’s rank in the national distribution at some time point on his or her rank at a previous time point. In the present application, this can be expressed as

R_{i, 8} = α_{i} + β_{i} R_{i, k} + ε_{i},

(1)

where $R_{i 8}$ represents the achievement percentile rank of individual i relative to all other individuals in eighth grade, and $R_{i, k}$ the percentile rank of the same individual in kindergarten. By combining the two parameters describing the linear function ( $α_{i}$ and $β_{i}$ ), one can estimate the expected rank of children in eighth grade based on their rank at the national level in kindergarten. The intercept $α_{i}$ can be used to measure the overall direction of academic mobility across subgroups (either upward or downward mobility), and the slope can be used to measure the degree of academic mobility at the population or subpopulation levels (e.g., if $β$ = 1, then the expected mobility is zero, as the percentile rank in kindergarten would be a perfect predictor of the percentile rank in eighth grade; on the other hand, if $β$ is close to zero, then there is high relative mobility, as the relative position in kindergarten would not be a strong predictor of the children’s relative position in eighth grade).

Provided that the relationship between children’s mean ranks in eighth grade and their ranks in kindergarten is well approximated by a linear function, this approach has several advantages: (1) it provides a small set of statistics capturing positional mobility estimates at the population and subpopulation levels, (2) the estimated statistics control for any structural patterns in growth (given that the marginal distribution for both $R_{k}$ and $R_{8}$ are uniform distributions), and (3) as Mazumder (2015) remarks, by using percentile ranks at the population level this specification can be used to measure mobility differences across subgroups with respect to the national distribution (while using scale scores would allow us to estimate mobility differences with respect to the subgroup mean).

Measures of the amplitude of academic mobility

The mobility estimates obtained using the rank-rank approach only consider the mobility between two time periods, and do not distinguish between and within-person changes. In this section, we present two raw-score metrics for quantifying the amplitude of within-person mobility: the intraindividual standard deviation, representing the mean intraindividual mobility; and the means square successive difference, representing the mean intraindividual mobility from one time point to the next (Wang, Hamaker, & Bergeman, 2012). These indicators can provide valuable information regarding the magnitude, direction, and timing of intraindividual mobility. As explained above, these individual-level processes can be obscured or misrepresented by group-level estimates.

The intraindividual standard deviation (ISD) is often used in psychology as a measure of intraindividual variability (Wang et al., 2012). In the current application, it is computed as follows:

IS D_{i} = \sqrt{\frac{\sum {(R_{i, t} - R_{i})}^{2}}{T_{i} - 1}},

(2)

where $R_{i, t}$ represents the rank score of individual i at occasion t; and $R_{i}$ represents the mean rank of individual i over the number of measurement occasions of that individual ( $T_{i})$ ). The $IS D_{i}$ is interpreted as the average mobility of individuals across the entire period (net of any systematic growth over time).

Another useful statistical indicator for quantifying intraindividual variability is the means square successive difference (MSSD), which measures the mean occasion-to-occasion mobility (Jahng, Wood, & Trull, 2008). Following Jahng et al. (2008), this statistic can be computed as follows:

MSS D_{i} = \frac{1}{T_{i} - 1} \sum_{t = 1}^{T_{i} - 1} {(R_{i, t + 1} - R_{i, t})}^{2},

(3)

where $R_{i, t}$ represents the rank score of individual i at occasion t, and $R_{i, t + 1}$ represents the next measurement occasion for the same individual. As Wang et al. (2012) explain, the MSSD measures both the amplitude of mobility (i.e., the ISD), as well as time dependencies, that is, the extent to which achievement at one point is determined by previous achievement. This metric can be further adapted in order to obtain more fine-grained information; for example, the amount of occasion-to-occasion mobility across groups in the upward or downward direction.⁹

Transition probabilities

One way of obtaining more fine-grained and easily interpretable estimates of directional mobilities is by using transition probabilities. These metrics measure the probability that a particular group of children will finish in a certain position conditional on their original rank. Thus, these metrics are commonly regarded as measuring origin independence (or dependence), as they indicate the extent to which children’s destination is related to their original position.

A common way of gauging positional movement is by constructing a “transition” or “mobility” matrix, which classifies individuals according to fixed and equal-sized categories (e.g., quartiles), with initial-period categories determining the row and final-period categories determining the column (Fields, 2006). As Fields (2006) explains, if the system is perfectly stable, then all the values will lie along the principal diagonal, and thus, the mobility matrix would be an identity matrix. On the other hand, assuming a quartile partition, in a system with complete mobility 25% of the values in each initial quartile will be placed in each final quartile.¹⁰

Using the 1998–1999 ECLS-K, McDonough (2015) used transition matrices to estimate the staying probabilities and directional rank mobilities of academic achievement for Black and White students. In this study, we present one kind of transition probabilities not covered by McDonough (2015), and which have important normative implications. These statistics measure the probability of ending up in the opposite side of the distribution, and are thus called “rank reversal” probabilities (Jäntti & Jenkins, 2013). In particular, we consider the two “extreme” rank reversal probabilities: the chances of ending in the highest quartile after beginning in the lowest quartile (the upward rank reversal), and the chances of ending in the lowest quartile after beginning in the highest quartile (the downward rank reversal). The upward rank reversal (URR) probability can be represented as follows:

UR R_{i} = \Pr (R_{i, 8} > 75 th percentile | R_{i, K} \leq 25 th percentile)

(4)

and the downward rank reversal (DRR) probability as follows:

DR R_{i} = \Pr (R_{i, 8} < 25 th percentile | R_{i, K} \geq 75 th percentile) .

(5)

These extreme forms of positional mobility reveal complete origin independence, and can serve useful normative purposes. The URR statistic is a rough indicator of a meritocratic society, as it represents the probability of rising from the bottom to the top quartile (thus, Chetty et al., 2014, refer to this probability as the “American dream statistic”). On the other hand, the DRR indicates a society that allows severe setbacks, where students who begin among the highest achieving in the nation end up at the bottom of the distribution. It is worth noting that these “corner probabilities” are usually undetectable using average-based measures.

Measuring stability and change

Most of the mobility metrics presented so far consider positional change between the origin and the destination. In order to model this positional change as well as temporal dependencies simultaneously, one can estimate the percent of the variance that (1) remains stable, (2) changes systematically over time, or (3) is due to idiosyncratic circumstances, using the stable trait, autoregressive trait, and state model (START) presented by Kenny and Zautra (1995, 2001). This model captures both time-invariant and time-varying dimensions of positional mobility, by disentangling the longitudinal structure of achievement in terms of a completely stable (or “trait”) factor, and a systematically varying (or “state”) factor, as well as an idiosyncratic (or error) component.¹¹ Figure A1 in the Appendix presents a path diagram of this model. As Newsom (2015) explains, the START model can be described using two equations. The first equation indicates that each observed measure of achievement is a function of three sources of variance:

Var (R_{it}) = Var (η) + λ_{it}^{2} (η_{t}) + Var (ε_{it})

(6)

where $Var (η)$ represents completely stable variance; $Var (η_{t})$ , referred to as state variance, represents systematic variance that changes over time; and $Var (ε_{it})$ , referred to as idiosyncratic variance, represents unaccounted variance in the observed outcome. The second equation indicates that each occasion-specific state factor is a function of the prior occasion and unaccounted variance:

Var (η_{t}) = β_{t, t - 1}^{2} Var (η_{t - 1}) + Var (ζ_{t}) .

(7)

In this study, we focus primarily on the three parameters included in Equation (6). First, we consider the variance of the stable factor, which indicates the degree of stability in academic achievement at the population and subpopulation levels. Second, we consider the variance of the state factors, which capture statistically predictable change in the rank ordering of individuals. If there is a high degree of systematic academic mobility, then state factors should account for a substantial amount of variance. Finally, we obtain an error term, which represents the idiosyncratic (or nonsystematic) variance that is unexplained by either the stable factor or the previous state.

Measuring group differences over time

The final mobility metric gauges the extent to which inequalities between groups increase or decrease over time. If educational processes serve as an equalizer—and all additional conditions remain the same—, then one can expect that the initial differences between groups will diminish over time; if, on the contrary, educational processes aggravate group disparities, then one can expect an increasing differentiation between groups. A simple way of estimating equalization (or disequalization) over time is, then, to compare the ratio of the between-group variance with the total amount of variance. A measure of this ratio can be obtained using the familiar coefficient of determination, or R², which is computed as

R^{2} = \frac{\sum {({\bar{R}}_{r} - \bar{R})}^{2}}{\sum {(R_{ir} - \bar{R})}^{2}},

(8)

where ${\bar{R}}_{r}$ represent the mean achievement percentile rank of individuals in race r; $\bar{R}$ represents the average achievement in the population; and $R_{ir}$ represent the achievement percentile rank of individual i in race r. This ratio can be interpreted as the amount of variance that is explained exclusively by racial differences. We examine the dynamics of group differentiation by comparing this cross-sectional metric over time. Similar to achievement gap measures, this metric is more sensitive to between-group (rather than within-person) differences. However, this metric measures the overall effect of group-belonging (e.g., race), rather than differences between two particular groups.

Estimates of Academic Mobility Across Racial Groups in the United States

Overall Degrees of Academic Mobility

We began by estimating the positional mobility for the entire population using Equation (1). Following Chetty et al. (2014) and Chetty et al. (2018), we plotted this relationship with a binned scatter plot, in which we divided the horizontal axis into 100 equal-sized bins, and then plotted the mean rank in eighth grade versus the mean rank in kindergarten in each bin. Figure 2 shows that this relationship can be well approximated by a linear function. The results indicate that, on average, a 10-percentile difference in children’s rank-order in kindergarten is associated with a 5.4 percentile difference in the rank-order in eighth grade. Even though this estimate can be compared with other outcomes (e.g., the estimated intergeneration family income rank-rank slope in the United States is 0.341; see Chetty et al., 2014), it would be more meaningful to compare it with the positional academic mobility of the same educational system at previous time points, or with other educational systems.

Figure 2.

Scatter plots representing the relationship between students’ achievement rank in eighth grade and students’ achievement rank in kindergarten (Panel A) and the relationship between students’ rank achievement in eighth grade and in kindergarten by race (Panel B).

Subsequently, we estimated rank-mobility parameters ( $α_{i}$ , $β_{i}$ ) for each racial group using the same specification. These estimates are displayed in Table A2 in the Appendix and illustrated in Figure 2. The slope parameter is smaller for Blacks ( $β_{b}$ = 0.43) and Hispanics ( $β_{h}$ = 0.45), compared with Asians ( $β_{a}$ = 0.51) and Whites ( $β_{w}$ = 0.51). This implies that the distance between these groups gets larger as initial achievement increases. For example, while the predicted difference between Whites and Hispanics who begin in the 25th percentile is 2.3 percentile points (p = .144), the predicted difference when children begin in the 75th percentile is 5.3 percentile points (p = .005). In general, however, the differences between the rank-rank slopes are small and not statistically significant (i.e., the lines in Figure 2B are approximately parallel).

At the same time, one can perceive in Figure 2B clear differences in the estimated intercepts across racial groups. In particular, one can perceive that Blacks have a lower intercept ( $α_{b}$ = 11.5) compared with Whites ( $α_{w}$ = 26.8), Asians ( $α_{a}$ = 28.4) and Hispanics ( $α_{h}$ = 26.1). This implies that, conditional on initial achievement, Black children are more likely to move downward, and these differences get slightly larger as initial achievement increases (given that Blacks have a smaller slope).

Amplitude of Academic Mobility

We estimated the ISD and the MSSD at the population level and for each racial group using the specifications in Equations (2) and (3), respectively. As can be seen in Table A3, the ISD is similar across races, with a minimum of 13.7 percentile points for Hispanics and 14.9 percentile points for Whites. This means that, on average, White students move 14.9 percentile points around their own mean throughout the entire time-period. The differences in ISD between Whites and Hispanics (1.2 percentile points) is statistically significant (t = 3.16, p < .01). Overall, however, the differences in ISD across races is small, suggesting that there are no major differences in intraindividual mobility across races.

One can also perceive in Table A3 that the MSSD^1/2 (we present the square root so the estimates are comparable with the ISD) is higher than the ISD by around 2.5 points. This is expected, as the MSSD is more sensitive to idiosyncratic fluctuations and measurement error. We also computed the average positive and negative occasion-to-occasion differences across races. As Figure A2 in the Appendix shows, academic mobility is larger in the first year of kindergarten (with an average positive or negative academic mobility of around 30 percentile points), and appears to stabilize after third grade at about 5 percentile points. One can also perceive clear racial differences in the patterns of academic mobility in kindergarten: while Black students’ downward movement in kindergarten is more pronounced, Hispanic students’ upward movement in kindergarten is larger compared with other racial groups.

Transition Matrices and Rank Reversal Probabilities

In order to examine the extent to which individuals beginning in one quartile in Kindergarten remained in that same quartile in eighth grade, we examined a transition matrix for the whole population. Table A4 presents this transition matrix, showing the probability that children in each quartile in kindergarten stay or move to a different quartile in eighth grade. The largest proportion of students in each row fall in the leading diagonal (representing staying probabilities), or close to the diagonal. For example, more than 50% of the students beginning in the bottom quartile stay in that same group, and around 46% of the students beginning in the highest quartile remain at the top of the distribution. At the same time, one can see that a large number of students move to a different position. For example, around 32% of the students beginning in the fourth quartile end up in the third quartile; 15% end in the second quartile and 7% even end in the first quartile.

Given this transition matrix, it is not clear to what extent there is a large or small amount of academic mobility in the educational system, as one can observe some positional change but not complete origin independence. As with other mobility metrics, comparisons between groups might be more meaningful than overall measures. The transition probabilities of White and Black students displayed in Figure 3 indicate that while the latter are more likely to move downward the former are more likely to move upward. It can be useful to consider Figure 3 in relation to Figure A3 in the Appendix, which presents transition bar charts of systems with no mobility or complete mobility.

Figure 3.

Transition bar chart comparing Whites’ and Blacks’ positional mobility.

Differences in directional mobility are more consequential at the extremes, that is, cases where the individuals who start in the lowest quartile move to the highest quartile or vice versa. These measures account for initial school readiness gaps, as they compare individuals in the same original quartile. In order to perform these group comparisons, we computed the rank reversal probabilities specified in Equations (4) and (5) by race. As Table 1 indicates (see also Figure 3 for the comparison between White and Black students), the URR probabilities vary significantly by race. For example, one can see that while 11% of Asians who begin in the bottom quartile finish in the top quartile, less than 1% of Blacks achieve this. One can also see that Blacks and Hispanics have significantly different URR probabilities compared with Whites. This simple measure of success, which has been interpreted as the chances of achieving the “American Dream,” shows substantial differences across racial groups.

Table 1

Rank Reversal Probabilities by Racial Group

	Estimates						Differences Across Groups
	Overall	Asian	Black	Hispanic	Other	White	Asian-White	Black-White	Hispanic-White
P (Q4 in eighth grade \| Q1 in kindergarten)	.052 (.007)	.108 (.062)	.006 (.003)	.044 (.012)	.031 (.019)	.082 (.013)	.026 (.064)	−.076*** (.013)	−.039* (.018)
P (Q1 in eighth grade \| Q4 in kindergarten)	.066 (.014)	.032 (.023)	.283 (.084)	.052 (.025)	.077 (.040)	.029 (.007)	.003 (.024)	.253** (.084)	.023 (.023)

Note. The appropriate longitudinal weight was used in the estimation. Jackknife standard errors are in parentheses.

Table 1 also displays the DRR probabilities, indicating the chances that students who begin at the top of the distribution fall into the lowest quartile. Surprisingly, one can observe that 28% of Black students exhibit this drastic rank reversal, compared with 3% of Asians, 5% of Hispanics, and 3% of Whites. The 25%-point difference in DRR between Whites and Blacks is statistically significant. However, the difference between Whites and Asians, and Whites and Hispanics, is not statistically significant. As Figure 3 shows, these findings are not limited to these extreme cases, as Black students show patterns of downward mobility (relative to White students) no matter where they begin.

Stability and Change

We first estimated a START model using the entire sample. As shown in Table 2, the model had an excellent fit to the data, $χ^{2}$ = 179.11, df = 11, root mean square error of approximation (RMSEA) = 0.044, comparative fit index (CFI) = 0.982, Tucker-Lewis index (TLI) = 0.975, standardized root mean square residual (SRMR) = 0.026. The largest source of variance was related to the stable factor (49%), followed by the state variance (36%) and idiosyncratic variance (16%). This indicates that half of the variance is completely stable across time points, and 36% is related to systematic mobility over time. The estimated autoregressive coefficient was 0.82, suggesting that the changes across states is relatively gradual.

Table 2

Estimated Parameters and Model Fit Statistics of the Stable Trait, Autoregressive Trait, and State (START) Model by Race

	Variance (%)			AR Path Coefficient		Fit Statistics
Group	Stable Trait	State	Error	Estimate	SE	df	χ ²	RMSEA	CFI	TLI	SRMR
Overall	48.7	35.6	15.6	0.82	0.034	11	179.11	0.044	0.982	0.975	0.026
Asian	58.0	28.8	13.2	0.72	0.111	11	15.246	0.030	0.995	0.993	0.030
Black	38.0	40.7	21.3	0.79	0.163	11	21.470	0.035	0.986	0.980	0.031
Hispanic	32.8	42.8	24.4	0.85	0.077	11	32.616	0.039	0.990	0.986	0.027
Other	59.7	31.6	8.6	0.73	0.088	11	32.255	0.063	0.976	0.970	0.037
White	47.9	36.5	15.6	0.79	0.045	11	120.38	0.045	0.982	0.975	0.030

Note. AR = autoregressive; RMSEA = root mean square error of approximation; CFI = comparative fit index; TLI = Tucker-Lewis index; SRMR = standardized root mean square residual. The appropriate longitudinal weight was used in the estimation. Jackknife standard errors are displayed.

We then estimated a START model for each racial group independently. As Table 2 shows, all the models had an excellent fit to the data. One can also observe that Asian students and students belonging to other racial groups have very high academic stability, as the stable factor explains 58% and 60% of the variance, respectively. These results are consistent with the rank-rank estimates. On the other hand, the stable factor only explains 38% of the variance in academic achievement for Black students, and 33% of the variance in academic achievement for Hispanic students. This means that for these students more than 60% of the variance is associated to some kind of mobility (either systematic or idiosyncratic mobility), compared, for example, with 42% for Asians and 52% for Whites.

Group Differences Over Time

In order to obtain the coefficient of determination at each time point, we regressed students’ percentile rank score on race at each occasion. The omnibus F test indicated a significant difference in achievement across groups in every grade. As Figure 4 shows, differences among racial groups explain around 6% of the variance in achievement at the beginning of kindergarten. The coefficient of determination decreases in the spring of kindergarten (4.9%), suggesting equalizing effects in the first year of schooling. This is consistent with the high academic mobility in kindergarten depicted in Figure A2. However, as Figure 4 shows, the coefficient of determination progressively increases thereafter, and seems to stabilize at around 13.5%. The R² in eighth grade (13.3%), is more than double the R² at the beginning of kindergarten. This indicates clear disequalizing effects, as the variance between groups tends to increase, while the variance within groups tend to decrease.

Figure 4.

Coefficient of determination (R²) of achievement predicted by race across grades.

Discussion

Descriptive analyses are a fundamental component of scientific research. By attending to particular features in the data, and summarizing these features in specific ways, descriptive analyses define the phenomena that subsequent research will try to explain. The knowledge we obtain through descriptive research is determinant, for example, for framing the research hypotheses underlying intervention and causal research (see Loeb, Dynarski, McFarland, Morris, & Reardon, 2017). In the education field, achievement gaps have become stylized facts (i.e., simplified representations presumed to be generally true) that have shaped the way we think about educational inequality, as well as guided a wealth of descriptive and explanatory research. In this study, we reflected on the appropriateness of these comparative assessments for describing and evaluating educational inequalities. We argued that important normative dimensions are overlooked with traditional mean-based measures, and that academic mobility metrics can provide useful analytic tools to complement and go beyond achievement-gap analyses.

A naive positivist approach will assume that descriptive analysis merely consists in applying measures of central tendency and measures of variation to the data in order to describe the world in a neutral and comprehensive fashion. Value judgments and factual judgments are deeply entangled—at least in the social sciences (e.g., Anderson, 2004; Putnam, 2002; Schumpeter, 1949)—, and identifying relevant and accurate patterns in the data can be a complex endeavor. In other words, defining the most adequate comparisons for describing educational inequality is not a trivial task—both from a normative and methodological perspective. We began, then, by discussing key normative principles that should guide the adoption of particular comparative assessments. In particular, we argued that we should complement achievement-gap analyses by considering metrics that are sensitive to how individuals (rather than groups) change their relative (rather than absolute) position in the overall distribution over time. We referred to the processes described by these metrics as academic mobility. Subsequently, we presented five metrics that are adequate for measuring different dimensions of academic mobility.

Complementing Achievement Gap Analyses With Academic Mobility Metrics

Several studies have investigated how achievement gaps change as students progress through school (e.g., Clotfelter, Ladd, & Vigdor, 2006; Fryer & Levitt, 2004; Reardon & Galindo, 2009).¹² An important shortcoming of this literature is that the use of different scale scores (e.g., standardized or unstandardized test scores) often yields different results, and most of the metrics utilized assume a measurement scale with interval properties (Reardon, 2008a). Despite these difficulties, most of the research agrees on some key points, for example, that the Black-White achievement gaps grow during the school years, while the Hispanic-White and the Asian-White gaps decrease (Reardon et al., 2015). For instance, using the ECLS-K (which is the most commonly used data set in this literature) Reardon et al. (2015) find that the Black-White achievement gap widens from −0.53 standard deviations in the fall of kindergarten to −0.95 standard deviations in the spring of eighth grade; while, in the same period, the Hispanic-White gap shrinks from −0.48 to −0.36 standard deviations, and the Asian-White gap also shrinks (albeit in the opposite direction) from 0.22 to 0.17 standard deviations. In addition, the achievement gap literature also concludes that much of the growth of the Black-White gap occurs during elementary school (Reardon et al., 2015).

Even though these achievement gap analyses provide valuable information and have several advantages (e.g., they are easy to understand), there are several policy and socially relevant aspects that are not captured by these measures and that can be obtained using the academic mobility metrics presented in this article. First, these metrics allow us to obtain an estimate of the overall academic mobility in the population. Thus, in the empirical analysis we found that around 50% of the variance in the rank-ordering of individuals in reading achievement is completely stable from kindergarten to eighth grade. Even though a more meaningful interpretation of this result would require a historical or cross-national comparison of academic mobility in the entire education system, we can acknowledge that 50% represents a considerable degree of stability. This overall lack of academic mobility should increase societal concerns regarding initial disparities in achievement. A larger degree of mobility, on the other hand, should alleviate these concerns, as—in the long run—high mobility makes the disparities at a given point less consequential (as outcomes will tend to equalize over time; see Fields, 2010). In other words, the overall degree of academic mobility is a key factor that should be considered when interpreting achievement gaps at a particular time point.

Second, we obtained academic mobility estimates for each racial group. In the changes-in-gaps paradigm, one estimates to what extent the distance between two groups widens, narrows, or remains stable over time. This statistic provides limited information for understanding the processes behind the development of educational inequalities. For example, the fact that the gap between White and Black students widens from one-half to a full standard deviation might be due to (1) upward mobility in some White students, (2) downward mobility in some Black students, or (3) some combination of (1) and (2). Without a clear understanding of these possibilities, we will remain with a nebulous grasp of the processes behind this particular phenomenon.

In the empirical analysis, we found significant differences in the amount of academic mobility across racial groups. For example, while Asian students have very high academic stability (as 58% of the variance in the rank-ordering of individuals in reading achievement remains stable across time points), Black and Hispanic students have relatively low academic stability (as less than 40% of the variance in the rank-ordering remains stable). This means that more than 60% of the variance in Black and Hispanic students’ reading achievement is associated with some kind of mobility. It is worth noting that, according to these results, Hispanic and Black students have a similar degree of academic mobility, which is something that we would not be able to infer from achievement gaps alone. In addition, these results suggest that the changes in gaps of Hispanics and Blacks with respect to other groups are due more to the mobility of the former than the mobility of the latter.

Third, we presented mobility metrics that allow us to estimate the direction of academic mobility at different segments of the achievement distribution. Previous studies have investigated this issue by examining differential growth rates, which assume that test scores are interval-scaled (Reardon, 2008b). In order to address this problem, we implemented (1) linear rank-rank measures, which provide a parsimonious summary of the degree and direction of academic mobility by relying on some parametric assumptions and (2) transition probabilities, which are completely nonparametric (McDonough, 2015). The results of these two metrics suggest that, conditional on initial achievement, Black students are more likely to move in the downward direction at virtually every segment of the achievement distribution.

These findings have important consequences for the way we think about the development of educational inequalities. Notably, they contradict the “Mathew effect” hypothesis, according to which the development of cognitive abilities (e.g., reading) can be characterized as a cumulative process in which the advantages or disadvantages of individuals accrue over time (see, e.g., Pfost et al., 2014; Stanovich, 1986). The findings described above suggest that initial status is not a determining factor for subsequent academic performance, and that there are institutional and societal forces that produce systematic inequalities, overriding the potential benefits or impediments related to initial achievement. A result that clearly supports this point—and that might be considered a case of manifest disadvantage—, is that around 28% of the Black students who begin in the highest quartile end up in the lowest quartile (compared, for instance, with 3% of Whites and 5% of Hispanics).

Fourth, we implemented mobility metrics that are only sensitive to within-person variation. Overall, the results are consistent with the achievement gap literature (e.g., Fryer & Levitt, 2004; Reardon & Galindo, 2009), according to which Hispanic students tend to be more upwardly mobile in the first years of schooling, and Black students tend to be less upwardly mobile throughout the entire period (see, Figure A2). In contrast to achievement gaps, however, these metrics provide novel insights into individual-level (rather than group-level) processes of change. For example, Figure A2 indicates that all students tend to be more mobile in kindergarten (with an average mobility of around 30 percentile points), and tend to stabilize after third grade at about 5 percentile points.

Finally, we used the coefficient of determination to measure the extent to which the inequalities between groups increase or decrease over time. Traditional methods do not provide comparable metrics of group differentiation and are better suited for comparing only two groups (e.g., White vs. Black students). In the empirical analysis, we found evidence of racial disequalization over time, as the differences between racial groups increase, while difference within groups decrease.

Limitations and Conclusion

It is worth mentioning that the purpose of the present article was not to conduct a thorough analysis of academic mobility in the United States. The goal, rather, was to argue why the concept of academic mobility provides a useful framework for describing and evaluating educational inequalities, and present various metrics that measure different aspects of academic mobility. For that reason, we presented the academic mobility metrics in their most basic form—both in their mathematical representation and empirical application. The full potential and significance of these measures will become more apparent as subsequent studies provide more detailed descriptive and explanatory evidence on students’ academic mobility, as well as more comprehensive analysis of each measure’s psychometric properties. Questions that can be answered with these metrics, and could not be answered with other traditional measures, include the following: How has academic mobility at the population and subpopulation levels changed over time? How does academic mobility in particular educational systems (e.g., districts or countries) compare with academic mobility in other educational systems? How do mobility patterns vary across the achievement distribution? To what extent do resources and experiences within school, family and neighborhood contexts independently and cumulatively explain differences in academic mobility? It is our hope that the metrics that we have presented, which have been used in disciplines outside of education, can be applied in the field of education in order to better describe and explain educational inequalities.

Footnotes

Appendix

Table A4

National Quartile Transition Matrix (%)

	Quartile in Eighth Grade
Quartile in Kindergarten	1	2	3	4
1	51.0	29.3	14.2	5.4
2	30.8	32.0	22.5	14.7
3	19.3	23.1	31.3	26.3
4	6.6	15.0	32.4	46.0

Note. The appropriate longitudinal weight was used in the estimation.

Notes

Rafael Quintana is a doctoral candidate in learning sciences and policy in the Learning Research & Development Center at the University of Pittsburgh, 3939 O’Hara Street, Pittsburgh, PA 15260; e-mail: raq5@pitt.edu . His research focuses on conceptual and methodological issues in skill development. Using an interdisciplinary approach, his work attempts to describe and explain individual differences in skill development.

Richard Correnti is an associate professor of learning sciences and policy at the University of Pittsburgh’s School of Education and a research scientist at the Learning Research and Development Center. His research interests center on the measurement and determinants of instruction and on how to improve teaching practice in order to improve student thinking and reasoning. Many of his current projects focus on analyzing teaching development in the context of interventions such as literacy and mathematics coaching.

References

Anderson

(1999). What is the point of equality? Ethics, 109, 287–337.

Anderson

(2004). Uses of value judgments in science: A general argument, with lessons from a case study of feminist research on divorce. Hypatia, 19(1), 1–24.

Anderson

(2007). Fair opportunity in education: A democratic equality perspective. Ethics, 117, 595–622.

Arneson

(1989). Equality and equal opportunity of welfare. Philosophical Studies, 56, 77–93.

Bailey

M. J.

Dynarski

S. M.

(2011). Gains and gaps: Changing inequality in U.S. college entry and completion (National Bureau of Economic Research Working Paper No. 17633). Cambridge, MA: National Bureau of Economic Research.

Betebenner

D. W.

(2009). Norm-and criterion-referenced student growth. Educational Measurement: Issues and Practice, 28, 42-51.

Bodovski

Farkas

(2008). “Concerted cultivation” and unequal achievement in elementary school. Social Science Research, 37, 903–919.

Bond

T. N.

Lang

(2013). The evolution of the Black-White test score gap in Grades K–3: The fragility of results. Review of Economics and Statistics, 95, 1468–1479.

Breen

Jonsson

J. O.

(2000). Analyzing educational careers: A multinomial transition model. American Sociological Review, 65, 754–772.

10.

Breen

Jonsson

J. O.

(2005). Inequality of opportunity in comparative perspective: Recent research on educational attainment and social mobility. Annual Review of Sociology, 31, 223–243.

11.

Briggs

Betebenner

(2009, April 14). Is growth in student achievement scale dependent?Paper presented at the invited symposium Measuring and Evaluating Changes in Student Achievement: A Conversation about Technical and Conceptual Issues at the annual meeting of the National Council for Measurement in Education, San Diego, CA.

12.

Brighouse

Swift

(2006). Equality, priority, and positional goods. Ethics, 116, 471–497.

13.

Cameron

C. E.

Grimm

K. J.

Steele

J. S.

Castro-Schilo

Grissmer

D. W.

(2015). Nonlinear Gompertz curve models of achievement gaps in mathematics and reading. Journal of Educational Psychology, 107, 789–804.

14.

Castellano

K. E.

A. D.

(2013). Contrasting OLS and quantile regression approaches to student “growth” percentiles. Journal of Educational and Behavioral Statistics, 38, 190–215.

15.

Chetty

Hendren

Jones

M. R.

Porter

S. R.

(2018). Race and economic opportunity in the United States: An intergenerational perspective (No. w24441). Cambridge, MA: National Bureau of Economic Research.

16.

Chetty

Hendren

Kline

Saez

(2014). Where is the land of opportunity? The geography of intergenerational mobility in the United States. Quarterly Journal of Economics, 129, 1553–1623.

17.

Clotfelter

C. T.

Ladd

H. F.

Vigdor

J. L.

(2006). The academic achievement gap in grades three to eight (Working Paper No. 12207). Cambridge, MA: National Bureau of Economic Research.

18.

Cohen

G. A.

(1989). On the currency of egalitarian justice. Ethics, 99, 906–944.

19.

Coleman

(1968). The concept of equality of educational opportunity. Harvard Educational Review, 38, 7–22.

20.

Cunha

Heckman

J. J.

(2008). Formulating, identifying and estimating the technology of cognitive and noncognitive skill formation. Journal of Human Resources, 43, 738–782.

21.

Curran

P. J.

Bauer

D. J.

(2011). The disaggregation of within-person and between-person effects in longitudinal models of change. Annual Review of Psychology, 62, 583–619.

22.

Davis-Kean

P. E.

Jager

(2014). Trajectories of achievement within race/ethnicity: “Catching up” in achievement across time. Journal of Educational Research, 107, 197–208.

23.

Domina

Penner

(2017). Categorical inequality: Schools as sorting machines. Annual Review of Sociology, 43, 311–330.

24.

Downey

D. B.

Condron

D. J.

(2016). Fifty years since the Coleman Report: Rethinking the relationship between schools and inequality. Sociology of Education, 89, 207–220.

25.

Estabrook

Grimm

K. J.

Bowles

R. P.

(2012). A Monte Carlo simulation study of the reliability of intraindividual variability. Psychology and Aging, 27, 560–576.

26.

Feinstein

(2003). Inequality in the early cognitive development of British children in the 1970 cohort. Economica, 70(277), 73–97.

27.

Feuer

(2015). Evidence and advocacy. In Feuer

M. J.

Berman

A. I.

Atkinson

R. C.

(Eds.), Past as prologue: The National Academy of Education at 50. Members reflect (pp. 95–101). Washington, DC: National Academy of Education.

28.

Fields

G. S.

(2006). The many facets of economic mobility. In McGillivray

(Ed.), Inequality, poverty, and well-being (pp. 123-142). Hampshire, England: Palgrave Macmillan.

29.

Fields

G. S.

(2010). Does income mobility equalize longer-term incomes? New measures of an old concept. Journal of Economic Inequality, 8, 409–427.

30.

Fisher

A. J.

Megdalia

Jeronimus

B. F.

(2018). A lack of group-to-individual generalizability is a threat to human subjects research. Proceedings of the National Academy of Sciences of the United States of America, 115, E6106-E6115.

31.

Fryer

R. G.

Jr. Levitt

S. D.

(2004). Understanding the black-white test score gap in the first two years of school. Review of Economics and Statistics, 86, 447–464.

32.

Glaser

(1963). Instructional technology and the measurement of learning outcomes: Some questions. American Psychologist, 18, 519-521. doi:10.1037/h0049294

33.

Hamaker

E. L.

(2012). Why researchers should think “within-person” a paradigmatic rationale. In Mehl

M. R.

Conner

T. S.

(Eds.), Handbook of research methods for studying daily life (pp. 43–61). New York, NY: Guilford Press.

34.

Hambleton

R. K.

Pitoniak

M. J.

(2006). Setting performance standards. In Brennan

R. L.

(Ed.), Educational measurement (4th ed., pp. 433–470). Westport, CT: American Council on Education and Praeger.

35.

Hamilton

L. S.

(2003). Assessment as a policy tool. Review of Research in Education, 27, 25–68.

36.

Hill

C. J.

Bloom

H. S.

Black

A. R.

Lipsey

M. W.

(2008). Empirical benchmarks for interpreting effect sizes in research. Child Development Perspectives, 2, 172–177.

37.

A. D.

Haertel

E. H.

(2006, January). Metric-free measures of test score trends and gaps with policy-relevant examples (CSE Report 665). Los Angeles, CA: National Center for Research on Evaluation, Standards, and Student Testing (CRESST). Retrieved from https://cresst.org/wp-content/uploads/R665.pdf

38.

Hoffman

(2015). Longitudinal analysis: Modeling within person fluctuation and change. New York, NY: Routledge.

39.

Jahng

Wood

P. K.

Trull

T. J.

(2008). Analysis of affective instability in ecological momentary assessment: Indices using successive difference and group comparison via multilevel modeling. Psychological Methods, 13, 354–375.

40.

Jäntti

Jenkins

(2013, November). Income mobility (IZA Discussion Paper, No. 7730). Bonn, Germany: IZA.

41.

Jencks

(1988). Whom must we treat equally for educational opportunity to be equal? Ethics, 98, 518-533.

42.

Kanbur

Wagstaff

(2016). How useful is inequality of opportunity as a policy construct? In Basu

Stiglitz

J. E.

(Eds.), Inequality and growth: Patterns and policy. Vol. I: Concepts and analysis (pp. 131–148). London, England: Palgrave Macmillan.

43.

Kenny

D. A.

Zautra

(1995). The trait-state-error model for multiwave data. Journal of Consulting and Clinical Psychology, 63, 52–59.

44.

Kenny

D. A.

Zautra

(2001). Trait-state models for longitudinal data. In Collins

L. M.

Sayer

A. G.

(Eds.), New methods for the analysis of change: Decade of behavior (pp. 243–263). Washington, DC: American Psychological Association.

45.

Kerckhoff

A. C.

(1993). Diverging pathways: Social structure and career deflections. New York, NY: Cambridge University Press.

46.

Lareau

(2003). Unequal childhoods: Class race and family life. Berkeley: University of California Press.

47.

Loeb

Dynarski

McFarland

Morris

Reardon

(2017). Descriptive analysis in education: A guide for researchers (NCEE 2017-4023). Washington, DC: U.S. Department of Education.

48.

Lucas

S. R.

(2001). Effectively maintained inequality: Education transitions, track mobility, and social background effects. American Journal of Sociology, 106, 1642–1690.

49.

Lucas

S. R.

Berends

(2002). Sociodemographic diversity, correlated achievement, and de facto tracking. Sociology of Education, 75, 328–348.

50.

Mazumder

(2005). Fortunate sons: New estimates of intergenerational mobility in the United States using social security earnings data. Review of Economics and Statistics, 87, 235-255.

51.

McDonough

I. K.

(2015). Dynamics of the black–white gap in academic achievement. Economics of Education Review, 47, 17–33.

52.

Messick

(1994). The interplay of evidence and consequences in the validation of performance assessments. Educational Researcher, 23(2), 13–23.

53.

Mislevy

R. J.

Steinberg

L. S.

Almond

R. G.

(2003). Focus article: On the structure of educational assessments. Measurement: Interdisciplinary Research and Perspectives, 1, 3–62.

54.

Molenaar

P. C.

(2004). A manifesto on psychology as idiographic science: Bringing the person back into scientific psychology, this time forever. Measurement, 2, 201–218.

55.

Muthén

(2004). Latent variable analysis: Growth mixture modeling and related techniques for longitudinal data. In Kaplan

(Ed.), Handbook of quantitative methodology for the social sciences (pp. 345–368). Thousand Oaks, CA: Sage.

56.

Nagin

D. S.

(1999). Analyzing developmental trajectories: A semiparametric, group-based approach. Psychological Methods, 4, 139–152.

57.

Nagin

D. S.

(2005). Group-based modeling of development. Cambridge, MA: Harvard University Press.

58.

Nesselroade

J. R.

Molenaar

P. C. M.

(2010). Emphasizing intraindividual variability in the study of development over the life span: Concepts and issues. In Overton

W. F.

(Ed.) & Lerner

R. M.

(Editor- in-Chief), Handbook of life-span development. Vol. 1: Cognition, biology, and methods across the lifespan (pp. 30–54). Hoboken, NJ: Wiley.

59.

Newsom

J. T.

(2015). Longitudinal structural equation modeling: A comprehensive introduction. New York, NY: Routledge.

60.

Nikolaev

(2016). Does other people’s education make us less happy? Economics of Education Review, 52, 176–191.

61.

Page

L. C.

Scott-Clayton

(2016). Improving college access in the United States: Barriers and policy responses. Economics of Education Review, 51, 4–22.

62.

Pallas

A. M.

(2000). The effects of schooling on individual lives. In Hallinan

M. T.

(Ed.), Handbook of the sociology of education (pp. 499–525). New York, NY: Kluwer Academic/Plenum.

63.

Pfost

Hattie

Dörfler

Artelt

(2014). Individual differences in reading development: A review of 25 years of empirical research on Matthew effects in reading. Review of Educational Research, 84, 203–244.

64.

Putnam

(2002). The collapse of the fact/value dichotomy and other essays. Cambridge, MA: Harvard University Press.

65.

Raudenbush

S. W.

(2001). Comparing personal trajectories and drawing causal inferences from longitudinal data. Annual Review of Psychology, 52, 501–525.

66.

Rawls

(1971). A theory of justice. Cambridge, MA: Harvard University Press.

67.

Reardon

S. F.

(2008a). Thirteen ways of looking at the Black-White test score gap (Working Paper No. 2008-08). Stanford, CA: Institute for Research on Educational Policy and Practice, Stanford University.

68.

Reardon

S. F.

(2008b). Differential growth in the Black-White achievement gap during elementary school among initially high- and low-scoring students (Working Paper Series). Stanford, CA: Institute for Research on Educational Policy and Practice, Stanford University.

69.

Reardon

S. F.

Galindo

(2009). The Hispanic-White achievement gap in math and reading in the elementary grades. American Educational Research Journal, 46, 853–891.

70.

Reardon

S. F.

Robinson-Cimpian

J. P.

Weathers

E. S.

(2015). Patterns and trends in racial/ethnic and socioeconomic academic achievement gaps. In Ladd

Goertz

(Eds.), Handbook of research in education finance and policy (2nd ed., pp. 491–509). Mahwah, NJ: Lawrence Erlbaum.

71.

Reeves

(2015). The measure of a nation. Annals of the American Academy of Political and Social Science, 657, 22–26.

72.

Roemer

(1998). Equality of opportunity. Cambridge, MA: Harvard University Press.

73.

Roemer

J. E.

Trannoy

(2015). Equality of opportunity. In Atkinson

A. B.

Bourguignon

(Eds.), Handbook of income distribution (Vol. 2, pp. 217–300). Amsterdam, Netherlands: North Holland.

74.

Schumpeter

(1949). Science and ideology. American Economic Review, 39, 345–359.

75.

Sen

(2009). The idea of justice. London, England: Allen Lane.

76.

Shepard

L. A.

(1997). The centrality of test use and consequences for test validity. Educational Measurement: Issues and Practice, 16(2), 5–24.

77.

Shepard

L. A.

Hannaway

Baker

(Eds.). (2009). Standards, assessments, and accountability. Washington, DC: National Academy of Education.

78.

Sohn

(2012). The dynamics of the evolution of the Black–White test score gap. Education Economics, 20, 175–188.

79.

Solnick

S. J.

Hemenway

(1998). Is more always better? A survey on positional concerns. Journal of Economic Behavior & Organization, 37, 373–383.

80.

Stanovich

K. E.

(1986). Matthew effects in reading: Some consequences of individual differences in the acquisition of literacy. Reading Research Quarterly, 21, 360–407.

81.

Thomson

J. J.

(2008). Normativity. Chicago, IL: Open Court.

82.

Tourangeau

Nord

Lê

Sorongon

A. G.

Najarian

Hausken

E. G.

(2009). Early childhood longitudinal study, kindergarten class of 1998-99 (ECLS-K) combined user’s manual for the ECLS-K Eighth-Grade and K-8 full sample data files and electronic codebook. Washington, DC: U.S. Department of Education.

83.

Van de Werfhorst

H. G.

Mijs

J. J.

(2010). Achievement inequality and the institutional structure of educational systems: A comparative perspective. Annual Review of Sociology, 36, 407–428.

84.

Wang

L. P.

Grimm

K. J.

(2012). Investigating reliabilities of intraindividual variability indicators. Multivariate Behavioral Research, 47, 771–802.

85.

Wang

L. P.

Hamaker

Bergeman

C. S.

(2012). Investigating inter-individual differences in short-term intra-individual variability. Psychological Methods, 17, 567–558.