Decomposing Ethnic Achievement Gaps across Multiple Levels of Analysis and for Multiple Ethnic Groups

Abstract

Ethnic achievement gaps are often explained in terms of student and school factors. The decomposition of these gaps into their within- and between-school components has therefore been applied as a strategy to quantify the overall influence of each set of factors. Three competing approaches have previously been proposed, but each is limited to the study of student-school decompositions of the gap between two ethnic groups (e.g., White and Black). The authors show that these approaches can be reformulated as mediation models facilitating new extensions to allow additional levels in the school system (e.g., classrooms, school districts, geographic areas) and multiple ethnic groups (e.g., White, Black, Hispanic, Asian). The authors illustrate these extensions using administrative data for high school students in Colombia and highlight the increased substantive insights and nuanced policy implications they afford.

Keywords

achievement gaps mediation analysis effect decomposition multiple-level decomposition multigroup decomposition

Ethnic achievement gaps, usually defined as the difference between the mean test scores of two groups of students (e.g., Black and White students), have been widely explored in education, psychology, sociology, and other behavioral sciences. This literature suggests these gaps are the result of differences in a wide range of variables defined at the student, school, and higher levels of the education system (Bidwell and Kasarda 1975; Coleman et al. 1966; Mohammadpour and Ghafar 2014; Rothstein 2004; Wenglinsky 2009). One way of addressing the challenge of finding the variables that potentially drive achievement gaps is to know where to start looking for them.

A natural approach is therefore to decompose the overall achievement gap into its separate component parts operating at each level of the education system. Once this has been achieved, variables at the corresponding levels (e.g., student, school, and school district characteristics) can be considered as potential explanatory variables of the achievement gap. For example, if most of the achievement gap is due to differences within schools, we might start with student-level variables, or if most of the achievement gap is due to differences between schools, we might prioritize school-level variables, including schools’ characteristics and composition. One might also examine how within- and between-school decomposition changes over cohorts and school grades and across regions and countries. Identifying the contribution of different levels of the education system to the achievement gap can also help prioritize policy efforts at specific levels and establish what level of the education system may be held accountable for disparities in achievement.

Prior work has proposed three competing methodological approaches to study the Black-White achievement gap in the United States. We refer to these as approach 1 (Cook and Evans 2000; Fryer and Levitt 2004, 2006), approach 2 (Hanushek and Rivkin 2006), and approach 3 (Page, Murnane, and Willett 2008; Reardon 2008). Authors applying approaches 1 and 3 argue that between-school differences explain at most 40 percent of the White-Black achievement gap in the United States (Cook and Evans 2000; Fryer and Levitt 2004, 2006; Page et al. 2008; Reardon 2008), whereas authors applying approach 2 argue that these differences explain 70 percent of the achievement gap (Hanushek and Rivkin 2006).

These approaches are limited in at least two ways. First, they do not consider the role of other levels of the school system, such as school districts. Second, they are restricted to a binary comparison, ignoring the role of other minority groups. Such omissions may hinder the pertinence and relevance of these gap decomposition analyses. For example, existing approaches might direct attention to school-level policies (e.g., providing teacher training), whereas a more relevant policy might focus on district-level interventions (e.g., reconsidering how resources are allocated).

The main contribution of this article is to extend the three current decomposition approaches to consider additional levels of the school system (e.g., districts) and multiple ethnic groups (e.g., White, Black, Asian, Hispanic) using mediation analysis as a framework to facilitate such extensions. We first show that the current achievement gap decomposition approaches are mathematically equivalent to a mediation problem. We then take advantage of this equivalence to extend the current decomposition approaches to consider additional levels of the school system and multiple ethnic groups. Note that we do not use mediation analysis in the traditional sense (Baron and Kenny 1986; Vanderweele and Vansteelandt 2009) but merely as a device to facilitate the effect decomposition. Hou (2014) proposed using mediation analysis as a general framework for effect decomposition. However, the use of mediation analysis as a methodological tool to extend ethnic achievement gap decomposition approaches has not been considered in the literature. Although this is not the only option to derive the extensions presented here (e.g., deriving the mathematical formulation directly is another option), it facilitates the presentation of these extensions. We apply these extensions to the three approaches to study ethnic achievement gaps in Colombia to illustrate the extensions and their importance.

Approaches for the within- and between-School Gap Decomposition

Reardon (2008) and Page et al. (2008) reviewed the three existing methodological approaches for the ethnic achievement gap decomposition and discuss their interpretation. We summarize these approaches from a mediation analysis perspective, in preparation for the extensions in the following sections, which present our main contribution. Here, we understand mediation analysis merely as a tool that allows us to decompose the effect of ethnicity (i.e., the ethnic achievement gap) into different components. Nonetheless, using the mediation analysis framework is not essential but a means to deriving the extensions we propose.

Consider the single-level linear regression model

\begin{matrix} y_{ij} = α + β M_{ij} + {e_{y}}_{ij} \\ {e_{y}}_{ij} ~ N (0, Ω), \end{matrix}

(1)

where the dependent variable $y_{ij}$ is the test score of student $i$ attending school $j$ , and the independent variable $M_{ij}$ is a dummy variable that equals 1 for ethnic minority students and 0 otherwise. In this case, $β$ represents the average difference in test scores between minority and White students, that is, the overall ethnic achievement gap.

The difference in average test scores between White and minority students within and between schools can be estimated using the hybrid-effect model¹:

\begin{matrix} y_{ij} = α + β^{W} (M_{ij} - {\bar{M}}_{. j}) + β^{S} {\bar{M}}_{. j} + {e_{y}}_{ij} \\ = α + β^{W} M_{ij} + (β^{S} - β^{W}) {\bar{M}}_{. j} + {e_{y}}_{ij} \\ = α + β^{W} M_{ij} + β^{C} {\bar{M}}_{. j} + {e_{y}}_{ij}, \end{matrix}

(2)

where ${\bar{M}}_{. j}$ is the proportion of minority students in school $j$ . Thus, $β^{W}$ represents the within-school achievement gap; that is, the average difference in test scores between White and minority students attending the same school. $β^{S}$ represents the between-school gap; that is, the average difference in test scores between schools that serve only White students ( ${\bar{M}}_{. j} = 0$ ) and schools that serve only minority students ( ${\bar{M}}_{. j} = 1$ ). The last line in equation (2) illustrates the equivalence between the hybrid-effect model and the contextual-effect model. In this last type of model, $β^{C} = β^{S} - β^{W}$ represents the school contextual effect of ethnicity, that is, the effect of attending a school with a larger proportion of minority students, over and above the effect of students’ own ethnicity.

To translate the three achievement gap decomposition approaches into the mediation analysis framework, it is enough to consider a model in which the ethnic composition of each school ${\bar{M}}_{. j}$ mediates the relationship between ethnicity $M_{ij}$ and achievement $y_{ij}$ , as we show in Figure 1.

Figure 1.

Model for the achievement gap decomposition into its within- and between-school components.

The upper part of Figure 1 describes the total-effect model (equation 1). The lower part of Figure 1 represents the outcome and mediation models. The outcome model is the contextual-effect model (equation 2). The mediation equation is

{\bar{M}}_{. j} = γ_{S} + ω M_{ij} + {e_{S}}_{. j}, \forall i \in j,

(3)

where the school proportion of minority students ${\bar{M}}_{. j}$ is repeated for all students $i$ in school $j$ . Here, $γ_{S}$ represents the average school proportion of minority students experienced by White students, and $ω$ represents the difference in the average school proportion of minority students experienced by White and minority students. If $ω = 0$ , White and minority students attend schools with the same proportion of minority students; if $ω = 1$ , White students attend schools with only White students and minority students attend schools with only minority students. As we will show in more detail, equation (3) is a simple linear regression used as a device to compute $ω$ . Given its interpretation, $ω$ is equivalent to the variance ratio index of segregation, as noted by Reardon (2008).

In the mediation analysis context, the total effect $β$ is decomposed into a direct effect $β^{W}$ and an indirect effect $ω β^{C}$ , such that

β = β^{W} + ω β^{C} .

(4)

Here, the direct effect is the within-school gap $β^{W}$ , and the indirect effect $ω β^{C}$ represents how much more minority students are affected by the school contextual effect of ethnicity $β^{C}$ , compared with White students, given their average additional exposure to minority students $ω$ .

Different strategies can be used to derive confidence intervals for the direct and indirect effect, as well as for the different components of the gap, including bootstrapping and Monte Carlo methods (MacKinnon, Lockwood, and Williams 2004). For the application in this article, we estimate 95 percent, 99 percent, and 99.9 percent Monte Carlo confidence intervals for the proportion of the gap that each of its components represents. We implemented the confidence intervals via the approach of Selig and Preacher (2008). The advantage of Monte Carlo methods is that they only require the estimated parameters and their standard errors, which can be estimated using cluster-robust estimators to account for the nested nature of the data (Preacher and Selig 2012). The supplemental material includes R and Stata code to implement the proposed decompositions.

Approach 1: Cook and Evans (2000) and Fryer and Levitt (2004, 2006)

Cook and Evans (2000) and Fryer and Levitt (2004, 2006) used a version of the Kitagawa-Oaxaca-Blinder decomposition (Blinder 1973; Kitagawa 1955; Oaxaca 1973) that includes school fixed effects, allowing them to identify achievement gaps within schools, after considering differences in gender and parental education. This model can be written as

y_{ij} = α_{j} + β^{W} M_{ij} + {e_{y}}_{ij},

(5)

where $α_{j}$ is a dummy variable (fixed effect) for school $j$ . In this case, $β^{W}$ is the within-school ethnic achievement gap, that is, the difference in mean test scores between White and minority students attending the same school. Cook and Evans and Fryer and Levitt attributed $β^{W}$ to within-school differences and the remainder of the gap ( $β - β^{W}$ ) to between-school differences, where $β$ is the overall gap estimated by equation (1). In the mediation analysis framework, this difference is understood as an alternative to equation (4) to calculate the indirect effect (of schools, in this case).

Therefore, the decomposition approach 1 is equivalent to equation (4), and Cook and Evans (2000) and Fryer and Levitt (2004, 2006) attributed $β^{W}$ to the within-school component of the gap and $ω β^{C}$ to the between-school component of the gap. The expression in equation (4) shows that the interpretation of Cook and Evans and Fryer and Levitt implies that eliminating the within-school gap $β^{W} = 0$ would leave the contribution of the between-school differences to the gap unaffected.

This statement is true in a scenario without segregation ( $ω = 0$ ), in which the ethnic achievement gap $β$ is fully explained by the within-school achievement gap ( $β = β^{W}$ ). In this case, there are no differences in the average test scores of schools typically attended by White and minority students. However, in a scenario with segregation ( $ω \neq 0$ ), suppressing the within-school ethnic achievement gap ( $β^{W} = 0$ ) leads to changes in ethnic differences between schools that the between-school component of this decomposition does not capture (Hanushek and Rivkin 2006; Reardon 2008). Therefore, approach 1 decomposes the achievement gap into its direct and indirect effects: the within-school gap $β^{W}$ and the effect of segregation $ω β^{C}$ .

Approach 2: Hanushek and Rivkin (2006)

Challenging Fryer and Levitt (2004, 2006), Hanushek and Rivkin (2006) deduced a mathematical formulation to write the Black-White achievement gap as a weighted sum of differences within and between schools. As Reardon (2008) showed, Hanushek and Rivkin used the within- and between-school gaps, which can be recovered from the hybrid-effect model (equation 2). Reardon also showed that Hanushek and Rivkin’s formula is equivalent to

β = (1 - ω) β^{W} + ω β^{S}, ω = \frac{Var ({\bar{M}}_{. j})}{Var (M_{ij})} .

(6)

Then, Hanushek and Rivkin argued that the contribution of the within-school achievement gap $β^{W}$ to the overall gap $β$ is $(1 - ω) β^{W}$ , and the contribution of the between-school differences $β^{S}$ to the overall achievement gap is $ω β^{S}$ , as has been shown in more general contexts (Burstein 1980; Cronbach, Deken, and Webb 1976).

Translating this approach to the mediation analysis framework requires recognizing that differences between schools are the sum of differences within schools and the contextual effect of ethnicity. In other words, differences between schools arise not only because of the effect of studying with a larger proportion of minority students ( $β^{C}$ , a school-level effect) but also because of differences between White and minority students across all schools ( $β^{W}$ ). Equivalently, this can be expressed as $β^{C} = β^{S} - β^{W}$ .

Recognizing this in equation (4) leads to equation (6). Accordingly, the ethnic achievement gap $β$ is a weighted² sum of differences within schools $β^{W}$ and between schools $β^{S}$ . As Reardon (2008) noted, in a scenario without segregation ( $ω = 0$ ), this decomposition is equivalent to Cook and Evans’s (2000) and Fryer and Levitt’s (2004, 2006), with $β = β^{W}$ . If there is segregation ( $ω \neq 0$ ), suppressing the within-school ethnic achievement gap ( $β^{W} = 0$ ) implies changes in both, the within-school $(1 - ω) β^{W}$ and between-school $ω β^{S}$ components of the gap, which would then be 0 and $ω β^{C}$ , respectively.

Approach 3: Reardon (2008) and Page et al. (2008)

Trying to conciliate Fryer and Levitt (2004, 2006) and Hanushek and Rivkin (2006), Reardon (2008) argued that the overall achievement gap can be decomposed into three different components: one that can be attributed to differences within schools, a second one due to differences between schools, and a third that is a combination of both and therefore cannot be uniquely assigned to either of those. This decomposition is based on the contextual-effect model (equation 2).

This is equivalent to further separating the contribution of between-school differences to the overall gap, acknowledging that $β^{S} = β^{W} + β^{C}$ in equation (6), which leads to

β = (1 - ω) β^{W} + ω β^{W} + ω β^{C} .

(7)

Under Reardon’s (2008) interpretation, $(1 - ω) β^{W}$ is the within-school component of the gap, $ω β^{C}$ is the between-school component, due to segregation, and $ω β^{W}$ is the part of the gap that cannot be unambiguously attributed to either of them.

To interpret the differences between approaches 1 and 2, Reardon (2008) and Page et al. (2008) proposed analyzing the type of policies that would be required to eliminate ethnic differences within and between schools. If eliminating a component of the gap requires a between-school intervention, it can be attributed to the between-school gap. Conversely, if it requires a within-school intervention, it can be attributed to the within-school component of the gap. Their argument is based on three policies:

Eliminating the within-school gap ( $β^{W} = 0$ ) while leaving segregation ( $ω$ ) and the school mean achievement unchanged.³ This within-school effort would lead to a decrease of $(1 - ω) β^{W}$ in the overall gap $β$ , which now is $β = ω β^{C}$ , composed only of differences between schools. Therefore, Reardon and Page et al. argued that $(1 - ω) β^{W}$ is the part of the gap that can “unambiguously” be attributed to within-school differences, in agreement with Hanushek and Rivkin (2006).

Eliminating segregation ( $ω = 0$ ). This between-school effort would lead to a decrease of $ω β^{C}$ in the overall gap, which would now only be the within-school gap $β = β^{W}$ , as Fryer and Levitt (2004, 2006) argued. Therefore, Reardon and Page et al. interpreted $ω β^{C}$ as the part of the gap that is attributable to segregation and “unambiguously” to between-school differences.

Eliminating the relationship between the school ethnic composition and its mean achievement ( $β^{W} + β^{C} = 0$ ) while keeping the within-school gap $β^{W}$ and segregation $ω$ at the same levels.⁴ This between-school effort would lead to a reduction of $ω (β^{W} + β^{C})$ in the overall gap $β$ , which would now be $β = (1 - ω) β^{W}$ ; the “unambiguously” within-school component of the gap, which Reardon and Page et al. argued is in agreement with Hanushek and Rivkin’s decomposition.

Using these policy scenarios, Reardon (2008) and Page et al. (2008) argued that $ω β^{W}$ cannot be “unambiguously attributed” to either the within- or the between-school component of the gap because it is an interaction between within-school differences $β^{W}$ and segregation $ω$ , which can be changed through interventions either within or between schools. The difference between approaches 1 and 2, they argued, is that approach 1 attributes $ω β^{W}$ to the within-school component, whereas approach 2 attributes it to the between-school component. This argument can be questioned⁵ by noticing that the “unambiguously within-school” component of the gap $(1 - ω) β^{W}$ could also be eliminated through a between-school intervention: completely segregating schools ( $ω = 1$ ). We argue that what Reardon and Page et al. called the “ambiguous” component of the gap ( $ω β^{W}$ ) is the contribution of segregation through differences in schools’ student intake, and $ω β^{C}$ is the contribution of segregation through the school contextual effect of ethnicity. Adding these two results in the contribution of between-school differences to the overall ethnic achievement gap, as Hanushek and Rivkin (2006) argued.

Additionally, Page et al. (2008) combined Reardon’s (2008) approach with the Kitagawa-Oaxaca-Blinder decomposition to incorporate covariates in the model, including indicator variables for other ethnicities. They were the first authors to consider other ethnicities, but their approach did not acknowledge that the influence of other groups can be part of the between-school component of the Black-White achievement gap, as we will discuss. We now turn our attention to our application of interest.

Data and Initial Application

At the end of secondary education, all Colombian students take a standardized exam, called SABER 11. The compulsory nature of the exam means that the data resulting from its administration are effectively a census of 11th grade students (age 16 and 17 years) in Colombia, including those attending private and public (state-funded) schools. The nationwide educational authority in Colombia is the Ministry of Education, but smaller administrative divisions—districts or entidades territoriales certificadas—are responsible for the management of education within their geographic boundaries.

The main ethnic groups in Colombia are the White-mestizo group (European descendants who mixed with other ethnic groups, 85.9 percent of the population), Afrocolombians (10.6 percent), and the Indigenous (3.4 percent); other ethnic minorities account for 0.01 percent of the population (DANE 2007). Afrocolombian, Indigenous, and other minority students are grouped together for the applications presented here and in the application for the extension including multiple levels. Hence, the overall ethnic achievement gap is defined as the difference in average test scores between White and minority students.

We focus on the SABER 11 math test scores for 2011, which included 458,947 students, 8,039 schools, and 94 districts. Among all students, 6.7 percent self-identify as belonging to ethnic minorities. Only 34.9 percent of schools serve both White and minority students, and there is large variation in the proportion of minority students within schools, regardless of their size. The math test scores have been normalized to have a mean of 0 and a standard deviation of 1.

Page et al. (2008) suggested excluding schools in which all students belong to the same ethnic group (i.e., ethnically homogeneous schools), because these do not contribute to estimation of the within-school gap. However, in Colombia, this results in the exclusion of 236,794 students, or 51.6 percent of the observations. Therefore, an analysis based on a data set without ethnically homogeneous schools is unlikely to provide an accurate representation of differences between schools. However, the magnitude of the results presented here may be skewed toward the scores obtained by students in these schools.

The first model in Table 1 estimates the overall ethnic achievement gap, showing that minority students score, on average, 0.46 S.D. ( $\hat{β}$ ) lower than their White peers. The second model in this table estimates the within-school gap and the school contextual effect of ethnicity. Accordingly, it shows that minority students score, on average, 0.07 S.D. ( $\hat{β^{W}}$ ) lower than White students attending the same school. Additionally, students attending schools with a 10 percentage points higher proportion of minority students score, on average, 0.06 S.D. ( $\frac{\hat{β^{C}}}{10}$ ) lower, independent of their own ethnicity. The second model in Table 1 also shows that the average test score of schools that serve only White students is, on average, 0.68 S.D. ( $\hat{β^{S}} = \hat{β^{W}} + \hat{β^{C}}$ ) higher than that of schools serving only minority students.

Table 1.

Estimation Results for the Models Underlying the Decomposition Approaches

	Math
	(1)	(2)
Intercept	.031***	.046***
Intercept	(.002)	(.002)
$M_{ij}$	−.459***	−.072***
$M_{ij}$	(.006)	(.010)
${\bar{M}}_{. j}$		−.606***
${\bar{M}}_{. j}$		(.012)
Observations	458,947	458,947
R ²	.013	.019
Adjusted R²	.013	.019
Residual standard error	.993	.991
F statistic	6,149.987***	4,336.309***

Note: $M_{ij}$ is a dummy variable for minority students, and ${\bar{M}}_{. j}$ is the school proportion of minority students. Cluster-robust standard errors are shown.

***

p < .001.

Estimation of the mediation equation (3), regressing ${\bar{M}}_{. j}$ on $M_{ij}$ , shows that minority students attend schools with 63.8 percentage points ( $\hat{ω}$ ) more minority students than do White students. That is, whereas White students attend schools with an average proportion of minority students of 2.4 percent ( $\hat{γ}$ ), minority students attend schools with an average proportion of minority students of 66.2 percent ( $\hat{γ} + \hat{ω}$ ).

Earlier, we showed we can combine this information to decompose the overall ethnic achievement gap. Figure 2 presents the results of applying these different approaches. Under approach 1, the 0.46 S.D. overall gap is decomposed into the 0.07 S.D. within-school gap, $\hat{β^{W}}$ , and the 0.39 S.D. effect of segregation, $\hat{ω} \hat{β^{C}}$ , corresponding to 15.7 and 84.3 percent of the overall gap, respectively.

Figure 2.

White-minority math achievement gap decomposition under the three different approaches using Colombian data.

Approach 2 decomposes the overall gap into the 0.03 S.D. that can be attributed to the within-school gap component, $(1 - \hat{ω}) \hat{β^{W}}$ , and the 0.43 S.D. that can be attributed to the between-school gap component, $\hat{ω} \hat{β^{S}}$ , accounting for 5.7 percent and 94.3 percent of the overall gap, respectively. Finally, approach 3 decomposes the gap into 0.03 S.D. attributed to the within-school component of the gap, $(1 - \hat{ω}) \hat{β^{W}}$ , 0.39 S.D. attributed to the effect of segregation through the contextual effect of ethnicity, $\hat{ω} \hat{β^{C}}$ , and 0.05 S.D. attributed to the effect of segregation through differences in student intake, $\hat{ω} \hat{β^{W}}$ , which correspond to 5.7 percent, 84.3 percent, and 10 percent of the overall gap, respectively.

In the Colombian context, differences within schools are very small, especially in comparison with those found in the United States. For that reason, unlike in the United States, in Colombia we can conclude that most of the ethnic achievement gap is related to school-level processes, regardless of the decomposition approach we use.

Considering Additional Levels

We now use the mediation analysis framework as a device to extend the current ethnic achievement gap decomposition approaches. This section focuses on incorporating additional levels, and the next section considers multiple ethnic groups.

To consider an additional level (i.e., school district) it is enough to consider a parallel multiple mediation model (Hayes 2017) in which the school and district proportions of minority students, ${\bar{M}}_{. jk}$ and ${\bar{M}}_{. . k}$ , mediate the relationship between the ethnicity, $M_{ijk}$ , and test scores, $y_{ijk}$ , of student $i$ attending school $j$ in district $k$ . The total-effect model in this case is

\begin{matrix} y_{ijk} = α + β M_{ijk} + {e_{y}}_{ijk} \\ {e_{y}}_{ijk} ~ N (0, Ω), \end{matrix}

(8)

which is similar to equation (1), as $β$ represents the overall achievement gap between minority and White students. Figure 3 represents the outcome and mediation models.

Figure 3.

Outcome and mediation models for the achievement gap decomposition with an additional level.

The outcome model is the contextual-effect model

y_{ijk} = α + β^{W} M_{ijk} + β^{CS} {\bar{M}}_{. jk} + β^{CD} {\bar{M}}_{. . k} + {e_{y}}_{ijk},

(9)

where $β^{W}$ is the within-school (and district) achievement gap, $β^{CS}$ is the school contextual effect of ethnicity, conditional on the district ethnic composition, and $β^{CD}$ is the district contextual effect of ethnicity, conditional on the school ethnic composition. Thus, students attending schools with a 10 percentage points higher proportion of minority students are expected to score $\frac{β^{CS}}{10}$ more, independent of their own ethnicity and the district’s ethnic composition. Similarly, students in districts with a 10 percentage point higher proportion of minority students are expected to score $\frac{β^{CD}}{10}$ more, independent of their ethnicity and the school ethnic composition. This implies that $β^{CS}$ is now the school contextual effect of ethnicity within districts.

Because the model includes two mediators, there are two mediation equations

\begin{matrix} {\bar{M}}_{. jk} = γ_{S} + ω^{S} M_{ijk} + {e_{S}}_{. jk}, \forall i \in j, k \\ {\bar{M}}_{. . k} = γ_{D} + ω^{D} M_{ijk} + {e_{D}}_{. . k}, \forall i \in k . \end{matrix}

(10)

Again, we use equations (9) and (10) as devices to decompose the ethnic achievement gap, rather than as models with a particular interpretation. In this case, equation (10) allows us to estimate the between-school segregation index, $ω^{S}$ , and the between-district segregation index, $ω^{D}$ . Note that considering the district level does not imply that the between-school segregation index, $ω^{S}$ , is conditional on the district, as it is the same as in equation (3). That is, $ω^{S}$ in this case is not a within-school/between-district index. Instead, we calculate the school segregation index without considering the district to which a school belongs. Additionally, the parallel multiple mediation model allows the different mediators ( ${\bar{M}}_{. jk}$ and ${\bar{M}}_{. . k}$ ) to be correlated, but it assumes the district ethnic composition, ${\bar{M}}_{. . k}$ , and academic achievement, $y_{ijk}$ , are only linked through the district contextual effect of ethnicity, $β^{CD}$ , and not through the school contextual effect of ethnicity, $β^{CS}$ . The model also assumes the processes that drive segregation at the school level are different from those at the district level.

In a parallel multiple mediation model (Hayes 2017), given these relationships, the overall gap $β$ can be decomposed as

β = β^{W} + ω^{S} β^{CS} + ω^{D} β^{CD},

(11)

where $β^{W}$ would be interpreted as the direct effect and $ω^{S} β^{CS}$ and $ω^{D} β^{CD}$ as specific indirect effects corresponding to the school and district ethnic composition, respectively, which add up to the total indirect effect. In approach 1, this would be equivalent to decomposing the overall gap into the within-school gap, $β^{W}$ , and the effect of segregation at the school ( $ω^{S} β^{CS}$ ) and district ( $ω^{D} β^{CD}$ ) levels.

We know that $β^{CS} = β^{BS} - β^{W}$ , that is, the school contextual effect, $β^{CS}$ , is the average change in school average test scores as the school proportion of minority students increases, $β^{BS}$ , net of the effect of students’ own ethnicity, $β^{W}$ . Similarly, $β^{CD} = β^{BD} - β^{BS}$ , the district contextual effect of ethnicity, $β^{CD}$ , is equivalent to the differences in mean test scores between districts with different ethnic composition, $β^{BD}$ , net of the effect of differences between schools, $β^{BS}$ . Recognizing these in equation (11) leads to approach 2, which allows us to write the gap as a combination of the within-school, between-school/within-district, and between-district gaps as

β = (1 - ω^{S}) β^{W} + (ω^{S} - ω^{D}) β^{BS} + ω^{D} β^{BD},

(12)

where $(1 - ω^{S}) β^{W}$ is the within-school component of the gap, $(ω^{S} - ω^{D}) β^{BS}$ is the between-school/within district component, and $ω^{D} β^{BD}$ is the between-district component of the gap.

The third approach further decomposes the between-school and between-district components of the gap, by recognizing that $β^{BS} = β^{W} + β^{CS}$ and $β^{BD} = β^{W} + β^{CS} + β^{CD}$ . This leads to the six-element decomposition:

β = (1 - ω^{S}) β^{W} + (ω^{S} - ω^{D}) β^{W} + (ω^{S} - ω^{D}) β^{CS} + ω^{D} β^{W} + ω^{D} β^{CS} + ω^{D} β^{CD},

(13)

where $(1 - ω^{S}) β^{W}$ is the within-school component of the gap, $(ω^{S} - ω^{D}) β^{W}$ is the contribution of school segregation through differences in schools’ student intake, $(ω^{S} - ω^{D}) β^{CS}$ is the contribution of school segregation through the school contextual effect of ethnicity, $ω^{D} β^{W}$ is the contribution of district segregation through differences in student intake, $ω^{D} β^{CS}$ is the contribution of district segregation through differences in districts’ school composition, and $ω^{D} β^{CD}$ is the contribution of district segregation through the district contextual effect of ethnicity.

The importance of considering additional levels of the education system when applying these three decomposition approaches will likely depend on the context. The following section illustrates how the implications for policy and practice can be better informed by providing more detailed evidence of the components of the ethnic achievement gap in Colombia.

Application

As shown in the first model in Table 2, when considering additional levels of the school system, the total-effect model remains the same as in the simple case shown in Table 1, which estimates an overall White-minority gap of 0.46 S.D. In turn, the outcome model (third model in Table 2) now includes the district proportion of minority students, ${\bar{M}}_{. . k}$ .

Table 2.

Estimation Results for the Models Underlying the Decomposition Approaches

	Math
	(1)	(2)	(3)
Intercept	.031***	.046***	.076***
Intercept	(.002)	(.002)	(.002)
$M_{ijk}$	−.459***	−.072***	−.072***
$M_{ijk}$	(.006)	(.010)	(.010)
${\bar{M}}_{. jk}$		−.606***	−.305***
${\bar{M}}_{. jk}$		(.012)	(.013)
${\bar{M}}_{. . k}$			−.747***
${\bar{M}}_{. . k}$			(.014)
Observations	458,947	458,947	458,947
R ²	.013	.019	.024
Adjusted R²	.013	.019	.024
Residual standard error	.993	.991	.988
F statistic	6,149.987***	4,336.309***	3,748.908***

Note: $M_{ijk}$ is a dummy variable for minority students, ${\bar{M}}_{. jk}$ is the school proportion of minority students, and ${\bar{M}}_{. jk}$ is the district proportion of minority students. Cluster-robust standard errors are shown.

***

p < .001.

As shown in Table 2, including ${\bar{M}}_{. . k}$ in the model leaves the 0.07 S.D. estimate of the within-school ethnic achievement gap, $\hat{β^{W}}$ , unaffected. In turn, it isolates the district contextual effect of ethnicity from the original estimate of the school contextual effect of ethnicity (second model in the table). Hence, we now estimate that within districts, students attending schools with a 10 percentage point higher proportion of minority students score, on average, 0.03 S.D. ( $\frac{\hat{β^{CS}}}{10}$ ) lower, independent of their own ethnicity. Additionally, students attending districts with a 10 percentage point higher proportion of minority students score, on average, 0.07 S.D. ( $\frac{\hat{β^{CD}}}{10}$ ) lower, independent of their own ethnicity and the school they attend.

The third model in Table 2 also shows that, within districts, schools that serve only White students score, on average, 0.38 S.D. ( $\hat{β^{BS}} = \hat{β^{W}} + \hat{β^{CS}}$ ) higher than schools serving only minority students. Additionally, districts that serve only White students score, on average, 1.12 SD ( $\hat{β^{BS}} = \hat{β^{W}} + \hat{β^{CS}} + \hat{β^{CD}}$ ) higher than those that serve only minority students. In our data, this is a hypothetical comparison, as no districts have 100 percent minority students.

The estimation of mediation equations (equation 10) shows that minority students attend schools and districts with 63.8 and 25.8 percentage points more minority students than do White students ( $\hat{ω^{S}}$ and $\hat{ω^{D}}$ ), respectively. As discussed in the previous section, the different combinations of these parameters lead to the three different decompositions (see Figure 4).

Figure 4.

White-minority math achievement gap decomposition under the three different approaches considering districts.

Under approach 1, the 0.46 S.D. overall gap is decomposed into the 0.07 S.D. within-school gap, $\hat{β^{W}}$ , the 0.19 S.D. effect of school segregation, $\hat{ω^{S}} \hat{β^{CS}}$ , and the 0.19 S.D. effect of district segregation, $\hat{ω^{D}} \hat{β^{CD}}$ . These correspond to 15.7 percent, 42.4 percent, and 41.9 percent of the overall gap, respectively. Approach 2 decomposes the overall gap into the 0.03 S.D. that can be attributed to the within-school gap component, $(1 - \hat{ω^{S}}) \hat{β^{W}}$ , the 0.14 S.D. that can be attributed to the between-school gap component, $\hat{ω^{S}} \hat{β^{BS}}$ , and the 0.29 S.D. that can be attributed to the between-district gap component, $\hat{ω^{D}} \hat{β^{BD}}$ . These elements account for 5.7 percent, 31.3 percent, and 63 percent of the overall gap, respectively.

Finally, approach 3 decomposes the gap into six different components. First, 0.03 S.D. (5.7 percent) attributed to the within-school component of the gap, $(1 - \hat{ω^{S}}) \hat{β^{W}}$ ; second, 0.12 S.D. (25.3 percent) attributed to the effect of school segregation through the school contextual effect of ethnicity, $(\hat{ω^{S}} - \hat{ω^{D}}) \hat{β^{W}}$ ; third, 0.03 S.D. (6 percent) attributed to the effect of school segregation through differences in student intake, $(\hat{ω^{S}} - \hat{ω^{D}}) \hat{β^{W}}$ ; fourth, 0.02 S.D. (4.1 percent) attributed to district segregation through differences in schools’ student intake, $\hat{ω^{D}} \hat{β^{W}}$ ; fifth, 0.08 S.D. (17.1 percent) attributed to district segregation through differences in districts’ school composition, ( $\hat{ω^{D}} \hat{β^{CS}}$ ); and finally, 0.19 S.D. (41.9 percent) attributed to district segregation through differences in districts’ ethnic composition ( $\hat{ω^{D}} \hat{β^{CD}}$ ).

These three decomposition approaches provide different insights and levels of granularity about the role of various levels of the school system in the White-minority achievement gap in Colombia. Nonetheless, they all draw attention to the importance of differences between districts, with implications for research and policy. An analysis based on the original decomposition approaches would lead to recommendations focusing on schools, such as teacher training or upgrades in school infrastructure (e.g., Fryer and Levitt 2004, 2006). The results in this section, however, show that the district where a school is located is an important school-level characteristic that contributes to the ethnic achievement gap. Hence, policies tackling inequality among districts are as (if not more) important for narrowing the White-minority gap in Colombia. Extending the current decomposition approaches to incorporate the role of districts thus leads to better informed policy recommendations.

Considering Multiple Ethnic Groups

So far, we have assumed the gaps between all minority groups and White students are the same, and that the extent to which the gap can be attributed to differences between students, schools, and districts is the same for all ethnic groups. These assumptions are probably incorrect. An alternative decomposition that allows different gaps and components for each ethnic group may thus be more attractive. We now explore this extension, which can be combined with the extension discussed in the previous section to decompose the achievement gaps of multiple ethnic groups while considering additional levels of the school system (see the Appendix). To facilitate the explanation, we limit the presentation to a two-level case.

Again, mediation analysis can be used as a device to extend the current gap decomposition approaches to consider multiple ethnic groups. This time, it is enough to combine a mediation model with a multicategorical independent variable (Hayes and Preacher 2014) and a parallel multiple mediation model (Hayes 2017). For our illustration, we compare three possible ethnic categories with the White category, which is used as a reference: Afrocolombian ( $A_{ij}$ ), Indigenous ( $I_{ij}$ ), and other minorities ( $O_{ij}$ ). Here, each ethnic group is represented by a dummy variable that equals 1 when a student belongs to that ethnic group and 0 otherwise. All dummy variables are independent variables, and each one is associated with a unique gap between students of that ethnic group and White students. Should the comparison of interest be between any other two groups (e.g., Afrocolombian vs. Indigenous students), it is enough to change the reference category (e.g., to Afrocolombian students).

The total-effect model, depicted in Figure 5, is

y_{ij} = α + β_{A} A_{ij} + β_{I} I_{ij} + β_{O} O_{ij} + {e_{y}}_{ij},

(14)

where $β_{A}$ , $β_{I}$ , and $β_{O}$ represent the gap in test scores between White students and Afrocolombian, Indigenous, and other ethnic minority students, respectively.

Figure 5.

Total-effect model for the decomposition of the gaps for multiple ethnic minority groups.

Figure 6 shows the outcome and mediation models; besides having an explanatory variable for each minority group, there is also one mediator for each of them (Hayes and Preacher 2014). These mediators are the school proportion of students belonging to each of the minority groups,⁶ ${\bar{A}}_{. j}$ , ${\bar{I}}_{. j}$ , and ${\bar{O}}_{. j}$ . Consequently, the outcome model corresponds to the contextual-effect model,

y_{ij} = α + β_{A}^{W} A_{ij} + β_{I}^{W} I_{ij} + β_{O}^{W} O_{ij} + β^{CA} {\bar{A}}_{. j} + β^{CI} {\bar{I}}_{. j} + β^{CO} {\bar{O}}_{. j} + {e_{y}}_{ij},

(15)

where $β_{A}^{W}$ , $β_{I}^{W}$ , and $β_{O}^{W}$ are the within-school achievement gaps for each group, and $β^{CA}$ , $β^{CI}$ , and $β^{CO}$ are the school contextual effects of each minority group, conditional on the school proportion of other ethnic minorities.

Figure 6.

Outcome and mediation models for the decomposition of the gaps for different ethnic groups.

Following Hayes (2017), there is one mediation model for each mediator, and thus for each ethnic minority group, and each mediator is a function of the school ethnic composition in terms of all minority groups. As in the prior section, this is only a device to decompose the achievement gap for each minority group, which results in the models

\begin{matrix} {\bar{A}}_{. j} = γ_{A} + ω_{A}^{A} A_{ij} + ω_{I}^{A} I_{ij} + ω_{O}^{A} O_{ij} + {e_{SA}}_{. j}, \forall i \in j \\ {\bar{I}}_{. j} = γ_{I} + ω_{A}^{I} A_{ij} + ω_{I}^{I} I_{ij} + ω_{O}^{I} O_{ij} + {e_{SI}}_{. j}, \forall i \in j \\ {\bar{O}}_{. j} = γ_{O} + ω_{A}^{O} A_{ij} + ω_{I}^{O} I_{ij} + ω_{O}^{O} O_{ij} + {e_{SO}}_{. j}, \forall i \in j . \end{matrix}

(16)

Again, the mediation equations provide estimates of indicators of segregation, given by the differences in the mean school proportion of minority students, for each minority group and White students. For instance, $ω_{A}^{I}$ is the difference in the mean school proportion of Indigenous students between Afrocolombian and White students; if $ω_{A}^{I} = 0$ , White and Afrocolombian students attend schools with the same average proportion of Indigenous students.

Following Hayes and Preacher (2014) and Hayes (2017), each gap can be decomposed into its relative direct and (specific and total) indirect effects analogously to equation (11). There is a relative direct effect for each ethnic minority, which corresponds to the within-school achievement gap. Each mediator contributes to the specific relative indirect effects, which add up to the total relative indirect effects for each ethnic group. Importantly, equation (16) implies that the achievement gap of each ethnic group is not only a function of its own contextual effect, but also a function of the contextual effects of other minority groups. For example, the overall ethnic achievement gap for Afrocolombians can be decomposed into the direct effect, $β_{A}^{W}$ , and the indirect effect, $ω_{A}^{A} β^{CA} + ω_{A}^{I} β^{CI} + ω_{A}^{O} β^{CO}$ . In general,

\begin{matrix} β_{A} = β_{A}^{W} + ω_{A}^{A} β^{CA} + ω_{A}^{I} β^{CI} + ω_{A}^{O} β^{CO} \\ β_{I} = β_{I}^{W} + ω_{I}^{A} β^{CA} + ω_{I}^{I} β^{CI} + ω_{I}^{O} β^{CO} \\ β_{O} = β_{O}^{W} + ω_{O}^{A} β^{CA} + ω_{O}^{I} β^{CI} + ω_{O}^{O} β^{CO} . \end{matrix}

(17)

This shows that multiple-level decomposition of the overall ethnic achievement gaps for multiple ethnic groups is not possible without incorporating contextual effects of the other minority groups, which are weighted according to their relative (to that of White students) exposure to other ethnic groups. For example, the overall achievement gap for Afrocolombian students depends not only on the within-school gap and the school contextual effect of Afrocolombian students, but also on the contextual effects of Indigenous and other minority students. Empirical attempts to incorporate several groups into the analysis (Dustmann, Machin, and Schönberg 2010; Page et al. 2008; Quinn 2015) have not recognized the role of the remaining ethnic groups as part of a school-level element of the gap. The importance of this omission depends on how segregated minority groups are with respect to each other and how strong their contextual effects are.

As before, the school proportions of each ethnic minority (the mediators) can be correlated (Hayes 2017), but the model implies they are linked to academic achievement only through the contextual effect of each ethnicity (as opposed to, e.g., the school proportion of Afrocolombian students, ${\bar{A}}_{. j}$ , being linked to academic achievement through the contextual effect of Indigenous students, $β^{CI}$ ). Alternatively, performing pairwise comparisons between White students and each minority group potentially brings additional distortions into the analysis. Restricting the sample to include only two groups (e.g., White and Afrocolombian) of students incorporates measurement error into the estimated school proportion of minority students, by reducing the total number of students within each school, and thus into the decomposition analysis. The alternative of focusing only on schools that are exclusively attended by each pair of ethnic groups (e.g., White and Afrocolombian) might exclude a large proportion of schools and therefore ignore a substantial source of variation when estimating school contextual effects.

Under approach 1, equation (17) is equivalent to decomposing the overall gap into the within-school gap (e.g., $β_{A}^{W}$ ) and the effect of segregation with respect to White students (e.g., $ω_{A}^{A} β^{CA} + ω_{A}^{I} β^{CI} + ω_{A}^{O} β^{CO}$ ), which results in different exposure to the contextual effects of minority groups, in comparison with White students. Recognizing that the between-school gap for each ethnicity depends on the within-school gap and the contextual effect of such ethnicity (e.g., $β^{SA} = β_{A}^{W} + β^{CA}$ ) in equation (17) leads to approach 2:

\begin{matrix} β_{A} = (1 - ω_{A}^{A}) β_{A}^{W} + ω_{A}^{A} β^{SA} + ω_{A}^{I} β^{CI} + ω_{A}^{O} β^{CO} \\ β_{I} = (1 - ω_{I}^{I}) β_{I}^{W} + ω_{I}^{I} β^{SI} + ω_{I}^{A} β^{CA} + ω_{I}^{O} β^{CO} \\ β_{O} = (1 - ω_{O}^{O}) β_{O}^{W} + ω_{O}^{O} β^{SO} + ω_{O}^{A} β^{CA} + ω_{O}^{I} β^{CI}, \end{matrix}

(18)

where, for example, $(1 - ω_{A}^{A}) β_{A}^{W}$ is the within-school component of the overall Afrocolombian-White gap, $ω_{A}^{A} β^{SA}$ is the component of the gap that can be attributed to the between-school gap for Afrocolombian students, and $ω_{A}^{I} β^{CI} + ω_{A}^{O} β^{CO}$ is the effect of differential exposure to other ethnic minority groups.

Intuitively, the overall Afrocolombian-White between-school gap, $β^{SA}$ , is the result of comparing schools with an increasing proportion of Afrocolombian students, while keeping everything else constant. Nonetheless, schools with the same proportion of Afrocolombian students may differ in their proportion of Indigenous and other ethnic minority students. Hence, when decomposing the gap we need to acknowledge these additional differences by including components of the other ethnic minority groups.

Approach 3 suggests we decompose the between-school component of the gap, recognizing that $β^{SA} = β_{A}^{W} + β^{CA}$ , $β^{SI} = β_{I}^{W} + β^{CI}$ , and $β^{SO} = β_{O}^{W} + β^{CO}$ , which leads to

\begin{matrix} β_{A} = (1 - ω_{A}^{A}) β_{A}^{W} + ω_{A}^{A} β_{A}^{W} + ω_{A}^{A} β^{CA} + ω_{A}^{I} β^{CI} + ω_{A}^{O} β^{CO} \\ β_{I} = (1 - ω_{I}^{I}) β_{I}^{W} + ω_{I}^{I} β_{I}^{W} + ω_{I}^{I} β^{CI} + ω_{I}^{A} β^{CA} + ω_{I}^{O} β^{CO} \\ β_{O} = (1 - ω_{O}^{O}) β_{O}^{W} + ω_{O}^{O} β_{O}^{W} + ω_{O}^{O} β^{CO} + ω_{O}^{A} β^{CA} + ω_{O}^{I} β^{CI}, \end{matrix}

(19)

where, taking the Afrocolombian-White achievement gap as an example, $(1 - ω_{A}^{A}) β_{A}^{W}$ is the part of the gap that can be attributed to the within-school gap, $ω_{A}^{A} β_{A}^{W}$ is the component that can be attributed to segregation through differences in schools’ student intake, and $ω_{A}^{A} β^{CA} + ω_{A}^{I} β^{CI} + ω_{A}^{O} β^{CO}$ is the effect of segregation through the contextual effect of ethnicity, which can be further decomposed to recognize the contribution of differential exposure to each minority group.

Application

As discussed in the prior section, the multiple-ethnic-group decomposition uses the parameters of the total and outcome models presented in Table 3. As shown, the 0.59 S.D. White-Afrocolombian achievement gap, $\hat{β_{A}}$ , is wider than the 0.52 S.D. White-Indigenous, $\hat{β_{I}}$ , and 0.14 S.D. White-other, $\hat{β_{O}}$ , achievement gaps. These differences in overall achievement gaps underscore the importance of distancing from the aggregated measure of the gap presented earlier to consider the richer information of multiple ethnic minorities.

Table 3.

Estimation Results for the Models Underlying the Decomposition Approaches

	Math
	(1)	(2)
Intercept	.031***	.043***
Intercept	(.002)	(.002)
$A_{ij}$	−.592***	−.089***
$A_{ij}$	(.008)	(.015)
$I_{ij}$	−.518***	−.112***
$I_{ij}$	(.011)	(.017)
$O_{ij}$	−.142***	−.026⁺
$O_{ij}$	(.011)	(.015)
${\bar{A}}_{. j}$		−.707***
${\bar{A}}_{. j}$		(.017)
${\bar{I}}_{. j}$		−.669***
${\bar{I}}_{. j}$		(.022)
${\bar{O}}_{. j}$		−.172***
${\bar{O}}_{. j}$		(.022)
Observations	458,947	458,947
R ²	.016	.021
Adjusted R²	.016	.021
Residual standard error	.992	.990
F statistic	2,42.402***	1,609.472***

Note: $A_{ij}$ , $I_{ij}$ , $O_{ij}$ are dummy variables for Afrocolombian, Indigenous, and other minority students; and ${\bar{A}}_{. j}$ , ${\bar{I}}_{. j}$ , and ${\bar{O}}_{. j}$ are the school proportions of Afrocolombian, Indigenous, and other minority students. Cluster-robust standard errors are shown.

p < .10. ***p < .001.

Table 3 shows that within schools, Afrocolombian, Indigenous, and other minority groups score 0.09 S.D. ( $\hat{β_{A}^{W}}$ ), 0.11 S.D. ( $\hat{β_{I}^{W}}$ ), and 0.03 S.D. ( $\hat{β_{O}^{W}}$ ) lower than their White peers. Additionally, a 10 percentage point increase in the proportion of Afrocolombian, Indigenous, and other minority students is linked to an average decrease of 0.71 S.D. ( $\frac{\hat{β^{CA}}}{10}$ ), 0.67 S.D. ( $\frac{\hat{β^{CI}}}{10}$ ), and 0.17 S.D. ( $\frac{\hat{β^{CO}}}{10}$ ) in students’ math test scores, respectively, independent of their own ethnicity.

As discussed earlier, considering multiple ethnic groups requires considering differences in exposure to them. These differences are displayed in Table 4, which shows that minority groups are more exposed to other students of the same ethnic minority, in comparison with White students. For example, Afrocolombian students attend schools with an average proportion of Afrocolombian students 0.7 percentage points ( $\hat{ω_{A}^{A}}$ ) higher than the schools White students attend. Minority students’ exposure to other minority groups is similar to that of White students. For example, Afrocolombian students attend schools with the same average proportion of Indigenous students as the schools that White students attend ( $\hat{ω_{A}^{I}} = 0$ ).

Table 4.

School Segregation Indices for All Ethnic Minority Groups

Group	% Afrocolombian	% Indigenous	% Other
Afrocolombian	.70	.00	.02
Indigenous	.01	.59	.02
Other	.04	.02	.43

Each of the three decomposition approaches studied here result from the combination of these parameters. These results are shown in Figure 7. Taking the White-Afrocolombian achievement gap as an example, approach 1 attributes 0.09 S.D. (15.1 percent) of the White-Afrocolombian achievement gap to the within-school gap, $\hat{β_{A}^{W}}$ , and the remainder of the gap, $\hat{ω_{A}^{A}} \hat{β^{CA}} + \hat{ω_{A}^{I}} \hat{β^{CI}} + \hat{ω_{A}^{O}} \hat{β^{CO}}$ , to the effect of a differential exposure to other minority groups.

Figure 7.

White-minority math achievement gap decomposition under the three different approaches considering multiple ethnic minorities.

Approaches 2 and 3 attribute 0.03 S.D. (4.5 percent) of the White-Afrocolombian gap to the within-school component of the gap, $(1 - \hat{ω_{A}^{A}}) \hat{β_{A}^{W}}$ , and 0.01 S.D. (1 percent) of this gap to the effect of a differential exposure to other minority groups, $\hat{ω_{A}^{I}} \hat{β^{CI}} + \hat{ω_{A}^{O}} \hat{β^{CO}}$ . The difference between these two approaches is that approach 2 attributes the remaining 0.56 S.D. (94.5 percent) of the gap to the between-school component of the gap, $\hat{ω_{A}^{A}} \hat{β^{SA}}$ . In turn, approach 3 provides two additional components: 0.5 S.D. (83.9 percent) for the effect of segregation via differential exposure to the contextual effect of same ethnic group, $\hat{ω_{A}^{A}} \hat{β^{CA}}$ , and 0.06 S.D. (10.6 percent) for the effect of segregation via differential student intake, $\hat{ω_{A}^{A}} \hat{β^{WA}}$ .

Alternative decompositions can be accommodated. For example, if it is relevant to know which minority group contributes the most to the ethnic achievement gap, the school-level component of the gap can be further decomposed to analyze the individual contributions of the contextual effects of each minority group.

On Serial Multiple Mediation Models

As discussed in the prior sections, the extensions to the three existing gap decomposition approaches to consider multiple levels and multiple ethnic groups are based on a parallel multiple mediation model. An alternative model, the serial multiple mediation model, considers multiple mediations. We now explain why the extensions are not based on a serial multiple mediation model. We illustrate the discussion using the extension to include multiple levels, but a similar argument can be made about the extension to consider multiple ethnic groups.

The parallel multiple mediation model assumes school segregation and district segregation are correlated but separate processes. Alternatively, one could use serial multiple mediation models. Figure 8 adds a new path between the school proportion of minority students, ${\bar{M}}_{. jk}$ , and the district proportion of minority students, ${\bar{M}}_{. . k}$ . In model (i), on the left, the assumption is that school segregation patterns influence district segregation patterns. In model (ii), on the right, we assume that district segregation patterns influence school segregation patterns. Note, however, that there is no a priori reason to make any of these assumptions, as we will explain.

Figure 8.

Outcome and mediation models for the achievement gap decomposition with an additional level using two alternative serial multiple mediation models.

The outcome model and the mediation equation for the school proportion of minority students, ${\bar{M}}_{. jk}$ , in model (i) in Figure 8 are the same as in equations (9) and (10), respectively, for the parallel multiple mediation model. The mediation equation for the district proportion of minority students, ${\bar{M}}_{. . k}$ , then becomes

{\bar{M}}_{. . k} = γ_{D} + ω_{W}^{D} M_{ijk} + ω_{S}^{D} {\bar{M}}_{. jk} + {e_{D}}_{. . k}, \forall i, j \in k .

(20)

However, it is not possible to estimate $ω_{W}^{D}$ from equation (20). Intuitively, district segregation, $ω^{D}$ , is now estimated conditional on school ethnic composition, that is, while holding constant the proportion of minority students, ${\bar{M}}_{. jk}$ , in the school each student $i$ attends, effectively making $ω_{W}^{D}$ the within-school district segregation. Because all students in the same school are also in the same district, this implies that the district proportion of minority students, ${\bar{M}}_{. . k}$ , is also held constant when attempting to estimate $ω_{W}^{D}$ . With no variation in ${\bar{M}}_{. . k}$ , $ω_{W}^{D}$ cannot be estimated. In other words, conditional on ${\bar{M}}_{. jk}$ , the covariance between $M_{ijk}$ and ${\bar{M}}_{. . k}$ is zero, because ${\bar{M}}_{. . k}$ is a constant, and hence $ω_{W}^{D} = 0$ .

The outcome and mediation equations for the district proportion of minority students, ${\bar{M}}_{. . k}$ , in model (ii) in Figure 8 are the same as in equations (9) and (10), respectively, for the parallel multiple mediation model. The mediation equation for the school proportion of minority students, ${\bar{M}}_{. jk}$ , becomes

{\bar{M}}_{. jk} = γ_{S} + ω_{W}^{S} M_{ijk} + ω_{D}^{S} {\bar{M}}_{. . k} + {e_{S}}_{. jk}, \forall i, j \in k .

(21)

In this case, it is possible to estimate the within-district school segregation parameter, $ω_{W}^{S}$ , and the contextual effect of the district proportion of minority students, $ω_{D}^{S}$ . Using the mediation analysis framework, this model allows one to decompose the overall ethnic achievement gap, $β$ , into one direct effect, $β^{W}$ , and three indirect effects, $ω_{W}^{S} β^{CS}$ , $ω^{D} β^{CD}$ , and $ω_{D}^{S} ω^{D} β^{CD}$ . In terms of the three decomposition approaches, this results in the following:

\begin{matrix} Approach 1 \\ β = β^{W} + (ω_{W}^{S} + ω_{D}^{S} ω^{D}) β^{CS} + ω^{D} β^{CD} \\ Approach 2 \\ β = (1 - ω_{W}^{S} - ω_{D}^{S} ω^{D}) β^{W} + (ω_{W}^{S} + ω_{D}^{S} ω^{D} - ω^{D}) β^{BS} + ω^{D} β^{BD} \\ Approach 3 \\ β = (1 - ω_{W}^{S} - ω_{D}^{S} ω^{D}) β^{W} + (ω_{W}^{S} + ω_{D}^{S} ω^{D} - ω^{D}) β^{W} + (ω_{W}^{S} + ω_{D}^{S} ω^{D} - ω^{D}) β^{CS} + \\ ω^{D} β^{W} + ω^{D} β^{CS} + ω^{D} β^{CD} . \end{matrix}

Note, however, that $ω^{S} = ω_{W}^{S} + ω_{D}^{S} ω^{D}$ , and hence all these decompositions, are equivalent to those in the “Considering Additional Levels” section, derived using a parallel multiple mediation model. Intuitively, the serial multiple mediation model further decomposes school segregation, $ω^{S}$ , into a direct effect, $ω_{W}^{S}$ , and an indirect effect, $ω_{D}^{S} ω^{D}$ . Arguably, these equations for the decomposition approaches could be reparametrized to further decompose the effects of school segregation, and to understand the contribution of within-school segregation separately from the district contextual effect of school segregation; that would then be a motivation to use serial mediation models.

Note, however, that the relation between the district and school ethnic composition is only deterministic when all districts have the same number of schools, and schools within these districts are the same size. In the more conventional situation in which schools within districts are different sizes, there is no deterministic relationship between these two; districts with the same proportion of minority students, $0 < {\bar{M}}_{. . k} < 1$ , can have schools with proportion $0 \leq {\bar{M}}_{. jk} \leq 1$ in different shares. Equivalently, schools with the same proportion of minority students, $0 \leq {\bar{M}}_{. jk} \leq 1$ , can be located in districts with very different proportions of minority students, $0 < {\bar{M}}_{. . k} < 1$ . This is the case for Colombia, where there is no clear relationship between the ethnic composition of schools and districts (see Figure 9).

Figure 9.

Relationship between the district proportion of minority students and the school proportion of minority students.

We encourage researchers to explore the potential extensions the mediation framework offers, using either parallel multiple mediation models or serial multiple mediation models as best suits their research context and research questions. If, in a particular context, there is reason to believe that district segregation patterns influence school segregation patterns, using serial multiple mediation models and reparametrizing the decomposition approaches accordingly might result in improved insights to the ethnic achievement gaps. Otherwise, extending the decomposition approaches using parallel multiple mediation models will likely provide enough insights about the contribution of within-school and between-school differences to the overall ethnic achievement gap.

Discussion and Conclusions

In this article, we examine three different approaches for a multilevel decomposition of the ethnic achievement gap. We argue that in approach 1, Cook and Evans (2000) and Fryer and Levitt (2004, 2006) decomposed the Black-White achievement gap into the within-school gap and the effect of segregation. In approach 2, Hanushek and Rivkin (2006) decomposed the gap into a part that is attributable to the within-school gap and a part attributed to the between-school gap. Finally, in approach 3, Reardon (2008) and Page et al. (2008) decomposed the gap into three parts: one that is attributable to the within-school gap, a second that is linked to the effect of school segregation through differences in student intake, and a third component attributed to the effect of school segregation through differences in school composition.

Each of these decomposition approaches is useful for different research questions and policy decisions. For example, if the debate is about school segregation, approach 1 provides a more direct way to analyze its potential effects on the achievement gap. If one’s focus is the within- and between-school components of the gap, approach 2 is appropriate. If one wants to examine the mechanisms behind the between-school component of the gap, approach 3 can be used.

The initial approaches are limited, however, in that they do not appropriately consider the role of additional levels of the school system (e.g., districts) or multiple ethnic groups (beyond the binary White-minority comparison), which restricts the kind of policy recommendations that follow from applying these methods. We address these limitations by extending the three decomposition approaches to consider multiple levels of analysis and for multiple ethnic groups.

The role of between-district differences (or school districts in the United States) has only before been considered using Cook and Evans’s (2000) method of including school (in this case, district) fixed effects into a Kitagawa-Oaxaca-Blinder decomposition. Arteaga and Glewwe (2019) proposed using this method to examine the extent to which the gap between Indigenous and non-Indigenous students could be attributed to community effects in Peru. Use of the Kitagawa-Oaxaca-Blinder (Blinder 1973; Kitagawa 1955; Oaxaca 1973) decomposition implies a different kind of decomposition than the one we explored here.

Similarly, Dustmann et al. (2010) considered different ethnic groups by using approach 1. Quinn (2015) used approach 3 to decompose the U.S. Black-White gap and includes dummy variables for other ethnic groups to ensure that all possible observations are included when comparing Black and White students (instead of Black and non-Black students). Nonetheless, these studies do not examine the role that other minority students play in explaining the Black-White achievement gap, as explained in the “Considering Multiple Ethnic Groups” section. Using Colombian data for illustration, we show that the importance of this omission depends on the magnitude of the contextual effects and the relative exposure to other ethnic groups. In Colombia, despite strong contextual effects of Afrocolombian and Indigenous students, the little differential exposure to other minority groups suggests this is a small component of the gaps.

This illustration also showed the potential of the two extensions to transform the discussion around ethnic achievement gaps. In our decomposition, we saw that districts play a role at least as important as schools. Therefore, it is worth incorporating districts into the discussion about ethnic achievement gaps. Similarly, considering multiple ethnic groups showed that important heterogeneity among ethnic minority groups is ignored when treating ethnic minorities as a single group. More important, the extension to the decomposition brings the discussion about segregation among minority groups to the forefront of the conversation about ethnic achievement gaps.

We show that the existing decomposition approaches are mathematically equivalent to a mediation problem, and we used this equivalence to extend these approaches. Our extensions consider multiple levels and multiple ethnic groups, but this equivalence can be used to consider a wider range of extensions. For example, extending the decompositions to understand if they are different in private or public schools could be modeled as a moderated mediation model. We also show that serial multiple mediation models offer the possibility to further extend these decomposition approaches in contexts where there is reason to believe that school segregation is influenced by district segregation.

Limitations

The methodological discussion about the ethnic gap decomposition is tailored to its substantive application to the Colombian context; this means we consider only three levels observed in the data—students, schools, and districts—and three ethnic minority groups. This restricts the detailed discussion of additional levels (e.g., cohorts and classrooms) and ethnic groups (which in England, for example, are often reported using many more categories than in Colombia or the United States). Nonetheless, the decomposition method can be generalized to these additional levels and groups.

The structure of the data may also be more complex than the hierarchical nesting of students within schools within districts studied here. For example, neighborhoods may be a relevant level to consider: students from multiple neighborhoods may attend the same school, yet not all students from the same neighborhood will attend the same school. If a cross-classified (like in the example) or multiple membership structure provides a better representation of ethnic achievement gaps, the decomposition in this article may overstate the contribution of schools and districts to the overall ethnic achievement gap (Browne, Goldstein, and Rasbash 2001; Goldstein 1994; Leckie 2013a, 2013b; Martínez 2012).

Another limitation of the method for decomposing the ethnic achievement gap, as presented here, is that it does not allow any interactions. Therefore, the models assume that the contextual effect of each minority group is the same for all ethnic groups or, equivalently, that within-school gaps are the same regardless of the proportion of minority students in the schools and districts. In the context of the Kitagawa-Oaxaca-Blinder decomposition, which Hou (2014) showed also fits under the mediation analysis framework, this is resolved by estimating separate equations for each group; this approach can be further explored to expand the decomposition approaches presented here. Additionally, potential equivalences with decomposition techniques that have been proposed for the analysis of segregation (e.g., Yamaguchi 2017) merit further analysis but are beyond the scope of this article.

We show that the current two-group two-level ethnic achievement gap decomposition approaches can be reformulated as a mediation problem. We use this equivalence to extend the existing decomposition approaches to consider multiple-group and multiple-level settings. These extensions have the potential to provide important insights, and hence policy implications, for the reduction of these gaps, as illustrated by the application to Colombian data. Mediation analysis provides a flexible framework to work within. We encourage researchers to incorporate the achievement gap decomposition as a descriptive step in analyses of ethnic achievement gaps and to take advantage of the unifying nature of the mediation analysis framework to tailor their own decomposition approaches.

Supplemental Material

sj-docx-1-smx-10.1177_00811750221099503 – Supplemental material for Decomposing Ethnic Achievement Gaps across Multiple Levels of Analysis and for Multiple Ethnic Groups

Supplemental material, sj-docx-1-smx-10.1177_00811750221099503 for Decomposing Ethnic Achievement Gaps across Multiple Levels of Analysis and for Multiple Ethnic Groups by Beatriz Gallo Cordoba, George Leckie and William J. Browne in Sociological Methodology

Footnotes

Appendix: Generalization of the Decomposition Approaches

Figures A1 and A2 show a generalized version of the model that allows decomposing the overall gap of $G$ groups into $L$ levels. In the general case, the categories of the independent variable (groups) are indexed by $g = 1, \dots, G$ (where 1 is the reference group), and the levels are indexed by $l = 1, \dots, L$ . The $ijk \dots$ subscripts are omitted to avoid confusion. The variables $y$ and $D_{g}$ always vary by individual (level 1) and the mediators vary at level $l$ . The subscripts of the coefficients indicate the variable they multiply within each equation. The subscripts indicate effects that group $g$ “receives” (e.g., the overall gap $β_{g}$ ), and superscripts indicate effects the group “produces” (e.g., the contextual effect of group $g$ at level $l$ , $β^{\lg}$ ).

The overall gaps are given by

(A1)

y = α + \sum_{g = 2}^{G} β_{g} D_{g} + e_{y} .

The outcome model is equivalent to the contextual-effect model

(A2)

y = α + \sum_{g = 2}^{G} β_{g}^{W} D_{g} + \sum_{l = 1}^{L} \sum_{g = 2}^{G} β^{\lg} E [D_{g} | l = l] + e_{y} .

Therefore, the $(G - 1) \times L$ mediation equations are

(A3)

\begin{matrix} E [D_{2} | l = 1] = ω_{012} + \sum_{g' = 2}^{G} ω_{g'}^{12} D_{g'} + e_{12} \\ ⋮ \\ E [D_{G} | l = 1] = ω_{01 G} + \sum_{g' = 2}^{G} ω_{g'}^{1 G} D_{g'} + e_{1 G} \\ ⋮ \\ E [D_{g} | l = l] = ω_{0 \lg} + \sum_{g' = 2}^{G} ω_{g'}^{\lg} D_{g'} + e_{\lg} \\ ⋮ \\ E [D_{G} | l = L] = ω_{0 LG} + \sum_{g' = 2}^{G} ω_{g'}^{LG} D_{g'} + e_{LG}, \end{matrix}

and the $G - 1$ gaps can be decomposed into multiple levels as

approach 1:

(A4)

β_{g} = β_{g}^{W} + \sum_{l = 1}^{L} \sum_{g'}^{'} ω_{g}^{\lg'} β^{\lg'}, g' = 2, . . ., G

approach 2:

(A5)

β_{g} = (1 - ω_{g}^{1 g}) β_{g}^{W} + \sum_{l = 2}^{L} (ω_{g}^{(l - 1) g} - ω_{g}^{\lg}) β^{B (l - 1) g} + ω_{g}^{Lg} β^{BLg} + \sum_{l = 1}^{L} \sum_{g' \neq g} ω_{g}^{\lg'} β^{\lg'}

where β^{Blg} = β_{g}^{W} + \sum_{l = 1}^{L} β^{\lg} .

approach 3:

(A6)

β_{g} = (1 - ω_{g}^{1 g}) β_{g}^{W} + \sum_{l = 1}^{L} \sum_{g'}^{'} ω_{g}^{\lg'} β^{\lg'} + \sum_{l = 2}^{L} (ω_{g}^{(l - 1) g} - ω_{g}^{\lg}) β_{g}^{W} + ω_{g}^{Lg} β_{g}^{W} .

Acknowledgements

The comments and suggestions from the editors and three anonymous reviewers have greatly contributed to the improvement of the original version of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Economic and Social Research Council (grants ES/J50015X/1; ES/R010285/1).

ORCID iD

Beatriz Gallo Cordoba

Supplemental Material

Supplemental material for this article is available online.

Notes

Author Biographies

Beatriz Gallo Cordoba is research fellow in quantitative data/statistics at the Monash Centre for Youth Policy and Education Practice, Faculty of Education at Monash University. Her research interests are in how alternative statistical modelling and measurement techniques shed light on specific aspects of inequality. Substantively, she is interested in inequality in educational outcomes.

George Leckie is professor of social statistics and co-director of the Centre for Multilevel Modelling (CMM) at the School of Education at the University of Bristol. His methodological research interests are in the development, application and dissemination of multilevel and related models to analyse educational and other data. His substantive interests focus on design, analysis, and communication issues surrounding school performance measures and league tables.

William J. Browne is professor of statistics and co-director of the Centre for Multilevel Modelling (CMM) at the School of Education at the University of Bristol. His research interests are in statistical modelling, from the development of statistical methods to fit realistically complex statistical models to describe real-life problems, through the implementation of those models in statistical software to the application of the methods in several application areas.

References

Arteaga

Irma

Glewwe

Paul

. 2019. “Do Community Factors Matter? An Analysis of the Achievement Gap between Indigenous and Non-indigenous Children in Peru.” International Journal of Educational Development 65(1):80–91.

Baron

Reuben M.

Kenny

David A.

1986. “The Moderator–Mediator Variable Distinction in Social Psychological Research: Conceptual, Strategic, and Statistical Considerations.” Journal of Personality and Social Psychology 51(6):1173–82.

Bidwell

Charles E.

Kasarda

John D.

1975. “School District Organization and Student Achievement.” American Sociological Review 40(1):55–70.

Blinder

Alan S.

1973. “Wage Discrimination: Reduced Form and Structural Estimates.” Journal of Human Resources 8(4):436–55.

Brincks

Ahnalee M.

Enders

Craig K.

Llabre

Maria M.

Bulotsky-Shearer

Rebecca J.

Prado

Guillermo

Feaster

Daniel J.

2017. “Centering Predictor Variables in Three-Level Contextual Models.” Multivariate Behavioral Research 52(2):149–63.

Browne

William J.

Goldstein

Harvey

Rasbash

Jon

. 2001. “Multiple Membership Multiple Classification (MMMC) Models.” Statistical Modelling: An International Journal 1:103–124.

Burstein

Leigh

. 1980. “The Analysis of Multilevel Data in Educational Research and Evaluation.” Review of Research in Education 8:158–233.

Castellano

Katherine E.

Rabe-Hesketh

Sophia

Skrondal

Anders

. 2014. “Composition, Context, and Endogeneity in School and Teacher Comparisons.” Journal of Educational and Behavioral Statistics 39(5):333–67.

Coleman

James

Campbell

Ernest Q.

McPartland

James

Mood

Alexander M.

Weinfeld

Frederic D.

York

Robert L.

1966. “Equality of Educational Opportunity.” Technical report. Washington, DC: National Center for Educational Statistics.

10.

Cook

Michael D.

Evans

William N.

2000. “Families or Schools? Explaining the Convergence in White and Black Academic Performance.” Journal of Labor Economics 18(4):729–54.

11.

Cronbach

Lee J.

Deken

Joseph E.

Webb

Noreen

. 1976. “Research on Classrooms and Schools: Formulation of Questions, Design and Analysis.” Technical report. Stanford, CA: Stanford Evaluation Consortium.

12.

DANE. 2007. “Colombia una Nación Multicultural.” Technical report. Bogota, Colombia: DANE.

13.

Dustmann

Christian

Machin

Stephen

Schönberg

Uta

. 2010. “Ethnicity and Educational Achievement in Compulsory Schooling.” Economic Journal 120(546):F272–F297.

14.

Fryer

Roland G.

Levitt

Steven D.

2004. “Understanding the Black-White Test Score Gap in the First Two Years of School.” Review of Economics and Statistics 86(2):447–64.

15.

Fryer

Roland G.

Levitt

Steven D.

2006. “The Black-White Test Score Gap through Third Grade.” American Law and Economics Review 8(2):249–81.

16.

Goldstein

Harvey

. 1994. “Multilevel Cross-Classified Models.” Sociological Methods & Research 22(3):364–75.

17.

Hanushek

Eric A.

Rivkin

Steven G.

2006. “School Quality and the Black-White Achievement Gap.” NBER Working Paper No. 12651. Cambridge, MA: National Bureau of Economic Research.

18.

Hayes

Andrew F.

2017. Introduction to Mediation, Moderation, and Conditional Process Analysis: A Regression-Based Approach. 2nd ed. New York: Guilford.

19.

Hayes

Andrew F.

Preacher

Kristopher J.

2014. “Statistical Mediation Analysis with a Multicategorical Independent Variable.” British Journal of Mathematical and Statistical Psychology 67(3):451–70.

20.

Hou

Feng

. 2014. “A General Approach to Effect Decomposition.” Social Science Quarterly 95(3):894–904.

21.

Kitagawa

Evelyn M.

1955. “Components of a Difference between Two Rates.” Journal of the American Statistical Association 50(272):1168–94.

22.

Leckie

George

. 2013a. “Cross-Classified Multilevel Models—Concepts.” LEMMA VLE Module 12. Retrieved May 9, 2022. http://www.bristol.ac.uk/cmm/learning/course.html.

23.

Leckie

George

. 2013b. “Multiple Membership Multilevel Models—Concepts.” LEMMA VLE Module 13. Retrieved May 9, 2022. http://www.bristol.ac.uk/cmm/learning/course.html.

24.

MacKinnon

David P.

Lockwood

Chondra M.

Williams

Jason

. 2004. “Confidence Limits for the Indirect Effect: Distribution of the Product and Resampling Methods.” Multivariate Behavioral Research 39(1):99–128.

25.

Martínez

José Felipe

. 2012. “Consequences of Omitting the Classroom in Multilevel Models of Schooling: An Illustration Using Opportunity to Learn and Reading Achievement.” School Effectiveness and School Improvement 23(3):305–326.

26.

Mohammadpour

Ebrahim

Ghafar

Mohamed Najib Abdul

. 2014. “Mathematics Achievement as a Function of within- and between-School Differences.” Scandinavian Journal of Educational Research 58(2):189–221.

27.

Mundlak

Yair

. 1978. “On the Pooling of Time Series and Cross Section Data.” Econometrica 46(1):69–85.

28.

Oaxaca

Ronald

. 1973. “Male-Female Wage Differentials in Urban Labor Markets.” International Economic Review 14(3):693–709.

29.

Page

Lindsay

Murnane

Richard

Willett

John

. 2008. “Trends in the Black-White Achievement Gap: Clarifying the Meaning of within- and between-School Achievement Gaps.” NBER Working Paper No. 14213. Cambridge, MA: National Bureau of Economic Research.

30.

Preacher

Kristopher J.

Selig

James P.

2012. “Advantages of Monte Carlo Confidence Intervals for Indirect Effects.” Communication Methods and Measures 6(1):77–98.

31.

Quinn

David M.

2015. “Kindergarten Black–White Test Score Gaps.” Sociology of Education 88(2):120–39.

32.

Reardon

Sean F.

2008. “Thirteen Ways of Looking at the Black-White Test Score Gap.” Working Paper 8. Stanford, CA: Stanford Institute for Research on Education Policy & Practice.

33.

Rothstein

Richard

. 2004. Class and Schools: Using Social, Economic, and Educational Reform to Close the Black-White Achievement Gap. New York: Teachers College Press.

34.

Selig

James P.

Preacher

Kristopher J.

2008. “Monte Carlo Method for Assessing Mediation: An Interactive Tool for Creating Confidence Intervals for Indirect Effects.” Computer software. Retrieved May 9, 2022. http://quantpsy.org.

35.

Vanderweele

Tyler J.

Vansteelandt

Stijn

. 2009. “Conceptual Issues Concerning Mediation, Interventions and Composition.” Statistics and Its Interface 2(4):457–68.

36.

Wenglinsky

Harold

. 2009. “How Money Matters: The Effect of School District Spending on Academic Achievement.” Sociology 70(3):221–37.

37.

Yamaguchi

Kazuo

. 2017. “Decomposition Analysis of Segregation.” Sociological Methodology 47:246–73.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.06 MB