Abstract
Mediation has played a critical role in developmental theory and research. Yet, developmentalists rarely discuss the methodological challenges of establishing causality in mediation analysis or potential strategies to improve the identification of causal mediation effects. In this article, we discuss the potential outcomes framework from statistics as a means for highlighting several fundamental challenges of establishing causality in mediation analysis, including the difficulty of meeting the key assumption of sequential ignorability, even in experimental studies. We argue that this framework—which, although commonplace in other fields, has not yet been taken up in developmental science—can inform solutions to these challenges. Based on the framework, we offer a series of recommendations for improving causal inference in mediation analysis, including an overview of best practices in both study design and analysis, as well as resources for conducting analysis. In doing so, our overall objective in this article is to support the use of rigorous methods for understanding questions of mechanism in developmental science.
Developmental science has long been invested in understanding mediation, or the process by which one variable causally affects another via an intermediary mechanism (MacKinnon et al., 2007). Indeed, questions of mediation—including not just whether but how or why a stimulus produces an outcome—form the basis of most foundational theories of human development (e.g., Bandura, 1986; Bronfenbrenner, 1977; Piaget, 1969; Vygotsky, 1981). In addition to being useful in basic scientific inquiry, mediation is also increasingly utilized by applied psychologists to understand the mechanisms through which programs and interventions may influence children’s outcomes (Schindler et al., 2019).
Despite its conceptual and practical importance, testing mediation is fraught with methodological challenges. In recent years, researchers have leveraged a variety of methods to improve causal inference in child development research (e.g., Foster, 2010; Miller et al., 2016). Although the field has made tremendous progress overall, most discussions and advice in developmental science concerning mediation have focused on the logic of mediation and the correct estimation of standard errors (e.g., Baron & Kenny, 1986; Dearing & Hamilton, 2006; MacKinnon et al., 2007), whereas the challenges of identifying causal mediation effects have rarely been discussed in the field. In particular, although several threats to validity hamper causal interpretations in most mediation analyses, even when researchers employ longitudinal or experimental data (Maxwell & Cole, 2007; VanderWeele, 2015), research in developmental science often fails to explicitly recognize threats to internal validity or discuss strategies to address them. We argue that a better understanding of such threats and approaches to their mitigation is fundamental for advancing developmental theory and practice, including the design, refinement, and scaling of effective intervention programming.
In this article, we overview a general framework (and language) from statistics that makes explicit a key assumption underlying causality in mediation analysis: sequential ignorability. We argue that a clear understanding of sequential ignorability makes threats to validity more visible, thus motivating and facilitating the use of novel study designs and sensitivity checks that can strengthen the identification of causal mediation effects in developmental science. Rather than providing a detailed tutorial on how to conduct causal mediation according to the proposed framework, the current article seeks, for the first time in developmental science, to explicitly discuss the challenges of establishing causality in mediation analyses, to offer a brief overview of current best practices, and to refer readers to additional resources (including articles, books, and software) for further guidance. To enhance interpretation and relevance, throughout the manuscript we employ two hypothetical examples. The first includes an experimental evaluation of a fictitious, randomly assigned early childhood development (ECD) intervention that aims to increase parental engagement in stimulating activities (such as reading and playing) with children and, in turn, to support children’s cognitive development. The second includes an observational study exploring the effect of poverty on children’s cognitive development, mediated by parental engagement in stimulating activities.
Traditional Approaches to Estimating Mediation in Developmental Science
Mediation analysis is designed to examine how an antecedent variable, which we call the exposure variable (X), leads to or affects an outcome variable (Y) through a mediating variable (M) in a causal process
With over 88,000 citations in Google Scholar, the most common approach to quantifying mediation in developmental science is the product method (Baron & Kenny, 1986), which can be applied using ordinary least squares (OLS) or structural equation modeling (SEM). The product method employs two regression models to estimate direct and indirect effects (see also Figure 1, Panel A). In the first model, Y is regressed on X and M (Equation 1). In the second model, M is regressed on X (Equation 2). The direct effect is represented by the coefficient

Mediation and Confounders.
Despite its popularity, the product method is often used to estimate how directly or indirectly variables are related to one another, without a clear articulation of the assumptions required to identify causal links (i.e., identifying assumptions; Keele et al., 2015; VanderWeele, 2015). We argue that a formal framework and language to think about the plausibility of causal mediation effects is fundamental to (1) better understand potential threats to the validity of mediation-related inferences, (2) design studies that minimize some of these threats, thus improving inferences, and (3) apply existing statistical tools (e.g., sensitivity checks and software) to improve mediation analysis in developmental science. In what follows, we review a general framework widely used in causal inference—the potential outcomes framework (POF; Rubin, 1974)—and explain its relevance for understanding and overcoming key challenges in mediation analysis within the developmental sciences.
A General Framework (and Language) for Causal Mediation
The POF (Rubin, 1974) offers a formal framework to think about causation. This framework was developed in the field of statistics; Neyman (see Neyman et al., 1990) introduced it in the context of experimental data, and Rubin (2005) extended it to define causal effects in both experimental and observational studies. Using the language of the POF, the effect of an exposure or treatment X (which can take the values of
In reality, these individual effects are not identifiable, as it is possible to observe only one potential outcome (i.e., one Yi
) for each individual—the one that corresponds to his or her exposure status in real life (e.g.,
The POF allows researchers to identify the ATE under the assumption of ignorability. Ignorability—also known as no omitted variable bias, no selection bias, or exogeneity—implies that the model controls for all confounders in the association between X and Y. In other words, ignorability implies that conditional on a set of pre-exposure confounders, exposure is ignorable, which is similar to saying that it is as good as random, that there are no omitted confounders, or that the potential outcomes between exposed and unexposed individuals would have been the same had exposure not taken place (Angrist & Pischke, 2011). In the context of the POF, this assumption is unsurprising: Given that we only can observe one potential outcome for each person (the one that corresponds to his or her exposure status), the challenge is to infer the missing potential outcome (i.e., the counterfactual scenario) in a convincing way.
In our first hypothetical example, if the ECD intervention was randomized, we would expect the treatment and control groups to be equivalent in all ways except for their exposure to the intervention, mitigating concerns about violating the ignorability assumption. We may have more concerns about ignorability, however, in observational studies where the exposure of interest was self-selected or not randomly assigned. In our second study about poverty, for example, there may be fundamental but unmeasurable differences between the experiences of lower versus higher income children (e.g., community-level factors) that may explain disparities in children’s cognitive development, thus confounding the “effect” of poverty. Developmental researchers have become increasingly interested in designing studies to satisfy the ignorability assumption as a means for establishing credible causal links between exposures and outcomes. Some examples of methods that help to support ignorability are randomized controlled trials and, in observational studies, instrumental variables, difference-in-difference, regression discontinuity, matching methods, and fixed-effects models (Duncan et al., 2004; Foster, 2010; Gennetian et al., 2008; McCartney et al., 2006; Miller et al., 2016).
It is possible to extend the POF to reflect causal mediation effects (Imai, Keele, & Tingley, 2010). In the framework,
As noted earlier, total effects in mediation analysis can be decomposed into direct and indirect effects. Using the language of the POF, the individual causal indirect (or mediation) effect is
Finally, the individual causal direct effect is
Just as in the traditional POF, it is impossible to identify individual-level mediation effects, as only one potential outcome can be observed for each person in a given data set (the one that corresponds to the actual exposure status, e.g.,
Consequently, the POF allows us to formalize a key assumption to identifying causal mediation effects, namely sequential ignorability (Imai, Keele, & Tingley, 2010). Sequential ignorability implies two ignorability assumptions in a row: first, controlling for pre-exposure confounders, exposure is assumed to be ignorable (i.e., as good as random or independent of potential outcomes and potential mediators); second, the mediator is assumed to be ignorable given exposure status and pre-exposure confounders (i.e., mediator is independent of potential outcomes) or, in other words, the model controls for all confounders in the relation between mediator and outcome. (Figure 1, Panel B, presents an observed confounder Z which can be included as a covariate in the model. Panel C presents an unobserved confounder U in the association between mediator and outcome that violates sequential ignorability.)
It is possible to satisfy the first part of the sequential ignorability assumption by randomizing the exposure variable within an experimental design or exploiting quasi-experimental approaches that developmental researchers already use (Foster, 2010; Miller et al., 2016). The second part of sequential ignorability, however, is a very strong assumption in that it requires controlling for all pre-exposure confounders in the relation between mediator and outcome, which likely does not occur even when exposure is randomized (VanderWeele, 2015). In our first example, even if the ECD program was successfully randomized, the second part of the ignorability assumption would be violated if an unobserved confounder such as parental stress or neighborhood crime simultaneously affects both parental stimulation and child development. In the second example, the sequential ignorability assumption is even harder to meet, as selection effects, or endogeneity bias, are a concern at both stages (i.e., poverty to parental stimulation and parental stimulation to child cognition). The use of quasi-experimental approaches (e.g., fixed-effects and regression discontinuity) could mitigate to some extent concerns about nonrandom selection into poverty, but it will likely not reduce concerns about potential confounders in the link between parental stimulation and children’s cognitive development.
Best Practices for Addressing Challenges in Causal Mediation
The general framework presented in this manuscript has important implications for developmental researchers who seek to improve their mediation analysis. In particular, the POF makes visible the key assumption to identify causal mediation effects (i.e., sequential ignorability), making it possible to think about and apply sensitivity checks and invest in study designs that make sequential ignorability more plausible. In this section, we highlight different approaches for mitigating challenges in mediation analysis.
Study Design
As noted above, experimental and quasi-experimental study designs rarely guarantee sequential ignorability (Imai et al., 2011). To further improve causality in mediation analysis, researchers can follow additional steps when designing studies to improve the plausibility of causal mediation claims. First, the use of longitudinal data with rich covariates, including pre-exposure measures of outcomes and mediators, guarantees temporal precedence and allows researchers to control for multiple confounders in the relation between the outcome and mediator, relaxing to some extent the sequential ignorability assumption. Second, a strong theoretical model can allow researchers to identify potential mediators and confounders in the causal chain, informing data collection and model design. Finally, quasi-experimental methods that exploit natural variation or that employ principal stratification or matching methods can aid researchers in checking balance in observed characteristics and reducing, to some extent, threats to the identifying assumption (Jo et al., 2011).
Alternative experimental approaches where the mediator can be directly or indirectly manipulated also offer promise (Imai et al., 2013). One such approach is a parallel design, where each individual is randomly assigned to either (a) random exposure only or (b) both random exposure and random mediator. Another approach is a crossover design, where each individual participates in two sequential experiments: First, exposure is randomly assigned, and the researcher observes the resulting values of the mediator and outcome variables. Then, exposure status is reversed, and the mediator is fixed to the value observed in the first experiment. A main limitation of these designs is that it is often difficult to manipulate mediators perfectly, particularly if mediators relate to psychological processes (Imai et al., 2013).
General Estimation Method for Mediation
Imai, Keele, and Tingley (2010) have proposed a general estimation method that makes more salient the rationale of and assumptions behind the POF, aiming to predict the (not observed) potential mediator and outcome. Moreover, the method can be used in parametric and nonparametric models, thereby removing linearity assumptions common to SEM and OLS methods. Linearity assumptions are not tenable in much developmental research, including cases where researchers use binary outcome variables (e.g., the child is developmentally on track or not). As such, removing such assumption aids causal identification.
The general estimation method comprises five steps. First, the researcher fits two regression models: (a) the mediator is regressed on the exposure variable and pre-exposure confounders, and (b) the outcome is regressed on the mediator, exposure variable, and pre-exposure confounders. These regression models can be linear or nonlinear (e.g., logit or Poisson). Second, using the estimates from the fitted mediator model (Model a), the researcher generates two sets of predicted values for the mediator for each observation, one corresponding to the mediator value under exposure and one for non-exposure conditions. Third, using the coefficients for fitted outcome model (Model b), the researcher predicts potential outcomes corresponding to the exposure condition and using the mediator value for the exposure conditions predicted in Step 2, and potential outcomes corresponding to the non-exposure condition and using its corresponding mediator value. The ACME will be the average difference between the predicted outcome under the exposure and non-exposure conditions. Fourth, the researcher repeats the process N times (typically 5,000–10,000, although there are not clear guidelines on minimum numbers of resamples necessary; Imai, Keele, & Tingley, 2010; Keele et al., 2015) to obtain uncertainty estimates or confidence intervals using bootstrapping. Finally, hypothesis testing is conducted using the uncertainty estimates to assess statistical significance.
Sensitivity Analysis
Given that sequential ignorability is a very strong assumption, it is of cardinal importance to quantify the sensitivity of the results from mediation analyses to potential confounders, or the degree to which identifying assumptions must be violated for a given conclusion to be invalid. Outside of mediation analysis, developmentalists have proposed sensitivity analyses, such as the coefficient of proportionality, to assess the impact that unobserved confounders would need to have to nullify a result (Dearing & Zachrisson, 2019).
Other fields, such as epidemiology, have proposed similar approaches to assess the validity of causal mediation claims (VanderWeele, 2015). Considering the general estimation method proposed by Imai, Keele, and Tingley (2010), the error terms for the outcome and mediator models should be independent (r = 0) if sequential ignorability holds (as these error terms should be random and not explained by an omitted variable). As such, the correlation between both error terms can serve as a sensitivity parameter, where the higher the correlation, the less likely sequential ignorability holds (Keele et al., 2015). Imai, Keele, and Yamamoto (2010) also proposed a sensitivity check similar to the coefficient of proportionality. This sensitivity check assesses the extent to which an omitted variable would change the coefficient of determination (R 2) of the mediation model, by assessing how much the R 2 varies when controlling for observed characteristics that may act as confounders. In this test, relative changes in the R 2 (in models ignoring observed confounders relative to models controlling by these confounders) serve as a sensitivity parameter. Even though we encourage the use of these approaches, they only allow researchers to examine the influence of pre-exposure confounders, not post-exposure confounders that may be influenced by exposure itself (Imai, Keele, & Tingley, 2010). Moreover, there is no established threshold for the sensitivity parameters, which makes their interpretation largely subjective (Keele et al., 2015).
Resources for Developmentalists
Resources for causal mediation analysis have evolved rapidly over the last decade and are increasingly available within multiple statistical software packages. Table 1 provides a brief list of resources for developmentalists interested in using causal mediation methods in their own work, including available statistical functions to apply the general estimation method for mediation, sensitivity analysis, and other estimation methods for causal mediation effects, along with references for detailed descriptions, tutorials, and examples. Developmental researchers interested in learning more about causal mediation approaches more generally can also reference work by Imai et al. (2010), Imai et al. (2011), VanderWeele (2015, 2016), and other references included in this manuscript. We argue that the wide-scale use of these resources can make the application of best practices in causal mediation more common in developmental science.
Sample Resources for Causal Mediation Analysis.
Note. a Other estimation methods are not based on the general estimation method described above (i.e., Imai et al., 2010). bStatistical function/command can be installed with package mediation.
Conclusion
Mediation analysis is fundamental to understanding developmental theories, questions, and processes. However, developmental science has long overlooked important methodological challenges to establishing credible causal mediation effects, in both experimental and observational studies. Methodological advances in other fields offer exciting opportunities for developmentalists to improve causal inference in mediation analysis. Even though the key assumption to identify causal mediation effects is very strong and unsatisfiable even in the context of a randomized experiment, a rich collection of covariates, novel experimental designs, and sensitivity analyses can offer evidence to evaluate the plausibility of causal claims. When broadly applied, the framework for mediation reviewed in this article can allow developmental researchers to explore mediating processes in a rigorous way to better inform both theory and practice.
