Testing Complex Social Theories With Causal Mediation Analysis and G-Computation: Toward a Better Way to Do Causal Structural Equation Modeling

Abstract

Complex social scientific theories are conventionally tested using linear structural equation modeling (SEM). However, the underlying assumptions of linear SEM often prove unrealistic, making the decomposition of direct and indirect effects problematic. Recent advancements in causal mediation analysis can help to address these shortcomings, allowing for causal inference when a new set of identifying assumptions are satisfied. This article reviews how these ideas can be generalized to multiple mediators, with a focus on the posttreatment confounding and causal ordering cases. Using the potential outcome framework as a rigorous tool for causal inference, the application is the theory of procedural justice policing. Analysis of data from two randomized experiments shows that making similar parametric assumptions to SEMs and using G-computation improve the viability of effect decomposition. The article concludes with a discussion of how causal mediation analysis improves upon SEM and the potential limitation of the methods.

Keywords

causal mediation analysis causal ordering structural equation modeling G-computation causal inference police legitimacy potential outcome framework posttreatment confounding procedural justice policing sensitivity analysis

Only when they must choose between competing theories do scientists behave like philosophers.

Thomas S. Kuhn

Introduction

The social sciences are full of relatively complex theories that involve direct and indirect causal pathways. For example, Rivera and Tilcsik’s (2016) survey experiment tested whether higher class signals in résumés, mediated by the applicants’ perceived fit and commitment to the job, influenced whether the respective male or female participant was invited for an interview, using multigroup linear structural equation modeling (SEM) to examine the indirect (mediated) effects. Many social scientific researchers—especially those in sociology, psychology, and criminology—rely on linear SEMs for testing theories of similar complexity, and this technique has several advantages in that, for example, it provides global and (in some cases) comparative model fit estimates and permits simultaneously the fitting of complicated measurement and structural models without aggregation of measurement error (e.g., Kaplan 2008; Tomarken and Waller 2005). Linear SEMs can also be expanded to accommodate data structures of a multilevel and/or longitudinal nature. A further testament to the popularity of linear SEMs is that the two most cited articles in Sociological Methods & Research (Bentler and Chou 1987; Browne and Cudeck 1992) were also written on this very subject.

For the social sciences to accumulate a robust body of knowledge that provides credible policy prescriptions, researchers need to test the causality of their often times relatively convoluted models. But many researchers seem—on the surface at least—to be unaware of the difficulties within such endeavors. Rivera and Tilcsik’s (2016) article is commendable, as it attempted to causally assess their hypotheses, but they did not test the causal identification assumptions of SEM (Bollen and Pearl 2013; Keele 2015b), nor did they draw upon the methodological literature on causal mediation analysis (Keele 2015a; VanderWeele 2015, 2016). As emphasized by Kenny (2008), mediation analysis is a form of causal analysis where disregarding the underlying causal assumptions can lead to misspecified models and thus misleading results.

To address these difficulties, this article examines the methodological challenges within and potential of causal mediation analysis with multiple mediators. Presenting some causal alternatives to linear SEMs, it draws upon papers by De Stavola et al. (2015) and Daniel et al. (2015), both of which use SEM with G-computation to consider statistical issues of posttreatment confounding and causal ordering in causal mediation analysis. Here, these two techniques are discussed with a focus on the interpretation of the results and necessary identification assumptions (for technical details regarding the estimation and modeling, please see the cited papers) and, as a motivating example, the theory of procedural justice policing is used (Jackson et al. 2012; Tyler 2006). This article provides a comprehensive overview of the different approaches available for causal mediation analysis with multiple mediators and goes beyond a recent publication on causal mediation analysis (VanderWeele 2015); both techniques considered here were devised contemporaneously to this book’s publication. The aforementioned two methods improve upon the traditional linear SEM approach in at least two ways: (1) They rely on the potential outcome framework as a rigorous tool to make the causal identification assumptions explicit and devise formal definitions of the direct and indirect effects (IE), and (2) these methods allow for more flexible modeling by loosening some of the parametric requirements, thus providing weaker, and more attainable assumptions, than the ones required for linear SEM.

This article is organized as follows. The first section focuses on a central prediction of procedural justice theory, namely that people’s judgments on the legitimacy of the police mediate the effects of their perceptions of police procedural justice and legality on their willingness to cooperate with the police. Two experiments are outlined: the first manipulated the perceived procedural justice of the police and the second manipulated the perceived legality of the police. The second section discusses how linear SEMs traditionally derive mediated effects and highlights some of the potential pitfalls of this approach. The third section briefly reviews causal mediation analysis with a single mediator. The fourth section discusses the different approaches researchers can take when working with multiple mediators. The fifth and sixth sections discuss two particular instances of complex social scientific theories: mediators with posttreatment confounding and causally ordered mediators, and the findings from the two experiments are presented. The article concludes with a consideration of the findings, an outline of some of the limitations of the methods, and some recommendations for applied researchers.

Procedural Justice Policing and the Legitimacy of the Police

The theory of procedural justice is built on the idea that when people evaluate their interactions with the police, they are primarily focused on whether or not the officer (a) makes objective and neutral decisions and (b) treats them in a fair and respectful manner. When the police act in procedurally just ways, citizens feel that their input is considered, their status in the community is affirmed, and that the police as an institution has legitimate authority (Mazerolle et al. 2013; Tyler 2006; Tyler, Goff, and MacCoun 2015; Tyler and Jackson 2014). In addition to procedural justice, the concept of “bounded authority” has recently been introduced into the police legitimacy framework (Huq, Jackson, and Trinkner 2017; Trinkner, Jackson, and Tyler 2018). Bounded authority captures the idea that people expect authority figures to respect the limits of their rightful authority, for example, that police officers do not act as if they are above the law or do not become involved in situations that they have no right to be in. People divide their lives into domains, and in each of these domains they put a cap on how much interference from the legal authorities they can tolerate, and this boundary condition can shape their judgments on the legitimacy of that authority.

Legitimacy judgments have two constitutive elements: the right to power and the authority to govern (Tyler 2006; Tyler and Jackson 2013). Applied to the police, right to power judgments can be operationalized as institutional trust (the belief that institutional actors can be trusted to wield their power appropriately, where trust constitutes the normative justifiability of power) or as normative alignment (the belief that institutional actors respect key societal norms regarding how they should behave, where normativity constitutes the normative justifiability of power). Authority to govern activates the moral duty to accept the right of the police to make decisions and dictate appropriate behavior, prompting voluntary consent and obedience because of the source rather than the content (Bradford 2014; Tyler and Jackson 2013). Even though the building blocks of legitimacy (normative alignment and duty to obey) are usually agreed upon, their relationship is sometimes debated: Some argue that these two elements mutually reinforce each other (e.g., Hough, Jackson, and Bradford 2013) while others claim that normative alignment is a predictor of duty to obey (e.g., Huq et al. 2017). A good deal of research (Tyler and Fagan 2006; Tyler et al. 2015; White, Mulvey, and Dario 2016) has shown that legitimacy has an impact on certain socially desirable outcomes, and among these outcomes, willingness to cooperate with the police is the focus of this article.

To test procedural justice theory, two experiments manipulate people’s perceptions of procedural justice or the legality of the police through descriptions of fake police encounters. Study 1 (n = 215) and Study 2 (n = 235) were conducted in July 2013 in two subsequent weeks on the Amazon Mechanical Turk Web site with participants from the United States. Both studies used a similar newspaper article about police roadside checks, manipulating either the perceived procedural justice or legality of the police. It is assumed here that procedurally unjust and illegal treatments have a negative impact on how people form their attitudes about police legitimacy, which in turn transmits their impact on willingness to cooperate with the police.

Procedural justice of the police, legality of the police, normative alignment with the police, obligation to obey the police, and willingness to cooperate with the police were measured with 3 items each on a 1–5 Likert-type scale almost exclusively with construct-specific response alternatives. Gender, age, ethnicity, and state of origin were also measured. For further details regarding the experiments and procedure, please refer to Online Appendix A.

SEM and the Traditional Definition of IEs

Throughout this article, SEM refers to the traditional linear models that most researchers use rather than certain recent developments in the field that have yet to become standard (e.g., Liu et al. 2014; Mayer et al. 2017; Sardeshmukh and Vandenberg 2017). SEM conventionally relies upon the product method (Baron and Kenny 1986) to estimate mediated effects. Let us assume that we have a treatment (T) that channels its effect (partially) through a mediator (M) for the outcome (Y). The direct effect is T’s unmediated impact on Y; the mediated effect is the product of the estimates for T’s effect on M and M’s effect on Y. Therefore, the name of the product method refers to how the point estimate of the mediated effect is derived. Crucially, the presence and absence of direct effect and IE is determined by the significance of the effects, although the effect sizes should still be considered even in case of nonsignificant coefficients.

Despite the widespread appeal of this approach, the product method has four limitations. First, SEMs posit effect homogeneity, that is, that every unit in the population has the same causal effect. This is an untestable and unrealistic assumption on the population level, as it must pertain to each individual (Robins and Greenland 1992). Second, the product method can only identify the total effect (TE) as the sum of the direct effect and IE in the absence of a treatment–mediator moderated effect that would influence the outcome (no interactions; Imai, Keele, and Yamamoto 2010; Imai et al. 2011). The presence of such a treatment–mediator interaction would be a clear sign that effect homogeneity is violated (Kline 2015). However, even if the effect homogeneity assumption can be relaxed, it remains unclear where to assign the interaction effect. This leads to the failure of the decomposition and makes the direct effect and IE inextricable (Mackinnon 2008; Mackinnon, Kisbu-sakarya, and Gottschall 2013). A third limitation is that the product method requires the linearity assumption, which should apply not only to the outcome variable but to all variables including the mediator(s; Jo 2008). This linearity assumption guarantees effect constancy, that is, that the effect of one variable on another will be independent of the level of a third variable. Conversely, in nonlinear systems, the chosen level of M would influence the effect of T on Y, thus prohibiting the additivity of effects (Pearl 2014).

The final limitation concerns not the method itself, but its application. Users of SEM often hope to answer causal questions, and one of the key assumptions to guarantee this is that there are no omitted influential variables (i.e., unmeasured confounders). Yet, even if the treatment T is randomly assigned, only the T-Y and T-M relationships are randomized, while the M-Y relationship is not. Rivera and Tilcsik’s (2016) study is instructive here. They assumed that the randomized treatment’s mediated effects can be deemed causal, when in fact, the effects might be influenced by an unmeasured confounder¹ (Judd and Kenny 1981; VanderWeele 2015).

A Brief Review of Causal Mediation Analysis With a Single Mediator

Causal mediation analysis with a single mediator helps to address the limitations mentioned earlier by making the casual identifying² assumptions more explicit. Moreover, it also helps to overcome them by permitting nonparametric identification and incorporating the treatment–mediator interaction while still allowing the decomposition of effects. For the new definitions of direct effect and IE, the potential outcome framework can be used. At the heart of this framework is a thought experiment in which (assuming a binary treatment for the sake of simplicity) a person receives both the treatment and control simultaneously at the same point in time. Naturally, a person can only receive one of these conditions, and we can never observe what would have happened had this person been assigned to the other condition. Nevertheless, provided that the preconceived assumptions are satisfied, the two outcomes are estimable on the population level. The potential outcome framework treats certain counterfactual values as missing, and the only way to address this missingness is to rely on identifying assumptions regarding these unobservable quantities (Keele 2015b; Westreich et al. 2015). If these identifying assumptions are met, they will permit the estimation of population level causal effects.³

For causal mediation analysis, the sequential ignorability assumption⁴ was proposed (Imai, Keele, and Tingley 2010; Imai, Keele, and Yamamoto 2010; Pearl 2001). This states that for a treatment T, a mediator M, and an outcome Y with T = t and M = m and controlling for a vector of pretreatment covariates C, there is:

no unmeasured confounding of the T-Y relationship or Y_tm_╨T|C;

no unmeasured confounding of the M-Y relationship also given T or Y_tm_╨M|C, T;

no unmeasured confounding of the T-M relationship or M_t_╨T|C; and

no unmeasured M-Y confounder L that was affected by T or Y_tm_╨M_t*|C.

As indicated earlier, random assignment of T only satisfies (i) and (iii) from the four assumptions. The first three assumptions are conventional “no unmeasured confounding” assumptions, while the fourth invokes the “cross-world independence” assumption where t and t* stand for two values of the treatment we wish to compare (e.g., in case of a binary treatment t = 1 is the treatment and t* = 0 is the control). Crucially, this cross-world independence assumption also prescribes that there can be only a single mediator affected by the treatment (no posttreatment confounder L).

The sequential ignorability assumption permits the definition of new effects. Overall, the conditional expectations for a particular outcome will take the form of E[Y_t,M( _t* )], where t and t* are set at a freely chosen level of the treatment for Y and M. The controlled direct effect (CDE) only requires assumptions (i) and (ii) and considers a specified value of M = m and captures the expected increase in Y when T changes from T = 0 to T = 1. This is a direct effect, since the effect of T is not transmitted through M. The value of CDE might change depending on the chosen value of m:

CDE (m) = E [Y (1, m) - Y (0, m)] .

Both natural effects require all assumptions (i–iv) to be estimable. The natural direct effect (NDE) is similar to the CDE, as it estimates the expected increase in Y when T changes from T = 0 to T = 1, but it does not hold m constant; instead it permits m to take its value in the “natural” way for each individual as if that individual had been assigned to the control condition:

NDE = E [Y (1, M (0)) - Y (0, M (0))] .

The natural indirect effect (NIE) does the opposite of NDE as it approximates the expected increase in Y when the treatment is kept at T = 1, while M is freed to take its natural value of m for the treatment and the control group, respectively. This is an IE that captures the effect of T on Y that is transmitted through M:

NIE = E [Y (1, M (1)) - Y (1, M (0))] .

Finally, the TE can be decomposed to the sum of the NDE and NIE:

\begin{array}{l} TE = E [Y (1) - Y (0)] = {E [Y (1, M (1)) - Y (1, M (0))]} \\ + {E [Y (1, M (0)) - Y (0, M (0))]} \\ = NIE + NDE. \end{array}

As described above, identification of the direct effect and IE through the potential outcome framework does not posit the no-interaction assumption, which allows for the effect decomposition even in the presence of such an association. Moreover, it is nonparametrically identified, hence it does not require the effect homogeneity or linearity assumptions, either of which permits more flexible modeling. For an example of how these effects are estimated using G-computation, please refer to Online Appendix B.

Causal Mediation Analysis With Multiple Mediators

As with SEMs, causal mediation analysis always starts with a qualitative stage of model building. This stage is inherently theoretical—as alluded to in the quote at the beginning of this article—with the researcher distilling knowledge about prior scholarship, the research design, and potential temporal order to logically structure the theoretical model (Bollen and Pearl 2013). As expressed by (iv), if more than one mediator is present, the sequential ignorability assumption might be violated, threatening the identifiability of the NDE and NIE. Thus, in the presence of multiple mediators, there are four different strategies an analyst can consider: assume causal independence, model the joint effects, assume posttreatment confounding, or assume sequential ordering. The decision regarding the appropriate strategy cannot be data driven, it has to be informed by the researcher’s knowledge regarding the existing literature.

When mediators are causally independent of one another (i.e., parallel or nonintertwined), the same analytical strategy can be pursued as with a single mediator. Notably, this causal independence is an untestable assumption that makes it difficult to assess whether Figure 1a or b is more suitable for the constructs analyzed. Nonetheless, an obvious way of examining the potential dependence between variables is to regress T, C, and L on the M of interest. Significant relationships can provide a reasonable indication that the variables are dependent on each other (Imai & Yamamoto, 2013). However, even if no statistically significant association emerges, it is usually difficult to argue for the orthogonality of the mediators, unless there is a convincing theoretical reason to do so or such orthogonality is artificially created (e.g., through varimax rotation in exploratory factor analysis). A rare example of such independence was provided by Taguri, Featherstone, and Cheng (2018), who examined two unrelated techniques to prevent dental cavities through antibacterial and fluoride therapy mediators. Nevertheless, it is usually difficult to encounter such clear-cut cases in the social sciences. Finally, assessing the mediators one at a time will also fail if there are interactions between the effects of the various mediators on the outcome (Lange, Rasmussen, and Thygesen 2014).

Figure 1.

Mediation analysis (a) with two causally independent mediators, (b) with posttreatment confounding where M is dependent on L, (c) with two sequential mediators M₁ and M₂, (d) with posttreatment confounding where M is dependent on L₁ and L₂, and (e) with three sequential mediators M₁, M₂, and M₃.

Assuming causal dependence between L and M, a simple solution is to examine their joint effect and treat them as a vector of mediators (Steen, Loeys, Moerkerke, and Vansteelandt 2017; VanderWeele and Vansteelandt 2014). Importantly, L and M are statistically equivalent, the posttreatment confounder label of L is only substantive; hence, L can be considered as a second mediator. Handling multiple mediators as a single vector is robust to unmeasured common causes of various mediators and can even accommodate cases with causal ordering of M₁ and M₂ similar to Figure 1c. Crucially, the sequential ignorability assumption (i–iv) is still valid, but now for a vector of mediators instead of a single mediator. Admittedly, this approach limits the scope of the analysis, yet it can be pragmatic for certain research questions (e.g., if we are only interested in the mediated effects of legitimacy as a whole on the outcome of interest). Even so, for many applied cases, this technique will remain untenable. For instance, it would be hard to justify this approach in Rivera and Tilcsik’s (2016) study where the commitment and fit for a job are fairly different aspects. Similarly, this strategy would not allow one to test the unique impact of moral alignment with the police and duty to obey the police. In such circumstances, assuming causal mediation analysis with posttreatment confounders or sequentially ordered mediators need to be considered, which will be discussed in the next two sections.

Causal Mediation Analysis With Posttreatment Confounding

Avin, Shpitser, and Pearl (2005) in their proof showed that conditioning on L does not permit the nonparametric identification of natural effects, but only the CDEs (for which, assumptions [i] and [ii] are sufficient enough). The biggest issue in the presence of causally dependent L is that testing the mediators one at a time will no longer be viable because this results in counting certain causal pathways more than once (VanderWeele and Vansteelandt 2014). A further problem emphasized by some (Daniel, De Stavola, and Cousens 2011) is that the direct effect of T on Y will not estimate consistently if there is an uncontrolled posttreatment confounder L that opens a backdoor path of T → L → Y. A seemingly easy fix to this problem is to condition for L as well. However, since L was affected by T controlling for L in the model for M, this blocks the T → L → Y path, which will also result in biased estimates for the NDE and NIE.⁵ This means that neither the inclusion nor the exclusion of L will solve the problem of identifiability.

Thus, to address these issues, we require a refined—and relaxed—sequential ignorability assumption to make the NIE and NDE identifiable. Crucially, these alternatives do not contradict Avin et al. (2005) but introduce additional assumptions to make the natural effects estimable. While several alternative sets of identifiability criteria have been established (Tchetgen Tchetgen and VanderWeele 2014), the one postulated by De Stavola et al. (2015) will be discussed here. De Stavola et al. (2015) modified the definition of Imai and Yamamoto (2013), positing that sequential ignorability in the presence of posttreatment confounder L holds when controlling for pretreatment covariates C when there is:

no unmeasured confounding of the T-Y, T-M, and T-L relationship or (Y_tml M_tl L_t)_╨T|C;

no unmeasured confounding of the M-Y relationship also controlling for T and L or Y_tml_╨M|C, T, L;

no unmeasured confounding of the L-Y relationship also controlling for T or Y_tl_╨L|C, T; and

no unmeasured M-Y confounder Z that was affected by T or Y_tm_╨M_t*|C, L.

These assumptions are analogous to (i–iv). Assumption (v) is satisfied in the case of a random assignment of T, while (vi) makes the mediated effect conditional on L. Assumption (vii) stresses that there cannot be any unmeasured confounder for the L-Y relationship, which is again a strong assumption similar to (ii). Finally, assumption (viii) establishes that there cannot be any posttreatment confounder Z that was not included in L (i.e., all posttreatment confounders—in other words, alternative mediators—are measured).

Under assumptions (v–vii), the NDE and NIE are identifiable with a few additional limitations. Firstly, the analyst needs to rely on the linearity assumption, which permits the additivity of the effects. Secondly, it needs to be assumed that there is no significant T-M interaction for Y (Robins and Greenland 1992) or that both the T-L interaction and L² are zero in the model for Y (Petersen, Sinisi, and van der Laan 2006). Importantly, this second modeling assumption is a loosened version of SEM’s effect homogeneity assumption that only needs to be true on average, not for each individual, thus it can be empirically assessed (Imai and Yamamoto 2013). Crucially, and as demonstrated by De Stavola et al. (2015), when these parametric assumptions are met, assumption (vii) is automatically satisfied.

Provided that the model is identifiable, a generalized structural equation model needs to be specified with L modeled on C and T; M modeled on C, T, and L, and Y modeled on C, T, L, and M. In addition, the interaction between T and L is entered in the model for both M and Y, and the squared transformation of M and L in the model for Y to control for potential quadratic and heterogeneous effects in line with the earlier identification assumptions.⁶ Then, a generalized version of the product method is used to obtain the parameters. G-computation of the causal estimates for NDE and NIE can be accomplished by combining these appropriate parameters from the SEM through estimation by combination. Unfortunately, with more complex models, this mathematical integration can become exceedingly cumbersome with potential issues of convergence. To overcome this difficulty, Monte Carlo simulation is used as a more flexible and efficient way to approximate the integration, while the standard errors and confidence intervals are bootstrapped (Daniel et al. 2011). As acknowledged by De Stavola et al. (2015), the results of this approach will coincide with a traditional SEM, provided that there are no interactions or nonlinear terms of M, L, or T (for the equations discussed in this paragraph, please refer to Online Appendix C).

Finally, it is crucial to assess the robustness of the M-Y relationship to potential unmeasured confounding. De Stavola et al. (2015) devised a sensitivity analysis based on Imai et al.’s (2011) method that can be applied in the presence of posttreatment confounders. This refined method fits an SEM which allows for the error terms of the models for Y and M to become correlated.⁷ These error terms are very instructive as they incorporate the impact of the unmeasured confounders. This method regresses L on X and C; M on L, X, and C; and Y on X, L, and C. M is not included in the model for Y as to do so would induce collinearity. Then, the error terms from the model for M and Y are systematically correlated, where ρ′ provides an indication of how big the correlation must be between the two error terms to make the M-Y relationship zero. A confidence interval for ρ′ can also be obtained with bootstrapping.

Preliminary Remarks

Confirmatory factor analysis was used to derive factor scores for the respective constructs in both studies. These factor scores were entered in the causal mediation analysis. Although this strategy might have resulted in increased measurement error bias compared to using latent variables to capture multiple indicators (Loeys et al. 2014), the reliance on latent variables, their interactions and transformations (i.e., squared forms) would have added to the computational complexity and prolonged the already fairly long estimation time. Moreover, the concepts and definitions of causal effects only apply to the structural model, not the measurement models. Thus, for pragmatic reasons, factor scores were used instead of latent variables to demonstrate the use of causal mediation analysis with multiple mediators.

All analyses in this article were carried out using STATA Version 14 and its multicore Multi-Processor - STATA/MP version with G-computation models relying on 100,000 Monte Carlo simulations and the number of bootstraps set to 300. The cap on the number of bootstraps was placed so that the analysis would mirror a realistic application, as causal mediation analysis with posttreatment confounding can be particularly time-consuming. With the current specification, it takes five days with a regular office computer and a single core and two days with a cluster computer and six cores to obtain results. The estimation of causal mediation analysis with sequentially ordered mediators is speedier, taking a matter of minutes.

Test of Identification

Two linear regression analyses were fitted for cooperation for the two studies. In both cases, the treatment, the covariates (gender, age, ethnic minority background) and the two potential mediators, their quadratic transformation, and their interaction with the treatment were included (Table 1). As discussed earlier, including these additional parameters in the regression models for the respective Y and examining their significance helps in determining which identification strategy—if any—is relevant for a particular model. However, as noted by VanderWeele (VanderWeele 2015; VanderWeele and Knol 2014), strong statistical power is usually needed to discover interactions; therefore, it is worth also examining effect sizes, especially with smaller sample sizes. In this article, the sample sizes are moderate and the effect sizes are relatively unequivocal, which means that considering the statistically significant effects should be sufficient.

Table 1.

Test of Identification for Posttreatment Confounding, Linear Regression Analyses With 300 Bootstraps.

Explanatory variables	Study 1	Study 2
Moral alignment	0.368*** [0.193, 0.544]	1.143*** [0.703, 1.583]
Moral alignment²	−0.044 [−0.144, 0.056]	0.120 [−0.018, 0.257]
Moral Alignment × Treatment	−0.037 [−0.248, 0.174]	−0.478*** [−0.699, −0.257]
Duty to obey	0.193* [0.014, 0.373]	0.570* [0.087, 1.107]
Duty to obey²	−0.006 [−0.101, 0.089]	0.059 [−0.075, 0.194]
Duty to Obey × Treatment	−0.097 [−0.324, 0.130]	−0.365** [−0.647, −0.083]
Treatment	0.282** [0.0822, 0.483]	1.136*** [1.067, 1.648]
Gender	0.096 [−0.051, 0.243]	0.117 [−0.017, 0.252]
Age (years)	−0.003 [−0.051, 0.243]	−0.006* [−0.012, −0.001]
Ethnic minority	−0.207* [−0.374, −0.040]	−0.111 [−0.300, 0.077]
Constant	0.014 [−0.365, 0.392]	−1.143** [−1.179, −0.497]
N	215	235

For Study 1, either identification strategy (Petersen et al. 2006; Robins and Greenland 1992) should suffice as neither the interactions nor the quadratic terms are significant. However, for the second study, neither will be appropriate, as there are moderately strong and significant interactions between the treatment and both mediators. The NDE and NIE can still be estimated, but they have no meaningful interpretation, thus they are not included in the results table (Table 2). Nevertheless, the TE is always estimable, as well as the CDE(m), provided that assumptions (v) and (vi) are satisfied.

Table 2.

Causal Mediation Analysis With Posttreatment Confounding Using Robins and Greenland’s (1992) Identification Assumption.

Direct, indirect, and total effects	Cooperation	Proportion Mediated (Percent)	ρ′
Study 1 (n = 215)
Moral alignment NIE	0.215*** [0.106, 0.325]	46	0.420 [0.301, 0.545]
Pj versus punj NDE	0.254** [0.106, 0.403]
Pj versus punj CDE(m)	0.255** [0.107, 0.325]
TCE	0.470*** [0.289, 650]
Duty to obey NIE	0.043 [−0.009, 0.095]	9	0.199 [0.047, 0.354]
Pj versus punj NDE	0.423*** [0.255, 0.591]
Pj versus punj CDE(m)	0.423*** [0.256, 0.591]
TCE	0.466*** [0.285, 0.647]
Study 2 (n = 235)
Moral alignment NIE	n.i.
Legal versus illegal NDE	n.i.
Legal versus illegal CDE(m)	1.276*** [0.872, 1.681]
TCE	1.755*** [1.253, 2.256]
Duty to obey NIE	n.i.
Legal versus illegal NDE	n.i.
Legal versus illegal CDE(m)	1.763*** [0.123, 2.296]
TCE	1.852*** [1.314, 2.390]

Note: n.i. = not identifiable; pj = procedural justice; NIE = natural indirect effect; NDE = natural direct effect; CDE = controlled direct effect; punj = procedurally unjust.

*p < .05. **p < .01. ***p < .001.

Results for Causally Dependent Mediators

As discussed earlier, some scholars (e.g., Hough et al. 2013) believe that the two aspects of legitimacy, moral alignment and duty to obey, mutually reinforce each other. In the SEM literature, this is depicted using a bidirectional arrow that denotes a correlation between the constructs. By contrast, the causal inference literature utilizes directed acyclic graphs (DAGs), which do not allow two-headed arrows as to do so would create a cycle. Hence, when mutual reinforcement is hypothesized, two graphs are created for the two different causal directions (Figure 2a and b).

Figure 2.

Mediation analysis where (a) moral alignment has a causal effect on duty to obey or (b) duty to obey has a causal effect on moral alignment.

In causal mediation analysis with posttreatment confounding, the NIE incorporates the mediated effect of the mediator of interest (including L’s impact on M) and the NDE the effect of the treatment not going through M (including L’s impact on Y). In Study 1 (Table 2), procedural justice treatment has a significant positive effect (NDE = 0.254, p < .01) and a significant mediated effect through moral alignment with the police (NIE = 0.215, p < .001), which carries approximately 46 percent of the TE. The sensitivity analysis indicates that on average a relatively strong correlation of ρ′ = 0.42 would be needed between the error terms to nullify the mediated effect with a 95 percent confidence interval of 0.301 and 0.545. Conversely, duty to obey the police has a weak nonsignificant impact on cooperation (NIE = 0.043, p > .05) and transmits only 9 percent of the TE. The sensitivity analysis implies that on average a ρ′ = 0.199 between the error terms could make the impact nonsignificant, but the confidence intervals show that a correlation of 0.047 might be enough to make the effect zero. Procedural justice has a strong and significant direct effect on willingness to cooperate (NDE = 0.423, p < .001). The estimates of CDE(m) and NDE are both within rounding error of each other in both cases, which is unsurprising given the absence of a treatment–mediator interaction, in which case they should approximately coincide (i.e., as a default, the CDE’s m is always set at the average value of the mediator, as this option allows the comparison to the NIE). Overall, it seems that moral alignment with the police has a fairly strong causally mediated effect on cooperation with the police, while duty to obey does not seem to have an impact. Receiving the procedural justice treatment also significantly increased the participants’ willingness to cooperate with the police.

For Study 2, only the CDE(m) and Total Causal Effect (TCE) were identifiable. For both moral alignment (CDE[m] = 1.276, p < .001) and duty to obey (CDE[m] = 1.763, p < .001) as mediators, the CDE of legality was significant. Unfortunately, CDE(m) does not permit effect decomposition, thus no information can be gained regarding the mediated effects. Still, the CDE(m) was much higher than in Study 2, which implies that legal police practices might have an even stronger effect when the police are otherwise thought to overstep legal boundaries.

Causal Mediation Analysis With Sequentially Ordered Mediators

Interdependence between mediators can take the form of a causal chain where the first mediator affects the second mediator and the outcome, but not the other way around. Crucially, this situation’s DAG takes the very same form as the posttreatment confounder case’s as shown in Figure 1b and c where the difference between the two graphs is only substantive. The distinction between the two approaches becomes clearer looking at Figure 1d and e, which show that in the posttreatment confounder cases L₁ and L₂ do not affect one another, while in the sequential cases M₁ and M₂ do, following a predetermined order. This difference in the causal structure leads to an alternative four-way decomposition in case of two mediators where there will be NIE₁ standing for M₁’s mediated effect on Y (T → M₁ → Y), NIE₂ for M₂’s mediated effect on Y (T → M₂ → Y), NIE₁₂ for M₁’s and M₂’s jointly mediated effect on Y (T → M₁ → M₂ → Y), and NDE, T’s effect that does not go through either of the mediators (T → Y). Although there have been other approaches addressing causally ordered mediators (Steen, Loeys, Moerkerke, and Steen 2017; Steen, Loeys, Moerkerke, and Vansteelandt 2017), Daniel et al. (2015) has been the only paper so far to allow for this finest four-way decomposition. As before, this new decomposition will require a modified set of sequential ignorability assumptions; controlling for pretreatment covariates C, these are:

no unmeasured confounding of the T-Y, T-M₁, and T-M₂ relationship or (Y_tm_1m2 M_2tm1 M_1t)_╨T|C;

no unmeasured confounding of the M₁–Y relationship also controlling for T or Y_tm_1m2╨M₁|C, T;

no unmeasured confounding of the M₂–Y relationship also controlling for T and M₁ or Y_tm_1m2╨M₂|C, T, M₁; and

no unmeasured M₁–Y, M₁–M₂, or M₂–Y confounder L₁ or L₂ that was affected by T or Y_tm_1m2╨M_1t*|C, M_2tm_1╨M_1t*|C, and Y_tm_1m2╨M_2t**|C, M_1t*.

When T is randomly assigned, (ix) will automatically be satisfied. Assumption (x) is analogous to (ii) and (xi) to (vi), while (xii) states again that there cannot be posttreatment confounders L₁ or L₂ that were affected by the treatment. As with the posttreatment confounder case, certain parametric restrictions are also needed. As earlier, the linearity assumption is required so the additivity of the effects is guaranteed. Furthermore, when M₁ has a nonzero effect on M₂ the conditional correlation between M₁’s potential outcomes is required to make the effects estimable. However, this conditional correlation is unknown for several of the effects,⁸ hence a sensitivity parameter κ² is used, which stands for the proportion of residual variance shared across the two hypothetical worlds. κ² can take values from 0 to 1, where 0 means no correlation between the potential outcomes conditional on C, and 1 means perfect correlation between the potential outcomes conditional on C (Daniel et al. 2015).

Because of the second mediator, the conditional expectations take more complex forms: Generally, they are E[Y(t,M₁(t*),M₂(t**,M₁(t***)))], where the different t-s (i.e., t, t*, t**, t***) stand for setting the treatment to one of its possible values. This increased complexity also means that the number of possible decompositions of the TE will be (2 ⁿ )!, where n stands for the number of mediators. In the case of two mediators, the 24 (i.e., (2²)! = (4)! = (4 × 3 × 2 × 1) = 24) possible decompositions are reduced to 6 when M₁ does not affect M₂. Overall, marked differences among the path-specific effects only emerge when there are significant T-M₁ and T-M₂ interactions, which are allowed with the current technique. In the absence of interactions, the estimates will be approximately the same as SEM’s estimates, albeit with wider confidence intervals (Daniel et al. 2015).

Because interpreting a high number of estimates from the different decompositions can be cumbersome, and usually not of particular interest, it is worth considering ways to summarize the effects. Based on earlier work (Kuha and Goldthorpe 2010), Daniel et al. (2015) recommended the usage of summary effects that are weighted averages of the NDE and various NIEs. In addition, they advised reporting the variance estimates for these summary effects, which indicate whether there are large differences across the various decompositions. The major advantage of these summary effects is that they provide a good approximation of the respective effects; however, it is hard to attach a substantive interpretation, which can prove problematic especially if the particular effect size on the outcome variable were directly interpretable and of particular interest.

Results for Sequentially Ordered Mediators

In the procedural justice literature, other scholars (e.g., Huq et al. 2017) have argued for the theoretical model depicted by Figure 2b, where duty to obey the police is influenced by moral alignment with the police, but not the other way around. The results (Tables 3 and 4) are presented conditionally only on the two extremes of the sensitivity value (κ² = 0 and κ² = 1), as they appear to be mostly robust to these extremes. In the case of wider disparities, it is advisable to look at further values of the sensitivity parameter.

Table 3.

Causal Mediation Analysis With Sequentially Ordered Mediators (Study 1).

Study 1 (n = 215)	Cooperation	Cooperation	Proportion Mediated
Sensitivity parameter (κ²)	= 0	= 1	= 0–1
Moral alignment SNIE₁	.231** [.059, .392]	.219** [.063, .374]	42–46 Percent
Moral alignment IE_1nointer	.225*** [.103, .347]	.225*** [.116, .333]	45 Percent
Moral alignment ${\sqrt{v a r NIE}}_{1}$	.006 [−.020, .032]	.001 [−.007, .009]
Duty to obey SNIE₂	−.010 [−.052, .032]	−.009 [−.052, .033]	2 Percent
Duty to obey IE_2nointer	−.017 [−.050, .017]	−.017 [−.053, .020]	3 Percent
Duty to obey ${\sqrt{v a r NIE}}_{2}$	∼.001 [−.004, .004]	∼.001 [−.005, .005]
Joint SNIE_1,2	.061* [.002, .12]	.038 [−.017, .094]	7–12 Percent
Joint IE_1,2nointer	.064* [.002, .126]	.064* [.007, .121]	12 Percent
Joint ${\sqrt{v a r NIE}}_{1, 2}$	.006 [−.017, .029]	.001 [−.006, .007]
Pj versus nopj SNDE	.218 [−.077, .513]	.273 [−.022, .569]
Pj versus nopj $\sqrt{v a r NDE}$	.007 [−.011, .024]	.002 [−.015, .018]
TCE	.499** [.166, .834]	.521** [.180, .863]

Note: pj = procedural justice; nopj = procedural injustice; IE = indirect effects; NIE = natural indirect effect; NDE = natural direct effect.

*p < .05. **p < .01. ***p < .001.

Table 4.

Causal Mediation Analysis With Sequentially Ordered Mediators (Study 2).

Study 2 (n = 235)	Cooperation	Cooperation	Proportion Mediated
Sensitivity parameter (κ²)	= 0	= 1	= 0–1
Moral alignment SNIE₁	0.623*** [0.321, 0.926]	0.640*** [0.358, 0.921]	32 Percent
Moral alignment IE_1nointer	0.424*** [0.240, 0.609]	0.441*** [0.277, 0.605]	27 Percent
Moral alignment ${\sqrt{v a r NIE}}_{1}$	0.0.22 [−0.032, 0.077]	0.023 [−0.010, 0.057]
Duty to obey SNIE₂	0.081 [−0.045, 0.206]	0.081 [−0.032, 0.195]	5 Percent
Duty to obey IE_2nointer	0.023 [−0.016, 0.062]	0.023 [−0.015, 0.061]	1 Percent
Duty to obey ${\sqrt{v a r NIE}}_{2}$	0.001 [−0.005, 0.007]	0.001 [−0.003, 0.007]
Joint SNIE_1,2	0.178 [−0.015, 0.370]	0.184* [0.033, 0.336]	9 Percent
Joint IE_1,2nointer	0.060 [−0.036, 0.123]	0.062* [0.001, 0.124]	4 Percent
Joint ${\sqrt{v a r NIE}}_{1, 2}$	0.006 [−0.036, 0.047]	0.005 [−0.005, 0.015]
Pj versus illegal SNDE	1.085*** [0.749, 1.421]	1.085*** [0.737, 1.432]
Pj versus illegal $\sqrt{v a r NDE}$	0.039 [−0.007, 0.085]	0.041 [−0.001, 0.083]
TCE	1.967*** [1.364, 2.569]	1.990*** [1.424, 2.556]

Note: pj = procedural justice; IE = indirect effects; NIE = natural indirect effect; NDE = natural direct effect.

*p < .05. **p < .01. ***p < .001.

The results from Study 1 (Table 3, Online Appendix Figures 3–6) show that moral alignment with the police has a moderately strong mediated effect ( ${SNIE}_{1}^{κ^{2 = 0}} = 0.231$ , p < .01; ${SNIE}_{1}^{κ^{2 = 1}} = 0.219$ , p < .01) on willingness to cooperate with the police with 42–46 percent of the effect transmitted by it. In contrast, duty to obey does not have an impact on any conventional level of statistical significance ( ${SNIE}_{2}^{κ^{2 = 0}} = - 0.01$ , p > .05; ${SNIE}_{2}^{κ^{2 = 1}} = - 0.009$ , p > .05). The joint effect of moral alignment with and duty to obey the police has a weak significant relationship with willingness to cooperate when κ² = 0 ( ${SNIE}_{1, 2}^{κ^{2 = 0}} = 0.061$ , p < .05), but it does not reach the 5 percent significance level when κ² = 1 ( ${SNIE}_{1, 2}^{κ^{2 = 1}} = 0.038$ , p > .05). Procedural justice treatment has a moderately strong effect on cooperation, but it does not reach statistical significance ( ${SNDE}^{κ^{2} = 0} = 0.218$ , p > .05; ${SNDE}^{κ^{2} = 1} = 0.273$ , p > .05). Juxtaposing the results of the two extremes of the sensitivity parameter shows that when the counterfactual outcomes of M₁ are assumed to be perfectly correlated, it slightly boosts the SNDE and widens its confidence intervals but at the same time reduces the mediated effects and narrows their confidence intervals. The variance estimates are tiny across the different effects, which is in accordance with the lack of interactions found earlier (Table 1). The absence of interactions also means that the IEs from an SEM should be very close to the SNIEs which, as expected, they are.

The findings from Study 1 seem to uphold moral alignment’s moderately strong mediated effect on willingness to cooperate while also confirming the lack of impact from duty to obey. The joint effect of the two mediators is either weak or nonexistent while the direct effect of procedural justice does not reach statistical significance; a bigger sample size would be needed to elucidate the treatment’s and the joint effect’s impact on the outcome.

The results from Study 2 (Table 4, Online Appendix Figures 7–10) show a similar pattern to Study 1. Moral alignment with the police has a strong mediated effect ( ${SNIE}_{1}^{κ^{2} = 0} = 0.623$ , p < .001; ${SNIE}_{1}^{κ^{2} = 1} = 0.640$ , p < .001) with 32 percent proportion mediated while duty to obey has a weak nonsignificant one ( ${SNIE}_{2}^{κ^{2} = 0} = 0.081$ , p > .05; ${SNIE}_{2}^{κ^{2} = 1} = 0.081$ , p > .05). Yet again the joint mediated effect is either significant ( ${SNIE}_{1, 2}^{κ^{2} = 1} = 0.184$ , p < .05) or not ( ${SNIE}_{1, 2}^{κ^{2} = 0} = 0.178$ , p > .05), depending on the value taken by the sensitivity parameter. The direct effect of legality is much stronger in Study 2 than procedural justice’s in Study 1 with a statistically significant impact ( ${SNDE}^{κ^{2} = 0} = 1.085$ , p < .001; ${SNDE}^{κ^{2} = 1} = 1.085$ , p < .001). The results are even less sensitive to changes in κ² than in Study 1, and the perfect correlation between the potential outcomes of M₁ here increases the effect sizes while decreasing the confidence intervals for all effects. The variance estimates are much higher than in Study 1, which is expected because of the interaction effects (Table 1). This also means that IEs conventionally obtained from an SEM will be different: In this case, they are much smaller than the ones from causal mediation analysis.

Overall, the results provide further support for the earlier findings. From the two components of legitimacy shared values (moral alignment) appears to be the important mediator of legality’s impact on cooperation while consent (duty to obey) does not seem to matter much. The joint effect of these two elements is very close to zero and requires further scrutiny. The strong direct effect indicates that procedurally just and legal messaging will have a powerful impact, especially when compared to the assumption that the police routinely overstep their boundaries.

In summary, both the posttreatment confounder and sequentially ordered approach concur that moral alignment is the primary conduit of the effect of procedural justice and legality on willingness to cooperate, while duty to obey either does not have an effect or has only a weak joint one with moral alignment. The direct effect of the two treatment conditions also seemed to be important, even if not consistently significant between the two methods. These results are in line with earlier research (e.g., Moravcová 2016; Tyler and Jackson 2014), which found a small or even nonsignificant relationship between duty to obey and cooperation, and which thus called into question its relevance. As with other experiments, the external validity of the results is limited and further studies are needed to attest to the effects found here.

Discussion

Over the last couple of decades, many social science disciplines have relied primarily on SEM (and path analytical models more generally) for assessing complex theories. Yet, adopting the potential outcome framework provides at least three advantages (Daniel, Stavola, and Vansteelandt 2016; Greenland 2017; Steen et al. 2017):

It makes explicit the identification assumptions needed to avoid model misspecification for the mediator(s).

It provides formal definitions of the estimated causal effects.

It devises ways to check for the robustness of the results through sensitivity analysis of certain causal identification assumptions.⁹

This article has argued that the traditional SEM framework has shortcomings that need to be addressed for more realistic identification and effect decomposition. In order to accommodate multiple mediated effects, parametric restrictions akin to SEM need to be made: linearity and relaxed effect homogeneity for the posttreatment confounder, and linearity for the causally ordered case (for a summary, see Table 5). The similarity between the two approaches does not end there; traditional SEM can be considered a special case of causal mediation analysis when certain conditions apply. This means that SEM and causal mediation analysis can be easily reconciled, and that the estimation method will be very similar to each other, which makes such techniques easily understandable and adaptable for those who were primarily trained for SEM (Daniel et al. 2015; De Stavola et al. 2015).

Table 5.

Summary of the Causal and Parametric Assumptions of the Causal Mediation Analysis Techniques Discussed in the Article.

Mediation Analysis Technique	Causal and Parametric Assumptions
Single mediator Causal independence (Figure 1a) Single vector of mediators (assumptions for a vector of mediators)	No unmeasured confounding of the T-Y relationship or Y_tm_╨T\|C No unmeasured confounding of the M-Y relationship also given T or Y_tm_╨M\|C, T No unmeasured confounding of the T-M relationship or M_t_╨T\|C No unmeasured M-Y confounder L that was affected by T or Y_tm╨ $M_{t *} \| C$ Nonparametrically identifiable
Posttreatment confounding (Figure 1b and d)	No unmeasured confounding of the T-Y, T-M, and T-L relationship or (Y_tml M_tl L_t)_╨T\|C No unmeasured confounding of the M-Y relationship also controlling for T and L or Y_tml_╨M\|C, T, L No unmeasured confounding of the L-Y relationship also controlling for T or Y_tl_╨L\|C, T No unmeasured M-Y confounder Z that was affected by T or Y_tm_╨ $M_{t *} \| C$ , L Linearity and some kind of no interaction
Sequential order (Figure 1c and e)	No unmeasured confounding of the T-Y, T-M₁, and T-M₂ relationship or (Y_tm_1m2 M₂ _tm₁ M₁ _t)_╨T\|C No unmeasured confounding of the M₁–Y relationship also controlling for T or Y_tm_1m2╨M₁\|C, T No unmeasured confounding of the M₂–Y relationship also controlling for T and M₁ or Y_tm_1m2╨M₂\|C, T, M₁ No unmeasured M₁–Y, M₁–M₂, or M₂–Y confounder L1 or L2 that was affected by T or Y_tm_1m2╨ $M_{1 t } \| C$ , M₂ _tm* _1╨ $M_{1 t } \| C$ , and Y_tm_1m2╨ $M_{2 t } \| C$ , $M_{1 t }$ Linearity and influence of sensitivity parameter κ² (when M₁ affects M₂)

Studies 1 and 2 exemplify how SEM compares to causal mediation analysis with multiple mediators. The results from Study 1 were approximately identical to the results one would have derived using SEM. In contrast, for Study 2, the posttreatment confounder case was not identifiable, while the sequentially ordered mediator case differed decidedly from the SEM results. Study 1 highlights how traditional SEM can sometimes hit the mark, while Study 2 illustrates that it can also fail. The sensitivity analysis from the different studies can help to determine whether the results are robust to certain conditions (unmeasured confounding or the correlation of certain potential outcomes). The reliance on these sensitivity measures can mitigate bad practices, like “p-hacking” and can help to identify spurious relationships and statistical flukes. This perspective also encourages researchers to adapt a priori model building since their decision will have a major impact on the modeling strategy employed and because the causal structure can never be decided by relying on statistical methods.

Nevertheless, there are certain limitations worthy of discussion. First, causal mediation analysis relies on very strong assumptions. Even in the case of a randomly assigned treatment, the M-Y relationship can be spurious unless a proper set of covariates is controlled for. In Studies 1 and 2, only three covariates (age, gender, and ethnic minority background) were considered, which are far from being sufficient (Steiner et al. 2010). The problem of no unmeasured confounding is further aggravated in observational studies where the treatment is not randomized. Some scholars have recommended conducting “comprehensive SEM” (Mackinnon and Pirlott 2015) with up to 50 covariates, yet even in such cases, it can be difficult to realistically argue for causal inference. Multiple mediators can even exacerbate this issue as it is more likely that at least one of them is affected by unmeasured confounding (VanderWeele 2015).

As an alternative to natural effects, some have recommended the use of interventional effects (Vansteelandt and Daniel 2017), which require weaker causal identifying assumptions. However, these loosened assumptions posit additional parametric restrictions to the ones that have been discussed in this article (e.g., fixing the mediator distribution). Arguably, these alternatives can sometimes be more policy relevant (i.e., the interventional IEs are set at the levels of the potential interventions), but they provide less information regarding the causal mechanisms and hence are often times less generalizable to other contexts.

Others are also critical of SEMs because of their restrictive parametric assumptions, which were (partially) adopted for causal mediation analysis in the current applications. VanderWeele (2012, 2015, 2016) has repeatedly insisted that these modeling assumptions are too strong, and SEMs and similar methods should only be used for hypothesis generation not hypothesis testing. In addition, both Keele (2015a) and Kennedy (2015) have argued that because causal effects are nonparametrically identified, parametric models are more likely to yield misspecification and the use of semi- or nonparametric models is more advisable. Although such alternatives are available for multiple mediators (Kim, Daniels, and Hogan 2017; Moerkerke, Loeys, and Vansteelandt 2015; Tchetgen Tchetgen and VanderWeele 2014), they have other restrictions and limitations (e.g., particular types of outcome, constrained effect decomposition, Bayesian model specification) that make them unappealing or hard to implement.

Even if these criticisms are valid, most of the propositions made in this article touch upon the fundamental limitations of SEM and can be considered as improvements upon it. For instance, the no-interaction assumption is a noncausal issue, yet applying a causal mediation perspective helps to address the matter. Similarly, the current methods allow to incorporate quadratic terms in the model, provide an alternative way to investigate cases when mediators are assumed to mutually reinforce each other, and propose sensitivity analysis for model assessment. In the end, causal mediation analysis provides a list to consider for causal analysis, a slightly modified estimation approach that allows for a more versatile model analysis and assessment, and provides a comprehensive improvement upon the traditional SEM.

Supplemental Material

Supplemental Material, online_Suppl_material_SMR_826159 - Testing Complex Social Theories With Causal Mediation Analysis and G-Computation: Toward a Better Way to Do Causal Structural Equation Modeling

Supplemental Material, online_Suppl_material_SMR_826159 for Testing Complex Social Theories With Causal Mediation Analysis and G-Computation: Toward a Better Way to Do Causal Structural Equation Modeling by Krisztián Pósch in Sociological Methods & Research

Footnotes

Author’s Note

Krisztián Pósch has moved to Postdoctoral Research Fellow at the Department of Methodology.

Acknowledgments

I am immensely grateful for comments made on earlier drafts of this article by Jonathan Jackson, Luke Keele, Jouni Kuha, Greg Ridgeway, Jason Roy, Dylan Small, and the attendees of the Causal Inference Reading Group at the University of Pennsylvania. I am also indebted to Rhian Daniel, Bianca DeStavola, and Stijn Vansteelandt not only for the two excellent articles that provided the inspiration for and basis of this article but also for sharing part of the STATA code for analysis.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Funding was provided by the Santander Fund for the author’s visit to the University of Pennsylvania.

Supplemental Material

Supplemental material for this article is available online.

Notes

References

Avin

Shpitser

Pearl

. 2005. “Identifiability of Path-specific Effects.” Proceedings of the International Joint Conferences on Artificial Intelligence 34:163–192.

Baron

R. M.

Kenny

D. A.

. 1986. “Moderator-Mediator Variable Distinction in Social Psychological Research: Conceptual, Strategic, and Statistical Considerations.” Journal of Personality and Social Psychology 51(6):173–82.

Bentler

P. M.

Chou

C. P.

. 1987. “Practical Issues in Structural Modeling.” Sociological Methods & Research 16(1):78–117.

Bollen

K. A.

Pearl

. 2013. “Eight Myths about Causality and Structural Equation Models.” Pp. 301–28 in Handbook of Causal Analysis for Social Research, edited by Morgan

S. L.

. New York, NL: Springer.

Bradford

2014. “Policing and Social Identity: Procedural Justice, Inclusion and Cooperation between Police and Public.” Policing and Society 24(1):22–43.

Browne

M. W.

Cudeck

. 1992. “Alternative Ways of Assessing Model Fit.” Sociological Methods & Research 21(2):230–58.

Coffman

D. L.

Zhong

. 2012. “Assessing Mediation Using Marginal Structural Models in the Presence of Confounding and Moderation.” Psychological Methods 17(4):642–64.

Daniel

R. M

De Stavola

B. L.

Cousens

S. N.

. 2011. “gformula—Estimating Causal Effects in the Presence of Time-varying Confounding or Mediation Using the g-computation Formula.” The Stata Journal 11(4):479–517.

Daniel

R. M

De Stavola

B. L.

Cousens

S. N.

Vansteelandt

. 2015. “Causal Mediation Analysis with Multiple Mediators.” Biometrics 71(1):1–14.

10.

Daniel

R. M.

De Stavola

B. L.

Vansteelandt

. 2016. “The Formal Approach to Quantitative Causal Inference in Epidemiology: Misguided or Misrepresented?” International Journal of Epidemiology 45(6):1817–29.

11.

De Stavola

B. L.

Daniel

R. M.

Ploubidis

G. B.

Micali

. 2015. “Mediation Analysis with Intermediate Confounding: Structural Equation Modeling Viewed through the Causal Inference Lens.” American Journal of Epidemiology 181(1):64–80.

12.

Greenland

2017. “For and Against Methodologies: Some Perspectives on Recent Causal and Statistical Inference Debates.” European Journal of Epidemiology 32(1):1–18.

13.

Hough

Jackson

Bradford

. 2013. “Legitimacy, Trust and Compliance: An Empirical Test of Procedural Justice Theory Using the European Social Survey.” Pp. 326–53 in Legitimacy and Criminal Justice—An International Exploration, edited by Tankebe

Liebling

. Oxford, England: Oxford University Press.

14.

Huq

A. Z.

Jackson

Trinkner

R. J.

. 2017. “Legitimating Practices: Revisiting the Predicates of Police Legitimacy.” British Journal of Criminology 57(5):1101–22.

15.

Imai

Keele

. 2014. “Comment on Pearl: Practical implications of theoretical results for causal mediation analysis”: Correction to Imai et al. 2014. Psychological Methods 19(4):482–487. https://doi.org/10.1037/met0000035.

16.

Imai

Keele

Tingley

. 2010. “A General Approach to Causal Mediation Analysis.” Psychological Methods 15(4):309–34.

17.

Imai

Keele

Tingley

Yamamoto

. 2011. “Unpacking the Black Box of Causality: Learning about Causal Mechanisms from Experimental and Observational Studies.” American Political Science Review 105(4):765–89.

18.

Imai

Keele

Yamamoto

. 2010. “Identification, Inference and Sensitivity Analysis for Causal Mediation Effects.” Statistical Science 25(1):51–71.

19.

Imai

Tingley

Yamamoto

. 2013. “Experimental Designs for Identifying Causal Mechanisms.” Journal of the Royal Statistical Society: Series A (Statistics in Society) 176(1):5–32.

20.

Imai

Yamamoto

. 2013. “Identification and Sensitivity Analysis for Multiple Causal Mechanisms: Revisiting Evidence from Framing Experiments.” Political Analysis 21(2):141–71.

21.

Jackson

Bradford

Hough

Myhill

Quinton

Tyler

T. R.

. 2012. “Why Do People Comply with the Law?” British Journal of Criminology 52(6):1051–71.

22.

2008. “Causal Inference in Randomized Experiments with Mediational Processes.” Psychological Methods 13(4):314–36.

23.

Judd

C. M.

Kenny

D. A.

. 1981. Estimating the Effects of Social Interventions. Cambridge, MA: Cambridge University Press.

24.

Kaplan

2008. Structural Equation Modeling—Foundations and Extensions. 2nd ed. Thousand Oaks, CA: Sage.

25.

Keele

2015a. “The Statistics of Causal Inference: A View from Political Methodology.” Political Analysis 23(3):313–35.

26.

Keele

2015b. “Causal Mediation Analysis Warning! Assumptions Ahead.” American Journal of Evaluation 46(4):500–513.

27.

Kennedy

E. H.

2015. “Semiparametric Theory and Empirical Processes in Causal Inference.” Pp. 141–167 in Statistical Causal Inferences and Their Applications in Public Health Research, edited by He

Chen

D. -G.

. Springer.

28.

Kenny

D. A.

2008. “Reflections on Mediation.” Organizational Research Methods 11(2):353–58.

29.

Kim

Daniels

M. J.

Hogan

J. W.

. 2017. “Bayesian Methods for Multiple Mediators: Relating Principal Stratification and Causal Mediation in the Analysis of Power Plant Emission Controls.” 1-36. https://dataverse.harvard.edu/dataverse.xhtml?alias=mmediators (accessed January 23, 2019).

30.

Kline

R. B.

2015. “The Mediation Myth.” Basic and Applied Social Psychology 37(4):202–13.

31.

Kuha

Goldthorpe

J. H.

. 2010. “Path Analysis for Discrete Variables: The Role of Education in Social Mobility.” Journal of the Royal Statistical Society Series a-Statistics in Society 173:351–69.

32.

Lange

Rasmussen

Thygesen

L. C.

. 2014. “Assessing Natural Direct and Indirect Effects through Multiple Pathways.” American Journal of Epidemiology 179(4):513–18.

33.

Liu

Chen

Song

. 2014. “Transformation Structural Equation Models with Highly Nonnormal and Incomplete Data.” Structural Equation Modeling: A Multidisciplinary Journal 5511:1–15.

34.

Loeys

Moerkerke

Raes

Rosseel

Vansteelandt

. 2014. “Estimation of Controlled Direct Effects in the Presence of Exposure-induced Confounding and Latent Variables.” Structural Equation Modeling 21(3):396–407.

35.

Mackinnon

D. P.

2008. Introduction to Statistical Mediation. Hoboken, NJ: Erlbaum.

36.

Mackinnon

D. P.

Kisbu-sakarya

Gottschall

A. C.

. 2013. “Developments in Mediation Analysis Oxford Handbooks Online Developments in Mediation Analysis.” Pp. 1–28 in Oxford Handbook of Quantitative Methods, edited by Little

T. D.

(Vol. 2). New York, NY: Oxford University Press.

37.

Mackinnon

D. P.

Pirlott

A. G.

. 2015. “Statistical Approaches for Enhancing Causal Interpretation of the M to Y Relation in Mediation Analysis.” Personality and Social Psychology Review 19(1):30–43.

38.

Manski

C. F.

2007. Identification for Prediction and Decision. Cambridge, MA: Harvard University Press.

39.

Mauro

1990. “Understanding L.O.V.E. (Left Out Variable Error): A Method for Estimating the Effects of Omitted Variables.” Psychological Bulletin 108(2):314–29.

40.

Mayer

Umbach

Flunger

Kelava

. 2017. “Effect Analysis Using Nonlinear Structural Equation Mixture Modeling.” Structural Equation Modeling: A Multidisciplinary Journal 24(4):556–70.

41.

Mazerolle

Antrobus

Bennett

Tyler

T. R.

. 2013. “Shaping Citizen Perceptions of Police Legitimacy: A Randomized Field Trial of Procedural Justice.” Criminology 51(1):33–63.

42.

Moerkerke

Loeys

Vansteelandt

. 2015. “Structural Equation Modeling versus Marginal Structural Modeling for Assessing Mediation in the Presence of Posttreatment Confounding.” Psychological Methods 20(2):204–20.

43.

Moravcová

2016. “Willingness to Cooperate with the Police in Four Central European Countries.” European Journal on Criminal Policy and Research 22(1):171–187. https://doi.org/10.1007/s10610-015-9271-0.

44.

Pearl

2001. “Direct and Indirect Effects.” Proceedings of the Seventeenth Conference on Uncertainty in Artificial Intelligence UAI’01 (1992), San Francisco, 411–20.

45.

Pearl

2010. “On the Consistency Rule in Causal Inference: Axiom, Definition, Assumption, or Theorem?” Epidemiology 21(6):872–75.

46.

Pearl

2014. “Interpretation and Identification of Causal Mediation.” Psychological Methods 19(4):459–81.

47.

Pek

MacCallum

R. C.

. 2011. “Sensitivity Analysis in Structural Equation Models: Cases and Their Influence.” Multivariate Behavioral Research 46:202–28.

48.

Petersen

M. L.

Sinisi

S. E.

van der Laan

M. J.

. 2006. “Estimation of Direct Causal Effects.” Epidemiology 17(3):276–84.

49.

Pirlott

A. G.

Mackinnon

D. P.

. 2016. “Design Approaches to Experimental Mediation.” Journal of Experimental Social Psychology 66:29–38.

50.

Preacher

K. J.

2015. “Advances in Mediation Analysis: A Survey and Synthesis of New Developments.” Annual Review of Psychology 66:825–52.

51.

Rivera

L. A.

Tilcsik

. 2016. “Class Advantage, Commitment Penalty: The Gendered Effect of Social Class Signals in an Elite Labor Market.” American Sociological Review 81(6):1097–1131. https://doi.org/10.1177/0003122416668154.

52.

Robins

J. M.

Greenland

. 1992. “Identifiability and Exchangeability for Direct and Indirect Effects.” Epidemiology 3(2):143–55.

53.

Sardeshmukh

S. R.

Vandenberg

R. J.

. 2017. “Integrating Moderation and Mediation: A Structural Equation Modeling Approach.” Organizational Research Methods 20(4):721–745. https://doi.org/10.1177/1094428115621609.

54.

Shadish

W. R.

Cook

T. D.

Campbell

D. T.

. 2002. Experimental and Quasi-experimental Design for Generalized Causal Inference. Boston, MA: Houghton Mifflin Company.

55.

Steen

Loeys

Moerkerke

Steen

. 2017. “Flexible Mediation Analysis with Multiple Mediators.” American Journal of Epidemiology 186(2):184–193.

56.

Steen

Loeys

Moerkerke

Vansteelandt

. 2017. “Medflex: An R Package for Flexible Mediation Analysis Using Natural Effect Models.” Journal of Statistical Software 76(11):1–45.

57.

Steiner

P. M.

Cook

T. D.

Shadish

W. R.

Clark

M. H.

. 2010. “The Importance of Covariate Selection in Controlling for Selection Bias in Observational Studies.” Psychological Methods 15(3):250–67.

58.

Taguri

Featherstone

Cheng

. 2018. “Causal Mediation Analysis with Multiple Causally Non-ordered Mediators.” Statistical Methods in Medical Research 27(1):3–19.

59.

Tchetgen Tchetgen

E. J.

VanderWeele

T. J.

. 2014. “Identification of Natural Direct Effects when a Confounder of the Mediator is Directly Affected by Exposure.” Epidemiology 25(2):282–91.

60.

Tomarken

A. J.

Waller

N. G.

. 2005. “Structural Equation Modeling: Strengths, Limitations, and Misconceptions.” Annual Review of Clinical Psychology 1(1):31–65.

61.

Trinkner

Jackson

Tyler

T. R.

. 2018. “Bounded Authority: Expanding “Appropriate” Police Behavior Beyond Procedural Justice.” Law and Human Beh 42(3):280–93.

62.

Tyler

T. R.

2006. Why People Obey the Law. Princeton, NJ: Princeton University Press.

63.

Tyler

T. R.

Fagan

. 2006. “Legitimacy and Cooperation: Why Do People Help the Police Fight Crime in Their Communities?” Ohio State Journal of Criminal Law 6:231–75.

64.

Tyler

T. R.

Goff

P. A.

MacCoun

R. J.

. 2015. “The Impact of Psychological Science on Policing in the United States: Procedural Justice, Legitimacy, and Effective Law Enforcement.” Psychological Science in the Public Interest 16(3):75–109.

65.

Tyler

T. R.

Jackson

. 2013. “Future Challenges in the Study of Legitimacy and Criminal Justice.” Pp. 83–104 in Legitimacy and Criminal Justice—An International Exploration, edited by Tankebe

Liebling

. Hoboken, NJ: Wiley.

66.

Tyler

T. R.

Jackson

. 2014. “Popular Legitimacy and the Exercise of Legal Authority: Motivating Compliance, Cooperation, and Engagement.” Psychology, Public Policy, and Law 20(1):78–95.

67.

VanderWeele

T. J.

2012. “Invited Commentary: Structural Equation Models and Epidemiologic Analysis.” American Journal of Epidemiology 176(7):608–12.

68.

VanderWeele

T. J.

2015. Explanation in Causal Inference—Methods for Mediation and Interaction. Oxford, England: Oxford University Press.

69.

VanderWeele

T. J.

2016. “Mediation Analysis: A Practitioner’s Guide.” Annual Review of Public Health 37(1):17–32.

70.

VanderWeele

T. J.

Knol

M. J.

. 2014. “A Tutorial on Interaction.” Epidemiological Methods 3(1):33–72.

71.

VanderWeele

T. J.

Vansteelandt

. 2014. “Mediation Analysis with Multiple Mediators.” Epidemiologic Methods 2(1):95–115.

72.

Vansteelandt

Daniel

R. M.

. 2017. “Interventional Effects for Mediation Analysis with Multiple Mediators.” Epidemiology 28(2):258–65.

73.

Wang

Arah

O. A.

. 2015. “G-computation Demonstration in Causal Mediation Analysis.” European Journal of Epidemiology 30(10):1119–27.

74.

Westreich

Edwards

J. K.

Cole

S. R.

Platt

R. W.

Mumford

S. L.

Schisterman

E. F.

. 2015. “Imputation Approaches for Potential Outcomes in Causal Inference.” International Journal of Epidemiology 44(5):1731–37.

75.

White

M. D.

Mulvey

Dario

L. M.

. 2016. “Arrestees’ Perceptions of the Police.” Criminal Justice and Behavior 43(3):343–64.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.48 MB