Accuracy of Parameter Estimates and Confidence Intervals in Moderated Mediation Models

The Regression Approach for Testing Moderated Mediation

The regression approach introduced by Muller et al. (2005) extends the methods for testing mediation by examining the moderator’s effect on each of the paths in the two models introduced by Baron and Kenny (1986). Specifically, researchers first examine the moderator’s effect on the overall effect that the independent variable has on the dependent variable in a model without the mediator. Researchers then examine the moderator’s effect on both the direct effect that the independent variable has on the dependent variable and the indirect effect that the independent variable has on the dependent variable through the mediator. The existence of moderated mediation and mediated moderation is assessed by estimating the regression coefficients in these two models. The framework introduced by Edwards and Lambert (2007) combines moderated multiple regression (MMR) and path analysis. These authors developed regression equations for testing the basic mediation model and seven moderated mediation models. Substituting different moderator values into each equation enables an analysis of the moderator’s effect on the direct, indirect, and total effects of the independent variable on the dependent variable. Preacher et al. (2007) developed an SPSS syntax named MODMED for testing moderated mediation. More recently, Hayes (2013) developed a new syntax called PROCESS for SPSS and SAS to analyze 76 moderated mediation models. Researchers can use these syntaxes to obtain estimates for all path coefficients, the conditional direct and indirect effects at certain particular values of the moderator, and the bias-corrected bootstrap confidence intervals (BCCIs) for the conditional effects.

One critical limitation of these methods is that they use regression analyses that assume that variables are measured without error. However, the variables in organizational and psychological studies are typically not directly observable and include measurement errors. The presence of measurement errors not only impedes the power to test for the significance of various parameters in the model but also results in biased estimates of the parameters. In fact, researchers have long realized this power issue in MMR and have stated that the power of MMR is a function of various factors, including the independent variable range restriction, the total sample size, the correlation between the independent variable and the moderator, the effect size of the moderating effect, and the reliabilities of the independent variable and moderator (Aguinis, 1995; Aguinis & Stone-Romero, 1997; Evans, 1985; Stone-Romero & Anderson, 1994). When the independent variable or the moderator has less than perfect reliability, the reliability of the product term of these two variables is likely to be adversely affected, which in turn results in a biased regression coefficient for the product term.

In fact, the biasing effects of measurement errors on the regression coefficients of a model that integrates moderation and mediation are complex, and these regression coefficients can be biased upward or downward. While the measurement errors in endogenous variables have no biasing effect on the regression coefficients (Ree & Carretta, 2006), the measurement error of one independent variable may result in a biasing effect on the regression coefficient of another independent variable, even if that variable is measured without errors if the two independent variables are correlated (Antonakis, Bendahan, Jacquart, & Lalive, 2010; Ree & Carretta, 2006). For example, researchers have asserted that measurement errors may result in the underestimation of mediating effects when using the regression approach. The simulation study conducted by Cheung and Lau (2008) demonstrated that both the size of the measurement errors and the effect size caused deviations in the parameters of a mediation model that was estimated by regression. Specifically, true mediation was underestimated by 5.5%, on average, when item reliability was equal to 0.90 and by 12.4%, on average, when item reliability was equal to 0.75. The simulation by Ledgerwood and Shrout (2011) further showed that while the estimates of the path between the independent variable and the mediator and of the path between the mediator and the dependent variable were attenuated by measurement errors, the direct path between the independent and dependent variables was underestimated in some cases and overestimated in other cases. This problem, which stems from less than perfect reliabilities in testing moderating and mediating effects with the regression approach, may become more serious when testing moderated mediation. In moderated mediation models, when the independent variable and/or the moderator have reliabilities that are lower than unity, the interaction effect is underestimated. When the mediator is also measured with errors, the estimated moderated mediation is further biased toward zero. As a consequence, the power available for detecting the moderated mediation suffers considerably.

The LMS Approach for Testing Moderated Mediation

Given the limitations of regression, we suggest the use of SEM to examine moderated mediation with latent interaction models. Because SEM corrects for measurement errors when estimating structural parameters, it avoids the problem generated by less than perfect reliability. However, the modeling of a latent interaction is not a straightforward procedure. Moosbrugger, Schermelleh-Engel, and Klein (1997) summarized five methodological problems of latent interactions in SEM as follows: (a) the measurement error of indicators, (b) the nonlinearity of the factor loadings of product indicators, (c) the non-zero mean structure of the latent product term, (d) the effect of linear transformation on the structural coefficient, and (e) the non-normally distributed latent product term. The search for appropriate analytical methods for the latent interaction effect has been undertaken for three decades. The first method to receive wide attention was proposed by Kenny and Judd (1984). This method requires the researcher to create products of indicators as indicators of the latent interaction term and to impose various nonlinear constraints in the model. Hayduk (1987), Jaccard and Wan (1995), Jöreskog and Yang (1996), and Algina and Moulder (2001) suggested further development of the product-indicator methods with modifications to the nonlinear constraints. The two-step approach proposed by Ping (1996) and the method recommended by Marsh, Wen, and Hau (2004), which uses matched indicators, do not require nonlinear constraints. However, all these methods have a number of practical and statistical drawbacks. There are many reviews and comparisons of various methods for modeling latent interactions (e.g., Cortina, Chen, & Dunlap, 2001; Jöreskog, 1998; Marsh et al., 2004; Marsh, Wen, Hau, & Nagengast, 2013; Moosbrugger et al., 1997), and interested readers are encouraged to read those reviews. While simulations have shown that the two-stage least squares (2SLS) method suggested by Bollen (1995, 1996) and Bollen and Paxton (1998), which uses instrumental variables (by using special linear combinations of indicators) to estimate interaction effects, provides unbiased estimates (Klein & Moosbrugger, 2000; Klein & Stoolmiller, 2003), the 2SLS method requires a large sample size to achieve a given power (it is less efficient) because the specific distributions induced by the product term are not exploited (Klein & Moosbrugger, 2000; Klein & Stoolmiller, 2003; Moosbrugger et al., 1997). In addition, because only one indicator of the outcome variable is used, the estimated regression coefficients and standard errors will depend on which indicator of the outcome variable is chosen (Jöreskog, 1998). In sum, in empirical studies, testing interactions using SEM is far less popular than employing the regression approach with observed variables.

Among all the methods for modeling latent interactions, the LMS approach is investigated in the current study for two reasons. First, this method does not require the creation of product indicators and can be easily implemented with Mplus without imposing complicated nonlinear constraints. Second, simulation studies have shown that the LMS approach provides unbiased estimators and standard errors (Klein & Moosbrugger, 2000; Schermelleh-Engel, Klein, & Moosbrugger, 1998). If the LMS approach provides accurate estimates and CIs, it might be a good alternative to the regression approach to estimate a moderated mediation effect. The primary criticism of the LMS approach is that it is computationally intensive. However, with the rapid development of desktop computing capabilities and power, computational intensiveness is no longer an issue. The LMS approach makes use of a new maximum likelihood (ML) method that explicitly takes into account the non-normal distributional characteristics of the joint indicator vector in a general latent interaction model. An elementary interaction model (Kenny & Judd, 1984) with one latent interaction effect can be represented as follows:

η = α + γ 1 ξ 1 + γ 2 ξ 2 + γ 3 ξ 1 ξ 2 + ζ.

1

By incorporating the matrix Ω, an upper triangular matrix that contains zeros in the diagonal, a structural equation of the general interaction model that includes multiple latent interactions is created (Klein & Moosbrugger, 2000, p. 460):

η = α + Γ ξ + ξ ’ Ω ξ + ζ,

2

where η is a (1 × 1) latent endogenous variable; α is a (1 × 1) intercept term; ξ is a (n × 1) vector of latent exogenous variables; Γ is the (1 × n) coefficient matrix representing ξ’s impact on η; Ω is the (n × n) coefficient matrix representing the impact of the product terms ξ _i ξ _j (i < j) on η; ζ is a (1 × 1) disturbance variable with E(ζ) = 0 and Cov(ζ, ξ′) = 0; and

m a t r i x Ω = [ 0 ω 1 , 2 … ω 1 , n ⋮ ⋮ ⋮ ⋮ 0 … 0 ω n - 1 , n 0 … … 0 ] ,

3

where ω i , j = 0 for i ≥ j. In Equation 2, the quadratic form ξ ′ Ω ξ is nonlinear and includes product terms ω i , j ξ i ξ j (i < j), which model the interaction between each pair of the latent exogenous variables ξ on the latent endogenous variable η. The structural equation for the interaction model in Equation 1 can be represented as follows (Klein & Moosbrugger, 2000, p. 460):

η = α + ( γ 1 γ 2 ) ( ξ 1 ξ 2 ) + ( ξ 1 ξ 2 ) ( 0 ω 12 0 0 ) ( ξ 1 ξ 2 ) + ζ.

4

In LMS, the latent exogenous variables ξ are decomposed into two vectors z₁ and z₂ , whereas η (and the y indicators) is linear in z₂ and nonlinear in z₁ . The vector z₁ is used to derive the multivariate density function (joint probability distribution) of the indicators, which is then used to estimate the parameters in Equation 2 with the expectation-maximation (EM) algorithm (Dempster, Laird, & Rubin, 1977; Redner & Walker, 1984). In adopting the EM algorithm, the LMS approach performs an iterative estimation of the model parameters and stops when the log-likelihood function of the observed variables is maximized. During the whole process, no products of indicators are created. The simulation conducted by Klein and Moosbrugger (2000) showed that LMS generated unbiased parameter estimations. Similarly, the LMS estimation of standard errors did not show substantial bias. When distribution assumptions were only moderately violated, the simulation results revealed that the approach remained rather robust.

The Reliability-Corrected Single-Indicator LMS Approach

The LMS approach may not be desirable for analyzing certain types of latent variable models, such as models with many indicators and/or studies with small sample sizes. Large numbers of indicators frequently lead to less than adequate overall model fit because a large number of parameters are fixed at zero. In addition, small sample size may result in model non-convergence. Under such conditions, researchers may consider using item parcels rather than all indicators for the constructs. At the extreme, researchers may revert to testing the structural equation model with a simple average score as the single-indicator of a construct. However, rather than assuming that the variables are measured without error, as in the regression approach, the unique variances can be fixed at (1 – reliability) × indicator variances, and the factor loadings can be fixed at 1 (Bollen, 1989; Coffman & MacCallum, 2005; Hayduk, 1987). Then, the model is analyzed with LMS to estimate the moderation, direct, and indirect effects. We refer to this approach as the reliability-corrected single-indicator LMS (RCSLMS) approach. Nevertheless, it is important to note that only items with a well-known unidimensional structure, no cross-loadings on multiple factors, no correlated uniqueness, and no other sources of misspecification could be combined into parcels or single indicators (Bandalos, 2002; Bandalos & Finney, 2001; Marsh, Ludtke, Nagengast, Morin, & Vov Davier, 2013).

Index of Moderated Mediation

Figure 1a shows a first-stage moderation model (Edwards & Lambert, 2007), where moderator W affects the relationship between independent variable X and mediator M. While moderation effect a₃ and effect b form the moderated mediation effect, either parameter informs the magnitude of the moderated mediation effect. Hence, Morgan-Lopez and MacKinnon (2006) and Hayes (in press) suggested examining the product term a₃b, which represents the change in the indirect effect of X on Y through M for a unit change in W, as a formal test of moderated mediation. Hayes (in press) referred to this parameter as the index of moderated mediation (Index MM) and derived the function for various moderated mediation models. When the CI of the index of moderated mediation does not include zero, it might be concluded that the indirect effects at various levels of the moderator are statistically significantly different.

OPEN IN VIEWER

Figure 1.

(a) Moderated mediation model with observed variables used in simulation. (b) Moderated mediation model with latent variables used in simulation.

Bias-Corrected Bootstrap Confidence Intervals

Researchers have long indicated that the sampling distribution of mediation (indirect effect) as the product of two path coefficients is not normal (Bollen & Stine, 1990; MacKinnon, Lockwood, & Williams, 2004; Stone & Sobel, 1990). It is also generally acknowledged that the distribution of the product of two independent normal distribution variables is not normal; in other words, even if both the independent variable X and the moderator W are normally distributed, their product term XW is not normally distributed. Thus, using a t test or a Sobel test, which assume normality, to test the significance of the parameters may not be appropriate. Previous studies have advocated for the construction of BCCIs for testing mediating effects (e.g., Cheung & Lau, 2008; MacKinnon et al., 2004; Williams & MacKinnon, 2008). The SPSS and SAS syntaxes developed by Preacher et al. (2007) and Hayes (2013) also generate BCCIs for moderated mediation effects in the output. Edwards and Lambert (2007) also presented SPSS syntax (CNLR syntax) to produce bootstrap estimates for the regression coefficients and Excel files to produce the BCCIs of the estimated coefficients. With bootstrapping, the sampling distribution of the moderated mediation effect is first estimated nonparametrically. Then, information from the bootstrap sampling distribution is used to construct a confidence interval for the moderated mediation effect (Preacher et al., 2007). We thus also recommend using BCCIs in testing moderated mediation effects in latent interaction models.

In the following sections, we first present the results of our comprehensive simulation that compares the observed variable regression approach, the LMS approach, and the newly proposed RCSLMS approach, illustrating their performance as measured by accuracy in parameter estimation, CI coverage, statistical power, and Type I error rates. Then, we provide a numerical example that demonstrates how to use the LMS approach with Mplus to test for moderated mediation with latent variable models.

Simulation

Method

We conducted two simulation studies to compare the regression approach, the LMS approach, and the RCSLMS approach. These studies aimed to illustrate the performance of the three approaches in correcting for measurement errors when estimating the confidence intervals of the estimated parameters. First, as shown in Figure 1a, a population model with the following four variables was generated: independent variable X, mediator M, moderator W, and dependent variable Y. Moderator W was proposed to moderate the effect of X on M, thus moderating the mediating effect of X on Y through M. Specifically, three levels of effect size of a₃ (0.0, 0.2, and 0.4) and three levels of sample size N (100, 200, and 500) were manipulated. The population parameters (true values) of the moderation effect (a₃ ) are shown at the top of Table 1. The manipulations of the effect size of a₃ = 0.0 and 0.2 followed those adopted by Klein and Moosbrugger (2000). The effect size of 0.7, as used by Klein and Moosbrugger in their simulation, was not included because 0.7 might be considered too large as an effect size in organization studies. Instead, 0.4 was included in the current study. The value of 0.0 was used to test for Type I error, and the values of 0.2 and 0.4 were used to test for power. Because a₃ (XW on M) was varied due to manipulations, a₃b (Index MM) was also varied. The manipulations of the sample size represent certain commonly used sample sizes in organizational research. The population values of other direct paths are shown on the left side of Table 1. The population parameters a₁ (X on M) = 0.2 and a₂ (W on M) = 0.4 followed those adopted by Klein and Moosbrugger (2000), and the direct paths c’ (X on Y) = 0.14 and b (M on Y) = 0.59 represented the small and large direct effects in a mediation model (MacKinnon et al., 2004). The covariance between independent variable X and moderator W was set at 0.5.

OPEN IN VIEWER

Table 1.

Simulation Results for Data With Small Unique Variance.

		N	True Value of a3 = 0			True Value of a3 = 0.2			True Value of a3 = 0.4
			REG	LMS	RCSLMS	REG	LMS	RCSLMS	REG	LMS	RCSLMS
X on M	Est.	100	0.1895	0.2028	0.1985	0.1891	0.2032	0.2004	0.1886	0.2038	0.2020
(a1 = 0.2)		200	0.1918	0.2022	0.2009	0.1916	0.2033	0.2105	0.1913	0.2045	0.2047
		500	0.1910	0.2018	0.2005	0.1912	0.2035	0.2027	0.1915	0.2050	0.2048
	SE	100	0.1032	0.1618	0.1477	0.1057	0.1656	0.1518	0.1180	0.1733	0.1601
		200	0.0697	0.1039	0.0994	0.0718	0.1078	0.0923	0.0760	0.1145	0.1100
		500	0.0419	0.0627	0.0600	0.0430	0.0645	0.0616	0.0453	0.0684	0.0655
	AB	100	0.0829	0.1243	0.1181	0.0853	0.1280	0.1221	0.0897	0.1348	0.1289
		200	0.0572	0.0835	0.0804	0.0588	0.0862	0.0712	0.0618	0.0914	0.0888
		500	0.0339	0.0489	0.0473	0.0347	0.0508	0.0491	0.0365	0.0546	0.0527
	COV	100	0.9420	0.9540	0.9560	0.9400	0.9460	0.9480	0.9300	0.9400	0.9420
		200	0.9580	0.9600	0.9580	0.9580	0.9560	0.9620	0.9500	0.9520	0.9500
		500	0.9560	0.9580	0.9700	0.9520	0.9520	0.9560	0.9480	0.9580	0.9520
	SIG	100	0.4260	0.2500	0.2500	0.4260	0.2320	0.2580	0.4280	0.2340	0.2360
		200	0.7520	0.4760	0.4840	0.7140	0.4600	0.5840	0.6880	0.4240	0.4400
		500	0.9900	0.8740	0.8800	0.9880	0.8700	0.8780	0.9840	0.8420	0.8480
W on M	Est.	100	0.3248	0.4026	0.4085	0.3249	0.4008	0.4071	0.3250	0.4003	0.4063
(a2 = 0.4)		200	0.3242	0.4037	0.4045	0.3246	0.4034	0.3854	0.3251	0.4034	0.4058
		500	0.3242	0.4032	0.4019	0.3239	0.4027	0.4017	0.3236	0.4021	0.4018
	SE	100	0.1019	0.1628	0.1486	0.1028	0.1633	0.1501	0.1062	0.1675	0.1557
		200	0.0696	0.1087	0.1006	0.0706	0.1107	0.0993	0.0735	0.1151	0.1075
		500	0.0422	0.0648	0.0598	0.0434	0.0668	0.0618	0.0457	0.0702	0.0659
	AB	100	0.1030	0.1283	0.1183	0.1035	0.1277	0.1190	0.1057	0.1302	0.1222
		200	0.0856	0.0861	0.0809	0.0856	0.0873	0.0836	0.0865	0.0912	0.0848
		500	0.0771	0.0527	0.0486	0.0776	0.0545	0.0502	0.0784	0.0570	0.0532
	COV	100	0.8660	0.9560	0.9540	0.8780	0.9520	0.9660	0.8900	0.9520	0.9520
		200	0.8200	0.9600	0.9620	0.8200	0.9560	0.9620	0.8400	0.9520	0.9500
		500	0.6040	0.9720	0.9600	0.6060	0.9660	0.9660	0.6100	0.9600	0.9620
	SIG	100	0.8760	0.6940	0.7440	0.8820	0.6860	0.7140	0.8580	0.6620	0.7020
		200	0.9920	0.9680	0.9740	0.9920	0.9720	0.9700	0.9880	0.9540	0.9580
		500	1.0000	1.0000	1.0000	1.0000	1.0000	1.0000	1.0000	1.0000	1.0000
XW on M	Est.	100	0.0020	0.0033	0.0028	0.1400	0.2058	0.2039	0.2781	0.4088	0.4053
(a3 )		200	0.0025	0.0029	0.0038	0.1405	0.2055	0.1999	0.2785	0.4081	0.4054
		500	0.0040	0.0053	0.0052	0.1418	0.2070	0.2049	0.2796	0.4085	0.4048
	SE	100	0.0822	0.1220	0.1196	0.0841	0.1325	0.1215	0.0891	0.1529	0.1285
		200	0.0570	0.0840	0.0812	0.0577	0.0873	0.0737	0.0609	0.0994	0.0866
		500	0.0336	0.0475	0.0471	0.0348	0.0525	0.0489	0.0376	0.0625	0.0532
	AB	100	0.0633	0.0933	0.0920	0.0819	0.1005	0.0928	0.1283	0.1183	0.1001
		200	0.0445	0.0649	0.0630	0.0685	0.0682	0.0549	0.1225	0.0786	0.0672
		500	0.0267	0.0372	0.0367	0.0597	0.0418	0.0388	0.1204	0.0499	0.0430
	COV	100	0.9420	0.9560	0.9520	0.8840	0.9500	0.9500	0.7160	0.9520	0.9540
		200	0.9360	0.9360	0.9280	0.8080	0.9500	0.9440	0.4720	0.9560	0.9280
		500	0.9360	0.9380	0.9420	0.6180	0.9340	0.9420	0.1120	0.9380	0.9380
	SIG	100	0.0580	0.0440	0.0480	0.3780	0.3760	0.4080	0.8580	0.8360	0.8620
		200	0.0640	0.0640	0.0720	0.6720	0.6780	0.7480	0.9880	0.9940	0.9920
		500	0.0640	0.0620	0.0580	0.9660	0.9780	0.9760	1.0000	1.0000	1.0000
M on Y	Est.	100	0.4716	0.6039	0.5979	0.4763	0.6030	0.5967	0.4884	0.6008	0.5951
(b = 0.59)		200	0.4705	0.5941	0.5922	0.4750	0.5923	0.5879	0.4871	0.5905	0.5896
		500	0.4738	0.5934	0.5941	0.4783	0.5918	0.5928	0.4902	0.5905	0.5916
	SE	100	0.0956	0.1585	0.1340	0.0943	0.1551	0.1288	0.0902	0.1434	0.1177
		200	0.0672	0.1088	0.0910	0.0666	0.1064	0.0788	0.0636	0.0985	0.0817
		500	0.0419	0.0667	0.0568	0.0413	0.0640	0.0555	0.0394	0.0584	0.0509
	AB	100	0.1281	0.1231	0.1073	0.1237	0.1213	0.1041	0.1126	0.1133	0.0952
		200	0.1221	0.0866	0.0711	0.1179	0.0842	0.0583	0.1062	0.0781	0.0635
		500	0.1163	0.0534	0.0451	0.1118	0.0514	0.0437	0.1000	0.0469	0.0399
	COV	100	0.7320	0.9500	0.9540	0.7280	0.9460	0.9440	0.7480	0.9480	0.9420
		200	0.5480	0.9400	0.9300	0.5540	0.9440	0.9440	0.6140	0.9560	0.9280
		500	0.1960	0.9480	0.9540	0.2240	0.9560	0.9440	0.2580	0.9520	0.9320
	SIG	100	0.9960	0.9900	0.9940	0.9960	0.9900	0.9900	0.9980	0.9940	0.9940
		200	1.0000	1.0000	1.0000	1.0000	1.0000	1.0000	1.0000	1.0000	1.0000
		500	1.0000	1.0000	1.0000	1.0000	1.0000	1.0000	1.0000	1.0000	1.0000
X on Y	Est.	100	0.1550	0.1478	0.1444	0.1534	0.1479	0.1448	0.1494	0.1480	0.1456
(c’ = 0.14)		200	0.1528	0.1434	0.1425	0.1514	0.1444	0.1399	0.1475	0.1453	0.1443
		500	0.1508	0.1414	0.1406	0.1495	0.1421	0.1411	0.1457	0.1427	0.1417
	SE	100	0.0979	0.1422	0.1362	0.0975	0.1401	0.1340	0.0970	0.1369	0.1308
		200	0.0656	0.0932	0.0896	0.0652	0.0918	0.0791	0.0648	0.0892	0.0855
		500	0.0427	0.0603	0.0584	0.0423	0.0595	0.0576	0.0419	0.0579	0.0561
	AB	100	0.0793	0.1125	0.1092	0.0789	0.1113	0.1079	0.0785	0.1086	0.1057
		200	0.0527	0.0727	0.0708	0.0520	0.0714	0.0560	0.0512	0.0697	0.0673
		500	0.0349	0.0474	0.0462	0.0345	0.0471	0.0456	0.0336	0.0463	0.0446
	COV	100	0.9180	0.9400	0.9440	0.9160	0.9400	0.9400	0.9180	0.9460	0.9440
		200	0.9380	0.9540	0.9500	0.9400	0.9440	0.9520	0.9460	0.9440	0.9520
		500	0.9360	0.9340	0.9300	0.9380	0.9360	0.9360	0.9400	0.9420	0.9420
	SIG	100	0.4000	0.2020	0.2020	0.3920	0.1980	0.2080	0.3840	0.2020	0.2120
		200	0.6540	0.3360	0.3440	0.6460	0.3460	0.4720	0.6200	0.3560	0.3520
		500	0.9460	0.6920	0.6940	0.9520	0.7060	0.7020	0.9440	0.7180	0.7180
Index MM	Est.	100	0.0013	0.0029	0.0024	0.0672	0.1227	0.1218	0.1365	0.2427	0.2407
(a3b)		200	0.0011	0.0018	0.0023	0.0668	0.1210	0.1176	0.1358	0.2398	0.2389
		500	0.0019	0.0031	0.0031	0.0678	0.1222	0.1215	0.1372	0.2408	0.2394
	SE	100	0.0404	0.0763	0.0743	0.0445	0.0869	0.0796	0.0528	0.1044	0.0899
		200	0.0271	0.0497	0.0485	0.0291	0.0546	0.0465	0.0351	0.0670	0.0601
		500	0.0159	0.0283	0.0281	0.0178	0.0329	0.0313	0.0221	0.0416	0.0375
	AB	100	0.0302	0.0564	0.0555	0.0576	0.0644	0.0604	0.1013	0.0793	0.0702
		200	0.0209	0.0383	0.0372	0.0525	0.0423	0.0358	0.1003	0.0523	0.0471
		500	0.0126	0.0220	0.0219	0.0502	0.0255	0.0243	0.0988	0.0323	0.0293
	COV	100	0.9360	0.9480	0.9400	0.7700	0.9560	0.9560	0.5540	0.9560	0.9540
		200	0.9300	0.9300	0.9240	0.6220	0.9560	0.9640	0.2680	0.9600	0.9460
		500	0.9340	0.9360	0.9360	0.2660	0.9440	0.9400	0.0240	0.9440	0.9340
	SIG	100	0.0640	0.0520	0.0600	0.4240	0.4060	0.4380	0.8740	0.8580	0.8760
		200	0.0700	0.0700	0.0760	0.6940	0.6960	0.7700	0.9880	0.9920	0.9920
		500	0.0660	0.0642	0.0640	0.9680	0.9780	0.9780	1.0000	1.0000	1.0000

Note: REG = regression; LMS = latent moderated structural equations; RCSLMS = reliability-corrected single-indicator LMS; Est. = estimate; SE = standard error; AB = absolute bias; COV = 95% coverage; SIG = Type I error/power.

Next, the four variables were treated as latent variables, and each was measured by four indicators, the values of which were simulated by adding random error to the indicators. The latent interaction model is presented in Figure 1b. Both the factor loading and unique variance of all indicators were set at 1. These parameters simulated the situation in which the Cronbach’s alpha of the simple average score for each latent variable was approximately equal to 0.8. To examine the effect of lower reliability of the estimated parameters and confidence intervals, we conducted a second simulation with the unique variance set at 2 and a₃ set at 0.2 and with three levels of sample size (N = 100, 200, and 500). These values simulated the situation in which the Cronbach’s alpha of the simple average score for each latent variable was approximately equal to 0.67. Taken together, there were 9 conditions (3 sample sizes × 3 moderated mediation effects) in Simulation 1 and 3 conditions (3 sample sizes) in Simulation 2. Five hundred simulated data sets were created for each of the 12 sets of population values, and all the simulated data sets were analyzed under the three approaches.

For the regression approach, the simple average scores of the four indicators of each latent variable were treated as observed variables for estimating the model parameters in Figure 1a, with the procedures repeated for all 500 simulated data sets. A bias-corrected bootstrap confidence interval for each estimated parameter was created with 1,000 bootstrap samples. This procedure simulates the most commonly used approach to analyzing moderated mediation models with latent variables. For the LMS approach, we used the Mplus 7.3 syntax in Appendix 1 to estimate the parameters and create BCCIs for each simulated data set. For the RCSLMS approach, the simple average score of the four indicators of each latent variable was calculated to form the single-indicator. The unique variance was fixed at (1 – 0.8) × 1.25 = 0.25 for the high reliability condition in Simulation 1 and at (1 – 0.67) × 1.5 = 0.5 for the low reliability condition in Simulation 2. The Mplus 7.3 syntax that was used is shown in Appendix 2. The procedure for testing moderated mediation with Mplus is explained in greater detail in the numerical example.

Six criteria were used to evaluate the performance of the three approaches. First, accuracy was measured by the average estimates, standard error of estimates, and average of the absolute bias values obtained from the 500 simulated samples. Absolute bias was computed as the difference between the estimate and the true value. Second, CI coverage was examined by calculating the percentage of the 500 simulated samples in which the CIs included the true value of the parameter. Next, statistical power was examined as the percentage of the 95% CIs that did not include zero when the population value of the parameter was not zero. When the population value of a₃ was 0.0, this percentage reflected Type I error for testing a₃ and Index MM. Finally, completion rate of the bootstrap samples was examined.

Results of Simulation 1

Table 1 presents the simulation results produced by the regression approach, the LMS approach, and the RCSLMS approach. The average and standard error of the parameter estimates, in addition to the average of the absolute bias over the 500 simulated samples, are presented for three sample sizes (100, 200, and 500) and three levels of the true values of the interaction effect, a₃ . The reliabilities of the four constructs, X, W, M, and Y, in the regression approach were 0.800, 0.798, 0.792, and 0.793, respectively.

Average estimates

In general, Table 1 reveals that the average values of the estimated coefficients of a₁ , a₂ , and b obtained from the observed variable regression approach were smaller than the true values under all conditions. When the true value of a₃ equaled 0.0, the average estimates of a₃ were close to the true value for all three sample sizes. However, when the true value of a₃ equaled 0.2 or 0.4, the average estimates of a₃ were all smaller than the true value. As a result, the average estimates of Index MM obtained from the regression were close to the true value (0.0) when a₃ was set at 0.0 but smaller than the true values of 0.118 and 0.236 when a₃ was set at 0.2 and 0.4, respectively. Finally, for c’, for all three sample sizes, the average values of the estimated coefficients were all larger than the true value.

However, Table 1 shows that the estimated coefficients obtained from the LMS and RCSLMS approaches, in general, were near the true values. As a result, in most cases, the estimates of Index MM that were calculated from these two approaches were also near the true values. For all three approaches, increasing the sample size had little effect on the average estimated coefficients.

Standard error of estimates

Following Ledgerwood and Shrout (2011), standard errors of estimates were used to measure the precision of the estimates. Table 1 shows that in all conditions, the standard errors of the estimates produced by the regression approach were the smallest among those produced by the three approaches. The standard errors of the estimates produced by the LMS and RCSLMS approaches were similar, although the LMS approach produced slightly larger standard errors. In all three approaches, increasing the sample size produced smaller standard errors under all conditions.

Average absolute bias

In addition, Table 1 presents the average absolute bias of the three approaches at different true values of a₃ and at different sample sizes. Because the standard error of estimates was centered on the estimated value rather than the true value, it was not a good measure of accuracy. Hence, we calculated the average absolute bias as an additional measure of accuracy. As the value of average absolute bias decreases, the accuracy increases. The results showed that in general, the estimates produced by the LMS and RCSLMS approaches were similar in accuracy, although the RCSLMS approach produced slightly more accurate estimates than the LMS approach under all conditions. A closer examination of the average absolute bias values reveals that the estimated coefficients produced by all three approaches were more accurate with smaller effect sizes and larger sample sizes. However, the effects were stronger for the regression approach than for the LMS and RCSLMS approaches. Hence, the regression approach produced more accurate estimates of parameters with small true values, namely, a₁ and c’, than did the LMS and RCSLMS approaches across all three sample size levels. However, the LMS and RCSLMS approaches produced more accurate estimates of b with effect size at 0.59 than did the regression approach. For direct effect a₂ with moderate effect size, the regression approach produced more accurate estimates when N = 100, whereas the LMS and RCSLMS approaches produced more accurate estimates when N = 500. Similarly, the estimates of interaction effect a₃ using the LMS and RCSLMS approaches achieved higher accuracy than the regression approach when the true value of a₃ was set at 0.4 for all three sample sizes. By contrast, when the true value of a₃ was set at 0, the accuracy of the regression approach was higher than that of the LMS and RCSLMS approaches for all three sample sizes. When the true value of a₃ was 0.2, the regression approach gave more accurate estimates when N = 100, whereas the LMS and RCSLMS approaches gave more accurate estimates when N = 500. The three approaches produced estimates with a similar level of accuracy when N = 200. The results of Index MM demonstrated a similar pattern.

CI coverage

Table 1 also presents the coverage rates of CIs using different approaches under various conditions of a₃ effect sizes and sample sizes. Because 95% CIs were constructed, a coverage rate close to 0.95 was desirable. For the LMS approach, the CI coverage rates ranged from 0.930 to 0.972, whereas for the RCSLMS approach, the CI coverage rates were between 0.924 and 0.970. The coverage rates of the true values were near 0.95 in most cases when these two approaches were employed, suggesting that these two approaches performed in a desirable manner. For the regression approach, the range of the CI coverage rates was much larger, from 0.024 to 0.958. Table 1 reveals that the regression approach produced CIs with good coverage rates for parameters with small population values. For instance, the CI coverage rate of a₁ with population value at 0.2 ranged from 0.930 to 0.958. Similarly accurate CIs were found for parameters c’ with population value at 0.14 and a₃ and Index MM with population values at zero. However, when the effect size was large, such as in the case of parameter b with population value at 0.59, the CI coverage rate was 0.748 when N = 100 but deteriorated to 0.258 when N = 500. The CI coverage rates for interaction effect a₃ and Index MM produced by the regression approach were even worse. When the true value of a₃ equaled 0.4, the CI coverage rate of the true value of a₃ was 0.716 when N = 100 and only 0.112 when N = 500. Similarly, the CI coverage rate for Index MM was only 0.554 when N = 100 and merely 0.024 when N = 500. These CI coverage rates that were generated by the regression approach suggested that the larger the sample size, the greater the likelihood that the 95% CIs constructed did not include the true values.

Power and Type I error

Table 1 suggests that the CIs of the parameters produced by all three approaches enjoyed higher power with larger effect sizes and larger sample sizes. All three approaches produced sufficient power for testing direct effect b, which had a large effect size, even with a sample size of 100. The regression approach had adequate power to test for a₂ when N = 100, whereas the LMS and RCSLMS approaches required a sample size of 200 or 500 to provide sufficient power for the test. When testing the parameter a₁ , which had a smaller effect size, all three approaches achieved adequate power only when N = 500. Finally, when the effect size was even smaller, only the regression approach provided adequate power for testing c’ when N = 500. Neither the LMS nor the RCSLMS approaches had sufficient power for testing a small effect, even when N = 500. While the regression approach produced CIs with greater capabilities of testing direct effects than the LMS and RCSLMS approaches, there were no substantial differences in power for testing the interaction effect a₃ and Index MM among the three approaches. All three approaches had adequate power to test for a₃ and Index MM with a sample size of 100 when the true value of a₃ equaled 0.4. When the true value of a₃ equaled 0.2, all three approaches provided adequate power for testing a₃ and Index MM only when N = 500. Finally, as shown in Table 1, an examination of the Type I error rates for true values of a₃ and Index MM = 0 reveals that all three approaches produced accurate Type I error rates with the 95% CIs, ranging from 0.044 to 0.076.

Completion rate

Although no instances of non-convergence or improper solutions were observed when using all three approaches, the completion rates of the 1,000 bootstrap samples for each simulated data set were less than 100% in some conditions when using LMS and RCSLMS approaches. The completion rate was less than 100% because there were instances of non-convergence or improper solutions. The completion rate decreases with smaller sample sizes. When using the LMS approach, the completion rate for N = 500 was 100%. The average completion rate for N = 200 was 99.9997%, whereas the completion rate for N = 100 was 99.8905%. When using the RCSLMS approach, the completion rates for N = 500 and N = 200 were both 100%. The average completion rate for N = 100 was 99.9989%.

Results of Simulation 2

Table 2 shows the results of the simulation for data with larger unique variance than the first simulation. The conditions simulated data with a population value of a₃ at 0.2 and scales with Cronbach’s alpha at approximately 0.67. The lower reliability resulted in larger standard errors, larger average absolute bias, and lower power for all three approaches. Although the lower reliability resulted in slightly inflated estimates and larger CI coverage rates for both the LMS and RCSLMS approaches, it resulted in more attenuated estimates and a much lower CI coverage rate using the regression approach. The average estimated value of a₃ dropped to 0.10, and the CI coverage rate was lowered to 0.574 when N = 200. In the extreme case, the CI coverage rate for Index MM when N = 500 was only 0.012 using the regression approach. That is, only 6 of the 500 simulated samples had CIs that included the true value of 0.118! No instances of non-convergence or improper solutions were observed when using all three approaches. When using the LMS approach, the completion rate for N = 500 was 100%. The average completion rate for N = 200 and N = 100 were 99.9757% and 96.022%, respectively. When using the RCSLMS approach, the completion rates for N = 500, N = 200, and N = 100 were 100%, 99.9964%, and 99.4826%, respectively.

OPEN IN VIEWER

Table 2.

Simulation Results for Data With Large Unique Variance and Population Interaction Effect = 0.2.

		N	REG	LMS	RCSLMS			N	REG	LMS	RCSLMS
X on M	Est.	100	0.1735	0.2090	0.2003	M on Y	Est.	100	0.4034	0.6218	0.6079
(a1 = 0.2)		200	0.1769	0.2070	0.2035	(b = 0.59)		200	0.4010	0.5958	0.5941
		500	0.1762	0.2064	0.2032			500	0.4044	0.5926	0.5951
	SE	100	0.1092	0.2627	0.2103		SE	100	0.0976	0.2421	0.1749
		200	0.0737	0.1499	0.1371			200	0.0680	0.1494	0.1158
		500	0.0444	0.0887	0.0823			500	0.0423	0.0888	0.0723
	AB	100	0.0905	0.1842	0.1655		AB	100	0.1885	0.1819	0.1397
		200	0.0627	0.1187	0.1111			200	0.1890	0.1178	0.0914
		500	0.0397	0.0694	0.0647			500	0.1856	0.0706	0.0570
	COV	100	0.9240	0.9640	0.9640		COV	100	0.4880	0.9760	0.9620
		200	0.9360	0.9640	0.9660			200	0.1780	0.9640	0.9440
		500	0.9280	0.9440	0.9500			500	0.0120	0.9640	0.9460
	SIG	100	0.3820	0.1200	0.1220		SIG	100	0.9820	0.8260	0.8400
		200	0.6500	0.2780	0.3040			200	1.0000	0.9980	0.9980
		500	0.9660	0.6720	0.6720			500	1.0000	1.0000	1.0000
W on M	Est.	100	0.2767	0.4198	0.4162	X on Y	Est.	100	0.1485	0.1515	0.1432
(a2 = 0.4)		200	0.2752	0.4110	0.4082	(c’ = 0.14)		200	0.1474	0.1467	0.1444
		500	0.2744	0.4041	0.4012			500	0.1450	0.1430	0.1414
	SE	100	0.1025	0.2747	0.2084		SE	100	0.1004	0.2126	0.1826
		200	0.0699	0.1532	0.1347			200	0.0670	0.1286	0.1164
		500	0.0431	0.0911	0.0801			500	0.0436	0.0792	0.0755
	AB	100	0.1345	0.1874	0.1604		AB	100	0.0804	0.1584	0.1452
		200	0.1269	0.1197	0.1068			200	0.0531	0.0975	0.0910
		500	0.1256	0.0734	0.0651			500	0.0349	0.0631	0.0599
	COV	100	0.7720	0.9720	0.9740		COV	100	0.9280	0.9640	0.9700
		200	0.5900	0.9720	0.9580			200	0.9400	0.9560	0.9660
		500	0.2100	0.9540	0.9600			500	0.9440	0.9380	0.9420
	SIG	100	0.7360	0.3720	0.4020		SIG	100	0.3460	0.0920	0.1020
		200	0.9700	0.8180	0.8280			200	0.5920	0.1980	0.2280
		500	1.0000	0.9980	1.0000			500	0.9240	0.4760	0.4580
XW on M	Est.	100	0.1024	0.2163	0.2059	Index MM	Est.	100	0.0418	0.1300	0.1244
(a3 = 0.2)		200	0.1027	0.2117	0.2076	(a3b = 0.118)		200	0.0412	0.1244	0.1232
		500	0.1037	0.2100	0.2057			500	0.0420	0.1236	0.1222
	SE	100	0.0821	0.2013	0.1595		SE	100	0.0365	0.1292	0.1036
		200	0.0562	0.1215	0.1078			200	0.0237	0.0753	0.0678
		500	0.0342	0.0715	0.0635			500	0.0147	0.0446	0.0475
	AB	100	0.1061	0.1437	0.1215		AB	100	0.0772	0.0912	0.0789
		200	0.0993	0.0937	0.0831			200	0.0769	0.0577	0.0789
		500	0.0963	0.0575	0.0510			500	0.0760	0.0342	0.0319
	COV	100	0.7720	0.9720	0.9500		COV	100	0.4840	0.9680	0.9620
		200	0.5740	0.9600	0.9320			200	0.1980	0.9640	0.9400
		500	0.1940	0.9340	0.9320			500	0.0120	0.9340	0.9420
	SIG	100	0.2460	0.1840	0.2660		SIG	100	0.2880	0.1940	0.2520
		200	0.4660	0.4680	0.5040			200	0.4960	0.5120	0.5420
		500	0.8600	0.8820	0.8900			500	0.8640	0.8900	0.8980

Note: REG = regression; LMS = latent moderated structural equations; RCSLMS = reliability-corrected single-indicator LMS; Est. = estimate; SE = standard error; AB = absolute bias; COV = 95% coverage; SIG = power.

Numerical Example

In this section, we provide the details of the analytical procedures that are involved in the LMS approach for examining moderated mediation. This is a three-step procedure for testing moderated mediation in the LMS approach with Mplus 7.3. The model shown in Figure 1b is used for illustration, and the same procedure can be used to analyze any moderated mediation model.

In the first step, the model is estimated without the interaction term. Because the LMS approach does not provide conventional fit indices for assessing overall model fit, the test for a moderated mediation model requires the estimation of two models (Muthén, 2012). In the first model, the latent interaction term is not included (the model in Figure 1b without latent interaction XW and path a₃ ). If the model without the latent interaction provides adequate fit and the factor loadings are significant, then we can conclude that the indicators measure the constructs well and proceed to the second step. Table 3 shows the various fit indices of the model without the latent interaction. For this numerical example, the χ²(99) = 116.81 (p = .1069) indicates that the model with no latent interaction fits the data well. The estimated factor loadings and their standard errors, t values, and p values under the assumption of normal distribution, in addition to the 95% and 99% BCCIs, are reported in Table 4. The factor loading of the first indicator for each latent variable was fixed to unity to provide identification. The results of the numerical example showed that the 99% BCCIs for all of the factor loadings did not contain zero. Hence, we conclude that all of the factor loadings were statistically significant.

OPEN IN VIEWER

Table 3.

Numerical Example: Fit Indices for Baseline Model Without Latent Interaction.

Chi-square test of model fit
Value	116.8066
Degrees of freedom	99
p value	0.1069
Root mean square error of approximation (RMSEA)
Estimate	0.0268
90% confidence interval	0.0000	0.445
Probability (RMSEA = 0.05)	0.9879
Comparative Fit Index	0.9862
Tucker-Lewis Index	0.9833
Standardized root mean square residual	0.0418
Akaike Information Criterion	12,848.182
Bayesian Information Criterion	13,034.819

Note: The results of a model with a latent interaction should only be interpreted if the model without a latent interaction has adequate fit.

OPEN IN VIEWER

Table 4.

Numerical Example: Factor Loadings and Bias-Corrected Confidence Intervals.

Factor Loadings for X by
	Estimate	Std Err	t Value	Norm p	95% BCLL	95% BCUL	99% BCLL	99% BCUL
X2	1.1278	.1026	10.9899	.0000	.9542	1.3766	.8834	1.4363
X3	1.0163	.0832	12.2125	.0000	.8506	1.1955	.8080	1.2498
X4	1.0685	.0898	11.9040	.0000	.8821	1.2632	.8322	1.3233
Factor Loadings for W by
	Estimate	Std Err	t Value	Norm p	95% BCLL	95% BCUL	99% BCLL	99% BCUL
W2	.8227	.0909	9.0515	.0000	.6522	1.0201	.5957	1.0853
W3	.9743	.1000	9.7454	.0000	.7818	1.1818	.7527	1.2549
W4	1.0528	.0958	10.9914	.0000	.8748	1.2430	.8259	1.3064
Factor Loadings for M by
	Estimate	Std Err	t value	Norm p	95% BCLL	95% BCUL	99% BCLL	99% BCUL
M2	1.0665	.1151	9.2696	.0000	.8609	1.3125	.8093	1.4165
M3	1.0991	.1291	8.5158	.0000	.8585	1.3679	.7729	1.4737
M4	1.1180	.1333	8.3868	.0000	.8845	1.4243	.8037	1.5136
Factor Loadings for Y by
	Estimate	Std Err	t value	Norm p	95% BCLL	95% BCUL	99% BCLL	99% BCUL
Y2	.9304	.0830	11.2056	.0000	.7798	1.1278	.7417	1.1963
Y3	.9392	.0929	10.1155	.0000	.7449	1.1368	.7011	1.2207
Y4	.9198	.0968	9.5068	.0000	.7425	1.1213	.6752	1.2003

Note: Std Err = standard error of estimate; Norm p = p value under the assumption of normal distribution; BCLL = lower limit of bias-corrected bootstrap confidence interval; BCUL = upper limit of bias-corrected bootstrap confidence interval.

In the second step, the model in Figure 1b with latent interaction XW and path a₃ is estimated (Klein & Moosbrugger, 2000; Muthén, 2012). The Mplus syntax used for the second step is presented in Appendix 1. ² Under the MODEL command, the XWITH option is short for “multiplied with” and is used in conjunction with the | symbol to name and define a product term of continuous latent variables (Muthén & Muthén, 1998-2012). For instance, “XW | X XWITH W” defines XW as the product of X and W. In the ANALYSIS command, the statement “TYPE = RANDOM” requests an estimation of random intercepts and slopes, and it must be stated to define interactions among latent variables. Because the interactions among latent variables require numerical integration, the statement “ALGORITHM = INTEGRATION” is also included. Then, the MODEL CONSTRAINT command and the NEW option are used to create a new variable for the index of moderated mediation. In this demonstration, the index of moderated mediation a₃b is given the label INDEXMM. Finally, the BOOTSTRAP option under the ANALYSIS command and the CINTERVAL(BCBOOTSTRAP) option under the OUTPUT command request 1,000 bootstrap samples and the BC bootstrap confidence intervals for the estimates. Bootstrapping with latent interactions has only been made available beginning with Mplus 7.3, which was released in May 2014.

Table 5 presents the estimates, standard errors, t values, p values under a normal distribution assumption, and the lower and upper limits of the 95% and 99% BCCIs for the model parameters. The results showed that regression coefficient a₃ of the latent interaction term on the mediator was 0.4621, 95% CI [0.3343, 0.6755], and regression coefficient b of the mediator on the dependent variable was 0.7156, 95% CI [0.4659, 0.9758]. The estimated index of moderated mediation a₃b was 0.3307, 95% CI [0.2155, 0.5091]. The regression coefficient c’, of the independent variable on the dependent variable was 0.1981, 95% CI [0.0375, 0.3450].

OPEN IN VIEWER

Table 5.

Numerical Example: Estimated Effects and Bias-Corrected Confidence Intervals.

	Effect	Std Err	t Value	Norm p	95% BCLL	95% BCUL	99% BCLL	99% BCUL
X on M (a1 )	.1061	.0655	1.6209	.1050	−.0086	.2611	−.0475	.3328
W on M (a2 )	.1811	.0702	2.5805	.0099	.0528	.3319	.0068	.3964
XW on M (a3 )	.4621	.0777	5.9490	.0000	.3343	.6755	.2798	.7274
X on Y (c’)	.1981	.0714	2.7762	.0055	.0375	.3450	.0006	.3984
M on Y (b)	.7156	.1243	5.7559	.0000	.4659	.9758	.3916	1.0794
Index MM (a3b)	.3307	.0678	4.8802	.0000	.2155	.5091	.1827	.5682

Note: Std Err = estimated standard error; norm p = p value under the assumption of normal distribution; BCLL = lower limit of bias-corrected bootstrap confidence interval; BCUL = upper limit of bias-corrected bootstrap confidence interval.

In the third step, following Preacher et al. (2007), Hayes and Preacher (2013), and Hayes (in press), we estimate the moderated mediation effect of X on Y at various levels of moderator W to probe the moderated mediation effect. The moderated mediation effect of X on Y for the model in Figure 1 is (a₁ + a₃ W)b. Although the moderator W is a latent variable without any defined value, the variance of W is estimated and the latent mean of W is set to zero as the default in Mplus. Hence, we can substitute five levels of W in the equation that defines the moderated mediation effect of X on Y, as follows: (a) mean plus two standard deviations, (b) mean plus one standard deviation, (c) mean, (d) mean minus one standard deviation, and (e) mean minus two standard deviations. The BCCI for each parameter is also estimated. The moderated mediation effects at various levels of the moderator are reported in Table 6. The results showed that when W was at one standard deviation above the mean (+1 sd_w), the mediating effect of X on Y through M was positive (0.4191, 95% CI [0.2654, 0.6529]). When W as at one standard deviation below the mean (–1 sd_w), the mediating effect of X on Y through M was negative (–0.2671, 95% CI [–0.4494, –0.1239]). These results imply that moderator W may change the direction of the mediating effect of X on Y through M.

OPEN IN VIEWER

Table 6.

Numerical Example: Moderated Mediation Effect of X on Y at Various Values of Moderator W.

	W	Effect	SE (Boot)	95% BCLL	95% BCUL	99% BCLL	99% BCUL
+2 sdw	2.0751	.7621	.1696	.4868	1.1418	0.4189	1.3790
+1 sdw	1.0375	.4191	.0993	.2654	0.6529	0.2148	0.7736
Mean W	0.0000	.0760	.0500	−.0051	0.1919	−0.0380	0.2435
−1 sdw	−1.0375	−.2671	.0817	−.4494	−0.1239	−0.5090	−0.0942
−2 sdw	−2.0751	−.6102	.1496	−.9400	−0.3565	−1.0426	−0.3000

Note: +2 sd_w = two standard deviations above mean; +1 sd_w = one standard deviation above mean; –1 sd_w = one standard deviation below mean; –2 sd_w = two standard deviations below mean; W = value of the moderator W; Effect = estimated moderated mediation effect; SE (Boot) = bootstrap standard error; BCLL = lower limit of bias-corrected bootstrap confidence interval; BCUL = upper limit of bias-corrected bootstrap confidence interval.

Discussion

Given the increasing popularity of moderated mediation models in organizational and psychological studies, the importance of identifying an accurate and user-friendly analytic strategy has increased. Because CIs of estimates provide more useful information than do significance tests, reporting CIs has become a requirement for publication in many organizational and psychological journals. Hence, in addition to examining the accuracy of the estimates and the statistical power for testing the statistical significance of the direct and moderated mediation effects, the current study examined the accuracy of CIs produced under three approaches.

The Regression Approach Versus the Latent Moderated Structural Equations Approach

Although regression has been the most frequently used approach for testing moderated mediation, the existence of measurement errors may lead to biased estimates of moderated mediation effects, which in turn may result in invalid conclusions for management theory and for implementing management practices. These biased estimates also lead to problems in meta-analyses. As the bias in the effect size of moderated mediation is replicated in all empirical studies that use the regression approach, the meta-analytic findings based on these inaccurate estimates of moderated mediation effects are also likely to be biased. In multiple regression with more than one independent variable, measurement errors may decrease or increase or even change the sign of a partial relationship (Cohen & Cohen, 1983). Similarly, the estimated parameters in a mediation model can be larger or smaller than the true values (Ledgerwood & Schrout, 2011). Our simulation results reflect these biases. Even when reliability reached 0.80, which is considered adequate in social science studies, the regression approach continued to produce estimates of parameters a₁ , a₂ , and b that were smaller than the true values by 5% to 20%. Both larger effect size and lower reliability result in larger attenuation. The estimates of the direct effect c’ from X to Y, however, were slightly larger than the true value. Because product term XW presented poorer reliability, the inaccuracy of the regression approach was particularly serious in relation to the estimates of the moderation effect a₃ , which was underestimated by 30% when reliability was 0.8 and by 50% when reliability was 0.67. The reliability ρ _xw of the product term XW is a decreasing function of reliabilities ρ _xx and ρ _ww and the correlation between X and W (Busemeyer & Jones, 1983). Busemeyer and Jones (1983) showed that ρ _xw is typically lower than ρ _xx and ρ _ww and reaches its minimum value of ρ _xx ρ _ww when X and W are not correlated. However, the average estimates produced by the LMS approach were all slightly larger than the true values under all conditions. With respect to the simulated data sets with reliability at 0.8, the relative bias in estimates were mostly below 2%, with the largest overestimation at 5.7% for c’ when N = 100. Although lower reliability enlarged the relative bias, the relative biases of estimates using the LMS approach were all within the acceptable range of 10% (Cham, West, Ma, & Aiken, 2012). These results demonstrate that the LMS approach is effective in correcting for measurement errors, even under marginally acceptable reliability conditions.

While the regression approach consistently produced a smaller standard error than the LMS approach, the absolute bias produced by the regression approach is not necessarily smaller than that produced by the LMS approach. Our simulation results show that the following three factors affect the magnitude of absolute bias: effect size, sample size, and reliability. In general, larger absolute bias is associated with larger effect size and smaller sample size. However, the impact of effect size on absolute bias is greater for the regression approach, whereas the impact of sample size on absolute bias is greater for the LMS approach. Thus, the regression approach resulted in smaller absolute bias than the LMS approach when the effect size was small and larger absolute bias when the effect size was substantial. For a moderate effect size, the regression approach resulted in smaller absolute bias with small sample sizes but larger absolute bias with large samples. The same pattern of absolute bias was observed for the low reliability samples. The absolute bias of the regression approach was smaller than that of the LMS approach when the sample size was small but was larger than that of the LMS approach when the sample size was large. Although lower reliability has a slight effect on the estimates but a substantial effect on the standard errors produced by the LMS approach, it has a substantial effect on the estimates but little effect on the standard errors produced by the regression approach.

What has drawn relatively less attention is that regression typically generates biased CIs, in other words, CIs that do not include the true values. This result can be attributed to the small standard errors and underestimated parameters. Following Collins, Schafer, and Kam (2001) and Cham et al. (2012), in the current study, it was considered acceptable if the coverage rate of the 95% CIs exceeded 90%. Our simulation results showed that all the CIs using the LMS approach had acceptable coverage rates and were near the theoretical 95%. However, the CI coverage rates using the regression approach were only acceptable for small direct effects (a₁ = 0.2 and c’ = 0.14). The coverage rates for medium and large direct effects (a₂ = 0.4 and b = 0.59) and for small and medium interaction effects (a₃ = 0.2 and 0.4) were all below 90%. The CI coverage rates for Index MM were worse than the interaction effects. The coverage rate was adversely affected by a larger effect size, larger sample size, and lower reliability because a larger effect size and lower reliability resulted in a more substantial underestimation of parameters and a larger sample size resulted in smaller standard errors using the regression approach. Taken together, the CI is narrower, and the center of the CI is farther from the true value. Hence, the CI coverage rate deteriorates.

Table 1 also reveals the power of all three approaches to test moderated mediation. In general, the regression approach has greater power than the LMS approach under all conditions. This result is consistent with previous simulations in which regression produced more powerful tests than SEM (Klein & Muthén, 2007; Ledgerwood & Shrout, 2011). All three approaches had adequate power for testing an interaction effect with effect size at 0.4, even when N = 100. When the effect size of a₃ was 0.2, the power deteriorated rapidly with the smaller sample size and lower reliability. None of the approaches provided adequate power for testing either the interaction effect or the Index MM when the sample size was 100 or 200. It is interesting to note that although its power for testing main effects and interaction effects is similar to that of the regression approach, the LMS approach provides more power for testing interaction effects than main effects with the same effect sizes. Regarding Type I error, both the regression and LMS approaches show accurate Type I error rates.

Reliability-Corrected Single-Indicator LMS Approach

An appealing finding of the simulation is the similarity in the results of the RCSLMS approach and the LMS approach and the slightly greater power of the RCSLMS approach. One limitation of the LMS approach is that when testing a complicated model with many indicators and when the sample size is small, the LMS approach may result in poor model fit or the model may not converge. However, because the RCSLMS approach combines the indicators of each construct into one single-indicator, it has a greater chance of producing converged solutions even when the model is complicated and the sample size is small. Despite advantages over the LMS approach, the current results of the RCSLMS approach should be generalized with caution because the conditions in the simulation were all favorable to the RCSLMS approach. First, parallel indicators with equal factor loadings and equal unique variance were simulated. We believe that tau-equivalent indicators with only equal factor loadings and congeneric indicators with both unequal factor loadings and unequal unique variance may work against the RCSLMS approach because simple average scores may not be good representations of the latent constructs under such conditions. In fact, the regression approach may face similar problems. Second, while the uniqueness term for each indicator is the sum of the random measurement error and the specific error, only random measurement error was included in the simulation. If the indicator specific error is not zero, then the random measurement error is an underestimation of the uniqueness and the correction will not be adequate. Future studies are needed to investigate how various factors, such as unequal factor loadings, unequal unique variance, and the existence of specific errors may affect the performance of the RCSLMS approach in testing moderated mediation. Although the RCSLMS approach may be a good alternative under the most favorable conditions, researchers should use the LMS approach as much as possible because the aforementioned factors will not affect the accuracy of estimates or the CIs of the LMS approach. Readers are referred to Petrescu (2013) for a review on using single indicators in SEM and Bandalos and Finney (2001) and Little, Rhemtulla, Gibson, and Schoemann (2013) for reviews on item parceling issues in SEM.

Forms of Moderated Mediation

When the index of moderated mediation is significant, we can refer to the results in Table 6, which provide a better picture of how the moderator W affects the mediating effect from X to Y through M. This is similar to examining the simple main effects when the interaction of a factorial design is significant in an ANOVA. One form of moderated mediation is where the mediating effects at various levels of the moderator are statistically significant but in opposite directions, as reported in Table 6. A second form of moderated mediation is where the mediating effects at various levels of the moderator are all statistically significant but in the same direction, which makes them stronger at some moderator levels and weaker at others. In this case, the moderator alters the strength of the mediating effect in the relationship between the independent and dependent variables. A third form of moderated mediation is when the mediating effects are statistically significant at some moderator levels and statistically nonsignificant at others. This form of moderated mediation assists researchers in identifying specific critical boundary assumptions and theoretical constraints, which are vital because a theory can be properly utilized only when the theory’s boundary conditions are understood (Bacharach, 1989). Identifying this form of moderated mediation is particularly useful in the interaction analysis suggested by Cortina and Folger (1998) for studies that attempt to retain the null hypothesis. In the mediation study context, if the objective is to find support for the nonexistence of a mediating effect (a null mediation hypothesis), then it should also be established that the mediating effect can be detected at a certain level of a moderator. Evidence of sufficient power to test for the mediating effects should also be provided, and the results should show that the index of moderated mediation is statistically significant and that the confidence interval for the mediating effect at one moderator level (at least) includes zero (statistically nonsignificant) while at another moderator level (at least) does not include zero (statistically significant).

Limitations

Although the current discussion is based on the first stage moderation model, in which moderator W affects the relationship between independent variable X and mediator M (Edwards & Lambert, 2007), the procedures described herein can be easily extended to other moderated mediation models, in which the moderator also affects the direct effect from X to Y (first stage and direct effect moderation model), second stage (and direct effect) moderation model, and models with multiple serial mediators. Hayes (in press) provided the equations for an index of moderated mediation for different models. One limitation of the index of moderated mediation is that when the moderator affects both the first stage and second stage mediation processes, although the CIs of all the path coefficients can be estimated with the LMS approach, the CI of the index of moderated mediation cannot be constructed by bootstrapping because it is a nonlinear function of W (Hayes, in press). However, it does not affect Step 3 in our proposed three-step Mplus procedures, and the results of this step will play a more important role in making inferences about the moderated mediation effects (Edwards & Lambert, 2007; Hayes, in press).

Although we have explicitly discussed the problem of measurement errors in examining moderated mediation effects, there are other assumptions for testing a mediating effect using regression, such as homoscedasticity and independence of errors. There is also the assumption of no missing variable that is correlated with both the mediator and dependent variable, such that the disturbances of the mediator and the dependent variable are not correlated (Shaver, 2005). Other assumptions that may affect the exogeneity of the independent variable when testing mediation and moderated mediation models include equivalent intact groups, no simultaneous causation, and the absence of common-source or common-method variance. Interested readers are referred to the list of 14 threats to validity for making causal claims by Antonakis et al. (2010). Although SEM can include correlated errors and clustered errors in the model, additional studies are required to examine the impact of violating other assumptions of the estimates and interpretations when testing moderated mediation effects with LMS.

An additional limitation of this study is that despite the inclusion of different levels of effect size, levels of random measurement error, and sample sizes in the simulations, this study did not examine the effect of the number of indicators, unequal factor loadings and unequal unique variances across items, multiple mediators, or non-normal observed variables on the accuracy of the parameter estimates and CIs. Nevertheless, it is speculated that unequal factor loadings and unequal random measurement errors tend to have a considerable impact on the regression and the RCSLMS approaches, which require the averaging of indicator scores to represent the constructs. However, the impact on the LMS approach is relatively small. Comparing the moderation effect a₃ and Index MM (a₃b) results in Table 1, we suspect that the inclusion of more mediators in the moderated mediation model will have little impact on the CI coverage rates when using the LMS and RCSLMS approaches but that the CI coverage rate for Index MM will be further decreased when using the regression approach. Although the impact of non-normal observed variables on the LMS approach is not examined in this study, previous simulation studies have found that the LMS approach provides unbiased estimates when the observed variables do not severely deviate from normality, such as when distributions are symmetric and when kurtosis is less than 1 (Cham et al., 2012; Klein & Moosbrugger, 2000). Because previous simulations have found that 2SLS also provides unbiased estimators in models with interactions, future simulation studies are needed to examine the behavior of the CIs produced by the 2SLS method under various conditions.

Conclusion

An increasing number of organizational and psychological researchers are incorporating moderated mediation effects (conditional indirect effects) into their theoretical models, and most empirical studies currently use the observed variable regression approach in their data analyses. Our simulation has demonstrated that despite the greater power provided by the regression approach, this approach underestimates the moderated mediation effects and produces inaccurate confidence intervals. Unfortunately, increasing the sample size does not make the biased estimates and confidence intervals from the regression approach more accurate. In other words, it is likely that the regression approach may conclude statistically significant moderation or moderated mediation effects even when the confidence interval of the estimated effects does not include the true value. Hence, the results of previous studies should be interpreted with caution. Researchers’ previous focus on conducting significance tests rather than reporting confidence intervals may explain why the bias in confidence intervals produced by regression analysis has been overlooked. Therefore, we suggest that future empirical studies on moderated mediation use the LMS approach, which provides accurate estimates and confidence intervals, although a larger sample size may be needed to achieve adequate power for testing small effect sizes.