Assessing Common-Metric Effect Sizes to Refine Mediation Models

Mediation analysis sheds light on whether and how much an antecedent or predictor variable X indirectly affects an outcome or criterion variable Y via a transmitter or mediator variable M (MacKinnon et al., 2007; Muller et al., 2005). Understanding this type of X → M → Y process that mediates between two variables X and Y is necessary to increase our understanding of organizational phenomena beyond simple bivariate relationships (MacKinnon et al., 2012), perhaps explaining why mediation studies account for a large proportion of the articles published in some journals (Rucker et al., 2011).

Owing to the practical constraints inherent in conducting research in organizations, mediation studies typically do not meet the necessary design conditions to establish causality like temporal precedence of cause over effect and the employment of double randomized experiments of the X → M and the M → Y relations (Eden et al., 2015; Spector & Meier, 2014; Stone-Romero & Rosopa, 2008, 2011). For this reason, many organizational studies infer mediation from a series of statistical significance tests expressed in a set of equations that MacKinnon et al. (2007) termed statistical mediation. The procedures employed to test the reliability of statistical mediation have been refined a great deal since Sobel (1982) proposed his relatively simple z-test, thereby providing superior protection against Type I and Type II errors. Refinements of these procedures include resampling (MacKinnon et al., 2004), bootstrapping (Preacher & Hayes, 2008; Preacher et al., 2007), and Monte Carlo simulated confidence intervals (Preacher & Selig, 2012).

The enhanced protection against Type I and Type II errors afforded by advances in significance tests of statistical mediation does not compensate for research design shortcomings like the absence of randomized groups. As Carver (1978) pointed out long ago, “too often statistical significance covers up an inferior research design” (p. 386). Statistical significance does not estimate the probability that the mediation hypothesis is true. Therefore, even if one obtains support for a specific mediation model and proceeds to reject the null hypothesis, the logic of significance testing is such that researchers should not rule out alternate explanations of the cause-and-effect inferences implied in an X → M → Y mediation chain.

Calling for a moratorium on mediation research in organizations unless mediation research designs meet the rigorous conditions demanded by cause-and-effect inferences would be irresponsible, because such a call would ignore the practical constraints of organizational research. Indeed, even though failing to acknowledge rival explanations does not fully answer recent calls for responsible research in organizations (Tsui, 2022), it is a milder error of omission that cannot be compared to errors of commission as severe as reporting only statistically significant findings (p-hacking) and formulating hypotheses after the results are known (HARKing) (Tourish, 2019). Instead, we argue that further insights and theoretical refinements of mediation models are possible if organizational researchers dare to look beyond the categorical findings (i.e., reject or fail to reject the null hypothesis) provided by significance tests of statistical mediation. Specifically, we maintain that researchers should closely scrutinize the entirety of residual variance not accounted for in the mediation model through a set of mediation-related effect sizes which, considered together, suggest gaps and potentially meaningful theoretical refinements to a mediation model.

An effect size typically expresses the strength of the relationship or the strength of the difference between two or more variables, thereby facilitating not only the integration of research findings across studies, but also an assessment of their practical significance (Cohen, 1994; Paterson et al., 2016). When effect sizes take the form of standardized descriptive statistics like a correlation coefficient or a standardized mean difference, they become common metrics comparable among studies (Glass et al., 1981; Hedges & Olkin, 1985; Rosenthal, 1991; Schmidt & Hunter, 2014). In the context of mediation, a common metric effect size not only makes results comparable among mediation studies of similar X, M, and Y variables, but more important for our purposes, offers valuable insights into potential refinements of a mediation model that are not necessarily obvious in the categorical conclusions yielded by statistical significance tests of mediation.

Miočević et al. (2018) noted that non-methodological, substantive mediation studies seldom report mediation effect sizes. This practice not only runs counter to the recommendations in favor of reporting effect sizes and their confidence intervals issued by professional associations such as the American Psychological Association (Pek & Flora, 2018), but also deprives researchers of valuable information that might suggest substantive refinements of mediation models. The importance of mediation effect sizes has been acknowledged in prevention and intervention research, where mediators are seen as key mechanisms of behavior change (O’Rourke & MacKinnon, 2018). One would think that a similar interest in understanding the magnitude of the process through which an organizational intervention brings the desired behavior change should exist in organizational research that, surprisingly, has largely ignored mediation effect sizes.

We argue in favor of three effect sizes that, considered together, shed light on potential gaps in one or the two stages of a mediation chain. O’Rourke and MacKinnon (2018) termed these two stages action versus conceptual theory. Action theory underlies the first X → M stage of the mediation model, which shows the extent to which the independent variable effectively changes the mediator. Conceptual theory refers to the second M → Y stage, which explains the extent to which the mediator influences the outcome or dependent variable. However, analyzing mediation essentially requires a joint examination of its two stages in what Fairchild et al. (2009) termed the overall mediation or R²_med effect, which represents the proportion of variance in Y jointly accounted for by M and X (Figure 1). We argue that examining the residual variance not captured in this overall mediation effect is key to pinpointing gaps and consequent potential theoretical refinement to the mediation model. To do so, we advocate two additional common metric effect sizes that are essentially squared partial correlations (i.e., R²_xy.m and R²_my.x) revealing the amount of residual variance in the action and conceptual theories of mediation that is not accounted for in the overall mediation effect (Figure 1).

OPEN IN VIEWER

Figure 1.

Venn diagram of overall mediation, residual X effect, and residual M effect.

To illustrate the limitations inherent in assessing mediation solely from the point of view of statistical significance, consider the example of Liu et al.'s (2010) findings. An examination of the residual variance captured by R²_xy.m and shared between X and Y after partialing out M revealed that employees’ trust in their leader acted as solely a partial mediator of the effect of transformational leadership on job satisfaction, because transformational leadership also had a residual direct effect on job satisfaction. Even though researchers typically stop at concluding that the direct effect co-exists with the indirect or mediation effect, the presence of this direct effect and other non-trivial residual variance in the mediation model raises a number of substantive questions that statistical significance tests alone cannot answer. We maintain that an examination of not only the residual variance shared between X and Y after partialing out M (i.e., R²_xy.m), but also the residual variance shared between M and Y after partialing out X (i.e., R²_my.x) should best reveal whether searches for alternate mediators and alternate antecedent variables are warranted. Just like a non-trivial R²_xy.m reveals potential for alternate mediators to improve the mediation model, which indeed explains why Liu et al. (2010) explored self-efficacy as an alternate mediator, a non-trivial R²_my.x calls for a search for alternate X antecedent variables, as we will explain.

In summary, we aim to illustrate how a set of standardized (hereafter referred to as common-metric) effect sizes based on variance proportions might fruitfully inform theoretical refinements of mediation models in organizational research. Our goal, however, is two-fold. First, we provide guidance to compute these common-metric mediation effect sizes using simple bivariate correlations among X, M, and Y, which are reported in virtually every mediation study. Second, we provide practical examples and normative data illustrating how the computation of the common-metric mediation effect sizes advocated here together with their comparison to those of other mediation studies using similar X, M, and Y variables might suggest theoretical refinements of the hypothesized mediation model.

Mediation Effect Sizes

The equations that form what MacKinnon et al. (2007) termed statistical mediation are as follows:

Y = i 1 + c X + e 1

(1)

Y = i 2 + c ′ X + b M + e 2

(2)

M = i 3 + a X + e 3

(3)

The magnitude of the ab mediation effect, however, depends on the specific scales employed to gauge the X, M, and Y variables. As a result, the ab effect size does not conform to the notion of a standardized or common metric that communicates the size of the mediation effect using a common language across studies. In other words, even when the ab mediation effects are indeed reported, readers have difficulties figuring out what numbers represent a small, a medium, or a large mediation effect. Similarly, the sign of the ab effect can be at times cumbersome to interpret, because it depends on how the X, M, and Y variables are scaled. For instance, a negative ab effect might very well suggest an ameliorating effect of a particular leadership style on work-related stress symptoms transmitted by the mediator, whereas a positive sign might in fact indicate a negative impact of said leadership style that elevates work tension. Given their unwieldly interpretability, it is not surprising that a majority of non-methodological, substantive mediation studies fail to report mediation effect sizes as denounced by Miočević et al. (2018). Next, we elaborate on our proposed approach to compute presumably more intuitive and easily interpretable common-metric mediation effect sizes.

Computing Common-Metric Mediation Effect Sizes

Our approach does not intend to replace any of the extant approaches to testing for statistical mediation either in a single study or in meta-analyses that cumulate mediation results across studies (e.g., Kammeyer-Mueller et al., 2013). Instead, we focus on illustrating how variance-proportion, common-metric mediation effect sizes provide insights that can fruitfully guide potential theoretical refinements of a mediation model that are not immediately obvious in significance tests of statistical mediation.

Our primary aim is making a compelling case for the need to go beyond statistical significance tests of mediation in organizational research by computing and assessing common metric mediation effect sizes (Cohen, 1994). Indeed, examining these common metric effect sizes provides organizational researchers with insights that should potentially refine their theories of which among a set of possible mediators is most relevant for transmitting the effects of an independent variable to a dependent variable. Similarly, these common metric effect sizes should help researchers choose those antecedents that are most capable of transmitting their effects through a given mediator among a list of potential antecedents.

Unlike the path coefficients typically employed in the computation of the scale-dependent ab mediation effect size (MacKinnon et al., 2007), the variance proportion-based mediation effect sizes advocated here stay within an easily interpretable common metric that makes them comparable to those in similar studies. Indeed, the ab effect does not clearly convey the magnitude of the mediation unless the X and Y variable scales are expressed in intuitive units of measurement such as ounces or dollars (Miočević et al., 2018), which is hardly the case in organizational research. In contrast, variance proportion-based mediation effect sizes are scale-independent proportions that range from 0.00 to 1.00, which turns them into an easily understandable common metric.

Variance Proportion-Based Mediation Effect Sizes

Owing to its intuitive and scale-independent nature, we favor assessing the mediation effect size using the proportion of variance in Y explained by the indirect mediation effect, which Fairchild et al. (2009) named the overall mediation or R²_med effect. As mentioned, this measure captures the proportion of variance in Y accounted for jointly by M and X (Figure 1). Fairchild et al.'s (2009) initial formulation of this proportion, however, is technically not a proportion, because it can return nonzero effect sizes for mediation when the indirect effect is in fact zero (Lachowicz et al., 2018). Lachowicz et al. (2018) reformulated the overall mediation effect into the upsilon or υ parameter, which corrects for the spurious correlation induced by the ordering in which variables enter the mediation model. The computation of the υ parameter, however, requires a standardized regression coefficient that not every primary study reports. We propose a simpler formulation of R²_med that is admittedly less precise than the υ parameter, and yet it constitutes a practical, easy to-compute estimate of the proportion of variance in Y accounted for jointly by M and X. Our approach subtracts a squared partial correlation (e.g., R²_my.x) from a squared bivariate correlation (e.g., R²_my). As a result, non-zero R²_med values are unlikely if the true indirect effect is zero. A practical advantage of our approach to computing R²_med is that it requires only the bivariate correlations between the X, M, and Y variables, which virtually every mediation study reports.

Together with the R²_med or overall mediation effect, we argue for two additional effect sizes that capture the residual variance left out of the overall mediation effect. These additional effect sizes have theoretical and practical implications concerning potential gaps in the first and second links of a mediation chain, which define what O’Rourke and MacKinnon (2018) referred to as action and conceptual theory, respectively. As mentioned earlier, the two additional effect sizes proposed here are essentially squared partial correlations (i.e., R²_xy.m and R²_my.x) that capture residual variance not included in the overall R²_med mediation effect. As such, these squared partial correlations pinpoint potential gaps in action theory, conceptual theory, or both.

Turning to the computation of the effect sizes that we advocate, R²_med might be estimated using a variation of Equation 7 in Fairchild et al. (2009),

R m e d 2 = ( R Y , X M 2 – r X Y . M 2 ) – r M Y . X 2

(4)

A potential obstacle to the estimation of R²_med using Equation 4 above is that R²_Y,_XM is not reported in many mediation studies. However, our alternate approach to estimating R²_med is suggested by Figure 1. That is, both X and M explain common and unique variance in Y and, therefore, one might estimate R²_med by subtracting the proportion of variance in Y that is uniquely explained by X (but not explained by M) from the total proportion of variance in Y that is explained by X,

R m e d 2 = r X Y 2 – r X Y . M 2

(5)

R m e d 2 = r M Y 2 – r M Y . X 2

(6)

A potential first explanation for a negligible R²_med effect is a partial failure in both action and conceptual theories, because M simply fails to transmit the full effect of X onto Y. This situation is commonly referred to as partial mediation, and it takes place when the R²_med effect co-exists with a residual and yet non-trivial direct effect of X on Y after partialing out the effects of M. As suggested by others (Fairchild et al., 2009; Rucker et al., 2011), we propose to estimate this residual direct effect through the squared semi-partial correlation between X and Y after partialing the influence of M,

R x y . m 2 = r X Y . M 2

(7)

OPEN IN VIEWER

Figure 2.

Venn diagram of extreme example of residual X or R²_xy.m effect.

Although largely overlooked, still another important common-metric effect size might pinpoint a gap in specifically the action theory underlying a mediation model. We are referring to R²_my.x or residual M → Y effect after partialing the effect of X. Figure 3 portrays an extreme example of this type of residual R²_my.x effect. R²_my.x is important because it captures the improvement in the prediction of Y afforded by adding M to the X → Y bivariate model thereby forming the X → M → Y mediation chain. This incremental prediction of Y afforded by adding M to the model helps researchers justify the position of M as closer to Y in the causal chain than X is. However, the combination of a large R²_my.x and a small R²_med suggests the need to revise the choice of X or antecedent variable in the mediation model, because M appears to have a sizable residual effect on Y that proceeds independently from X. Therefore, a non-trivial R²_my.x accompanied by a trivial R²_med signals a serious failure in the X → M stage or action theory of the mediation model, thereby calling for a theory-driven search for alternate X variables closely associated with M that better capitalizes on the currently large residual effect of M on Y.

OPEN IN VIEWER

Figure 3.

Venn diagram of extreme example of residual M or R²_my.x effect.

We estimated this residual effect of the mediator through the squared semi-partial correlation between M and Y after controlling for the influence of X as follows:

R m y . x 2 = r M Y . X 2

(8)

OPEN IN VIEWER

Figure 4.

Recommended flowchart to examine mediation effect sizes.

Once the steps of the process for a comprehensive examination of mediation-related effect sizes outlined in Figure 4 are completed, and particularly in cases where the final decision is to identify alternative Xs, alternative Ms, or both, researchers should consider whether their results suggest theoretically and/or practically significant effect sizes. This determination should not proceed according to a set of monolithic cut-offs, but relative to theory and the entire body of extant findings in the relevant topic area. A comparison of one's study findings with those of prior studies including similar X, M, and Y variables should play an important role in this consideration. Such a comparison requires not only an identification of similar studies, but also the computation of proportion-based common metric effects in those studies using the formulas we provided earlier.

In the next section, we illustrate every step of the process outlined in Figure 4 using examples of two X variables (i.e., abusive and transformational leadership) and an M variable (i.e., social exchange quality) that might transmit the indirect effects of the two Xs variables onto employee outcomes. We trust that this practical illustration will assist those researchers who contemplate theoretical refinements of their mediation model using our set of guidelines.

Practical Illustration

To illustrate how a combined examination of the three aforementioned mediation effect sizes can help us refine specific mediation models, we chose the compound construct termed social exchange quality, which is defined as “the perception and expectation that the organization does and will provide support and fair treatment to the employee” (Colquitt et al., 2013, p. 201). According to Newman et al. (2016), social exchange quality consists of the following constituent constructs: organizational justice, support, and employee trust. Newman et al. (2016) identified social exchange quality as one of the most important compound constructs in the OB and HR literatures, because the three constituent constructs that form it (i.e., justice, support, and trust) have been among the most widely studied over the past 30 years. A Web of Science search described later in our method section also revealed that these three constituent constructs of social exchange quality (i.e., trust, perceived organizational support, and organizational justice) are among the most commonly examined mediators in organizational research. This trend is not altogether surprising because positive social exchanges trigger perceptions of support and fair treatment, which are seemingly key intermediate states likely to transmit a positive effect on employee attitudinal and behavioral outcomes. The employment of a compound construct as a mediator expanded the scope of mediators under consideration and, therefore, our ability to understand the types of X variables best mediated by the compound constituent constructs.

We provide practical illustrations of our approach in the form of two substantive examples focusing on the indirect effects of two X variables, namely abusive and transformational leadership transmitted through the M variable termed social exchange quality. We examined these indirect effects across all types of employee outcomes for the sake of simplicity, but obviously, researchers might choose to focus on specific outcomes being those attitudinal like organizational commitment or behavioral like organizational citizenship behavior. We focused on abusive and transformational leadership as the X variables not only because they represent opposite types of leadership, but also because our review of the social exchange theory literature revealed that they are among the most common X variables paired with one or more of the constituent constructs of social exchange quality when the latter act as mediators. We define abusive leadership as employees’ perceptions of the extent to which supervisors engage in the sustained display of hostile verbal and nonverbal behaviors, excluding physical contact (Tepper, 2000). We define transformational leadership as the extent to which leaders provide intellectual stimulation, individualized consideration, inspirational motivation, and idealized influence to followers (Bass, 1999).

Method

Results

A percentile table for the three effect sizes introduced here is included in Table 1. We computed these percentiles based on 887 to 1,037 effect sizes identified in our literature review in which at least one of the constituent constructs of social exchange quality was the mediator. Although this distribution begins to provide normative data against which researchers might benchmark the magnitude of their own effect sizes, these effect sizes correspond to a small number of variables and, therefore, they should not be blindly generalized across all potential variables. Nevertheless, the uncorrected and corrected median size (i.e., 50th percentile) of the R²_med or overall mediation effect size in Table 1 (i.e., r = .24 and .28, respectively) were very close to the median effect sizes in HR/OB studies reported by Paterson et al. (2016) (i.e., r = .227 and .278, respectively). Thus, our distribution figures support Paterson et al.’s (2016) argument that Cohen's (1994) breakpoints for small, medium, and large effect sizes overestimate those in HR/OB studies. Even though it is likely that other individual-level mediators in the organizational sciences might follow a similar distribution, the generalization of the effect size distribution portrayed in Table 1 warrants continued research on other mediators and antecedent variables.

OPEN IN VIEWER

Table 1.

Effect Size Distribution Percentiles.

Percentile	R2med (zmed)	R2med (zmed) a	R2xy.m (zxy.m)	R2xy.m (zxy.m) a	R2my.x (zmv)	R2my.x (zmv) a
1	.00 (.02)	.00 (.02)	.00 (.00)	.00 (.00)	.00 (.01)	.00 (.01)
10	.01 (.08)	.01 (.10)	.00 (.02)	.00 (.02)	.01 (.08)	.01 (.08)
20	.02 (.13)	.03 (.16)	.01 (.04)	.01 (.04)	.03 (.14)	.03 (.14)
30	.03 (.16)	.04 (.19)	.01 (.06)	.01 (.06)	.04 (.18)	.04 (.19)
40	.04 (.20)	.06 (.24)	.01 (.08)	.01 (.08)	.05 (.21)	.06 (.22)
50	.06 (.24)	.08 (.29)	.01 (.10)	.01 (.11)	.07 (.27)	.08 (.28)
60	.08 (.29)	.11 (.34)	.02 (.13)	.02 (.14)	.10 (.32)	.12 (.35)
70	.10 (.33)	.15 (.41)	.03 (.15)	.03 (.17)	.14 (.38)	.16 (.42)
80	.14 (.39)	.23 (.48)	.04 (.20)	.04 (.21)	.18 (.45)	.21 (.50)
90	.23 (.48)	.29 (.60)	.08 (.28)	.08 (.29)	.24 (.53)	.31 (.63)
100	.57 (.98)	.81 (1.48)	.96 (2.29)	.76 (1.34)	.99 (3.25)	.76 (1.35)
k	952	887	1,035	970	1,035	968
Σn	481,366	448,500	523,334	490,468	523,334	489,456

Note. R²_med = overall mediation effect; R²_xy.m = residual direct effect of X on Y; R²_my.x = residual effect of M on Y; z values were computed using Fisher's r-to-z transformation.

k = number of effect sizes, Σn = total sample size across k studies.

a

Computed using correlations corrected for unreliability.

To estimate the presence of spurious variance in R²_med estimates owing to order of variable entry, we estimated R²_med using both formulas (5) and (6) as described in our manuscript:

R m e d 2 = r X Y 2 – r X Y ⋅ M 2

(5)

R m e d 2 = r M Y 2 – r M Y ⋅ X 2

(6)

To illustrate our recommended procedure as outlined in Figure 4 with an example, let us consider Hammond et al.'s (2015) analysis of managerial support for work family balance as a mediator of transformational leadership on work-family conflict. Using the bivariate correlations between X, M, and Y reported in their study, we employed the following, well-known formula to compute the partial correlation between X and Y controlled for M:

r x y . m = ( r X Y – r X M r Y M ) / √ ( 1 - r X M 2 ) ( 1 − r Y M 2 )

(9)

For our second example, consider Shoss et al.'s (2013) study on organizational support as a mediator of the effects of abusive leadership on counterproductive work behavior. Following the procedure outlined in our flowchart, we classified R²_med as small at .03, which led to an assessment of R²_xy.m = .04, which was classified as large according to Table 1. Our flowchart in Figure 4 recommends searching for better or additional mediators in this situation. A large R²_my.x effect of .13 suggested a search for better or additional X variables too.

Next, we want to illustrate how the three recommended common-metric mediation effect sizes, as well as the recommendations in our flowchart (Figure 4), can also guide theoretical refinement when considering multiple studies at the same time. Table 2 presents the sample-size weighted average effect sizes across mediation studies for transformational and abusive leadership as X variables, broken down by each constituent construct as M variables. We report the average effect sizes, both corrected and uncorrected for unreliability, for the three common-metric effect sizes advocated here, namely R²_med, R²_xy.m, and R²_my.x. To average these effect sizes across studies without violating distribution assumptions, we first computed the square root of the R²_med, R²_xy.m, and R²_my.x mediation effect sizes to obtain what would be their equivalent multiple correlations, which we then transformed to z-scores using the Fisher's r-to-z formula to ensure a normal distribution. After averaging these z-scores by each type of social exchange quality constituent construct, z-scores were back-converted to multiple correlations and then squared to compute R² proportions of variance for ease of understanding. We repeated the same process using correlations corrected for unreliability.

OPEN IN VIEWER

Table 2.

Mean Differences in Effect Sizes by Type of Social Exchange Quality Constituent Construct.

Effect Size	M = Justice				M = Support			M = Trust
	k	n		Mean(SD)	k	n	Mean(SD)	k	n	Mean(SD)
	X = Transformational Leadership
R2med	7	1,608		.16 (.02)	3	1,021	.03 (.01)	10	2,427	.13 (.04)
R2med a	7	1,608		.19 (.03)	3	1,021	.04 (.01)	9	2,004	.18 (.08)
R2xy.m	7	1,608		.05 (.04)	3	1,021	.00 (.00)	10	2,427	.03 (.02)
R2xy.m a	7	1,608		.05 (.05)	3	1,201	.00 (.00)	9	2,004	.03 (.03)
R2my.x	7	1,608		.06 (.01)	3	1,201	.04 (.02)	10	2,427	.01 (.00)
R2my.x a	7	1,608		.07 (.02)	3	1,201	.06 (.03)	9	2,004	.01 (.01)
	X = Abusive Leadership
R2med	32		9,713	.04 (.01)	8	1,658	.02 (.00)	na
R2med a	32		9,713	.05 (.01)	8	1,658	.03 (.00)	na
R2xy.m	32		9,713	.01 (.01)	8	1,658	.02 (.01)	na
R2xy.m a	32		9,713	.01 (.01)	8	1,658	.02 (.01)	na
R2my.x	32		9,713	.04 (.02)	8	1,658	.07 (.00)	na
R2my.x a	32		9,713	.04 (.02)	8	1,658	.11 (.00)	na

Note. R²_med = overall mediation effect; R²_xy.m = residual direct effect of X on Y; R²_my.x = residual effect of M on Y. The difference between every pair of means within the same raw is statistically significant at p < .01; these means were compared using a significance test of two independent means by first computing √R² to obtain r and then using the Fisher's r-to-z transformations to compute z scores for the mean comparison.

a

Computed using correlations corrected for unreliability.

 Table 2 shows that transformational leadership had generally larger R²_med mediation effects on employee outcomes when justice and trust were the mediator than when support was the mediator. Interestingly, a comparison of the R²_xy.m effect size among mediators suggested that justice is not only the best mediator of transformational leadership amongst the three constituent constructs considered here, but also the one that leaves the most potential for finding additional mediators. Furthermore, a comparison of the R²_my.x effect size among mediators revealed that there is a larger chance of finding additional X variables other than transformational leadership that transmit their effects on employee outcomes through justice or through support rather than through trust.

In short, justice and trust did a better job at transmitting the mediation effect of transformational leadership on employee outcomes than did support. In fact, studies relying on justice and trust had an R²_med mediation effect size almost five times larger than the one found in those using support. This finding suggests that the action theory underlying justice and trust as mediators best fits the positive impact of the transformational leader's inspirational ideals on employee cognitions and emotions. Again, this difference does make sense, because transformational leaders project primarily ideals and inspiration not necessarily accompanied by an implicit promise of support, which seems more characteristic of, for instance, servant leaders (Van Dierendonck, 2011).

Turning our attention to the studies employing abusive leadership as the X variable, those using justice had a larger R²_med mediation effect than those using support. This finding is not surprising because perceived justice represents a cognitive calculation of the extent to which organizational actions are fair and proportional to employee efforts (Cropanzano et al., 2007). As such, abusive leadership constitutes a clear violation of an implicit social contract assuming fair employee treatment. In contrast, support stems from the degree to which employees believe that their organization and its agents demonstrate care, concern, and supportive behaviors toward employees (Eisenberger et al., 1986; Mayer et al., 1995). In this respect, one might argue that abusive leadership is neither a positive nor a negative signal of support, but an index that captures a completely different category of negative leader behavior likely to infringe on employees’ perception of what constitutes fair treatment, which explains why justice seems a better mediator of abusive leadership than support. This conjecture is consistent with the fact that an assessment of R²_my.x revealed a larger chance of additional antecedent or X variables other than abusive leadership transmitting their effects on employee outcomes when support rather than justice was the mediator.

The finding that all constituent constructs of social exchange quality had generally larger mediation effects as transmitters of transformational leadership than as transmitters of abusive leadership is revealing in our opinion. Indeed, social exchange quality might understandably be most conducive of a leadership style like transformational leadership obsessed with introducing positive elements in the leader-member exchange such as inspiration, consideration, and idealized influence. By contrast, a negative leadership style like abusive leadership might transmit its effect through similarly negative mediators (e.g., frustration) that were not part of our literature review. The valuable lessons learned from the examples in our practical illustration notwithstanding, our primary goal was developing a generic set of guidelines for those interested in the employment of common-metric effect sizes to enhance the theoretical and practical significance of mediation models. The next section summarizes these guidelines and their merits.

Guidelines and Conclusions

We argue that intuitive, common-metric mediation effect sizes provide researchers with valuable insights on potential gaps in mediation studies, and that such insights help refine the theoretical basis of mediation models. We illustrated how three common-metric effect sizes based on variance proportions inspired on those originally formulated by Fairchild et al. (2009) and later refined by Lachowicz et al. (2018) might help fill this gap. Specifically, we propose a specific sequence whose first step is an examination of the overall R²_med mediation effect. This examination should focus on the theoretical and practical significance of the overall mediation effect. The second step involves a two-fold examination of residual variance through (1) the R²_xy.m residual effect of X on M to identify the need for better or additional mediators, and (2) the R²_my.x residual effect of M on Y to pinpoint the need for better or additional antecedent X variables. As a last step of our proposed process, we advocate a comparison of one's effect sizes with those obtained in prior studies using similar X, M, and Y variables. This comparison should facilitate the refinement of mediation theory through the identification of potentially better mediators and antecedent variables.

A non-trivial R²_xy.m suggests partial mediation because M fails to capture the full effect of X onto Y. Therefore, a non-trivial R²_xy.m suggests the need to identify alternate mediators even in the presence of a non-trivial R²_med. Perhaps more novel are our recommendations concerning the residual variance captured by R²_my.x, which is indeed a critical and yet largely neglected aspect of a mediation chain. According to Occam's razor, other things being equal, one should favor the most parsimonious explanation of a phenomenon that includes the smallest number of entities. Thus, introducing a third variable or mediator M to explain the relationship between X and Y is justified solely if R²_my.x improves the prediction of Y in the X → Y model through an X → M → Y mediation model. It is this superior prediction and improved understanding of the phenomenon under study (i.e., Y) that justifies the addition of M to the chain as a more proximal variable to Y than X is. When R²_my.x is trivial, one may question why should one place M as a more proximal antecedent of Y than X is, because M does not explain any incremental variance in Y that X did not already explain. In such cases, M adds only redundancy and unnecessary complexity to an X → Y bivariate model in which X by itself already explains Y equally well. A non-trivial R²_my.x, on the other hand, invites a search for alternate Xs missing in the model that might explain the phenomenon transmitted via M better than the current X does.

The fact that up to 95% of the studies that we reviewed as part of our practical illustration gathered the X and M variables using a cross-sectional, mono-method design underscores the need to go beyond statistical significance tests of mediation. Indeed, significance tests do not compensate for shortcomings in research design that preclude cause-and-effect inferences of mediation. The logic of hypothesis testing cannot by definition rule out support for alternate mediation models even when there is support for a specific model. Spector and Meier (2014) pointed out that the typical cross-sectional design employed to analyze mediation often misses a key characteristic of these processes, namely the temporal sequence in which conditions and events unfold. Unfortunately, while coding our data we discovered that many mediation studies fail to justify their choice of time interval between the administrations of the X, M, and Y measures, thus underscoring the need for future research to make well-informed choices of time lags reflecting the expected onset of variables in mediation studies. For instance, gathering the X and M variables at the same time appears to be much more frequent than gathering the M and Y variables at the same time. Even though this practice seems convenient, it likely exacerbates the X → M or action theory path while it weakens the M → Y or conceptual theory path, thereby adding systematic bias to mediation tests and effect sizes. Researchers should consider designs that counterbalance the order in which X and M are gathered so that order effects can be estimated.

The steps listed in the process recommended to assess mediation-related effect sizes in Figure 4 require making determinations regarding the theoretical and practical significance of these effects. Researchers should keep in mind that, in the realm of organizations, judgments of practical significance should consider the strategic fit of the variables included in the model. In fact, we argue that strategic reasons play an equal if not more important role in the refinement of a mediation model than theoretical reasons. For instance, if top management sees employee trust in leadership as a core strategic value, top management should insist on leadership styles that transmit a positive effect onto the desired employee outcomes through employee trust. Alternatively, if the organization firmly endorses a particular type of inspirational leadership that they deem critical for their people strategy, for instance transformational leadership, a search for additional mediators should focus on those that warrant the transmission of transformational leadership effects onto employee outcomes. We hope that our description of mediation-related common-metric effect sizes, together with our practical illustrations and recommended guidelines, become a springboard for researchers interested in refining mediation models by going beyond significance tests while taking stock of the rising numbers of mediation studies in organizational research.