All Things in Moderation,Including Tests of Mediation (at Least Some of the Time)

Abstract

Seeking to explain the processes by which one construct causes another is a staple of psychological research. This is typically done with statistical procedures using observations of the hypothesized cause, mediator, and effect. Unfortunately, interpretations of mediation are generally weak on internal validity when using this approach. As an alternative, a causal chain approach has been advocated, but that approach is often impractical. In this article, a third method that involves measuring or manipulating a moderator is described. The core of the argument for a moderation approach hinges on the recognition that mediation refers to a mechanism or process that might be blocked or enhanced via a moderator. Thus, finding interactions with manipulations, variables, or constructs that might affect the efficiency, rate, or operation of a mechanism, or the links to or from the mechanism, implies that mechanism is involved in determining the relationship between a cause and an effect. Examples and implications regarding the potential use as well as the advantages and disadvantages of the approach for applied researchers are described.

Keywords

mediation moderation research design

Theories are an important aspect of organizational, management, and applied psychology (Edwards, 2010). A key element of theory involves articulating and testing mediating mechanisms or processes thought responsible for important relationships (Mathieu, DeShon, & Bergh, 2008). Because of this, mediational analysis, where a key intervening variable or construct is identified and assessed for its role as an intermediator between a presumed cause and effect, has garnered much attention in the organizational (e.g., see Mathieu, DeShon, & Bergh, 2008) and psychological (MacKinnon, Fairchild, & Fritz, 2007) literatures. Several designs for assessing mediation have been identified (MacKinnon & Fairchild, 2009; Spencer, Zanna, & Fong, 2005), but one method dominates: the statistical mediation analysis approach (SMAA; MacKinnon, Lockwood, Hoffman, West, & Sheets, 2002), which is most commonly associated with a paper by Baron and Kenny (1986). Indeed, in a review of 409 studies testing 709 mediational models published in five top organizational research journals between 1981 and 2005, Wood, Goodman, Beckmann, and Cook (2008) found that 100% used the SMAA.¹

Unfortunately, the internal validity is often weak for many of the inferences made in studies using the SMAA (e.g., Freedman, 2004; Mathieu & Taylor, 2006; Shaver, 2005; Stone-Romero & Rosopa, 2004; Wood et al., 2008). This is a problem because mediational hypotheses are inherently causal (James & Brett, 1984). That is, mediational hypotheses claim that some cause (x) affects one or more mediators (m) that affect some endogenous variable (y). When using SMAA to assess mediation, the presumed mediator is passively observed (i.e., not manipulated by the researcher), and the presumed causal variable (x) may be as well—arguably an advantage of the approach because of difficulties manipulating variables (e.g., James, 2008; Kenny, 2008). Yet, strict assumptions are required to make causal inferences—assumptions that observational studies rarely meet (Morgan & Winship, 2007; Stone-Romero & Rosopa, 2004). Indeed, to avoid making causal claims when the data meet the statistical criteria, researchers are encouraged to say the data are consistent with a model of mediation. Unfortunately, Wood et al. (2008) noted that researchers made causal claims for 471 (66%) of the models they reviewed, despite the fact that only 224 (31%) of those models included manipulated variables. Even then, the manipulated variable was the hypothesized cause (x). Thus, alternative directions of causality were still possible (e.g., y causes m; x causes m and y, but m is not involved in causing y). For example, self-efficacy is frequently considered a full or partial mediator of the effect of training on performance (e.g., Colquitt, LePine, & Noe, 2000). Yet, self-efficacy, which refers to belief in one’s capacity (Bandura, 1986), may simply be a marker of a capacity that increases with training and it is the capacity (not belief in the capacity) that is actually responsible for training’s effect on performance (Heggestad & Kanfer, 2005).

To address this shortcoming of the SMAA, Stone-Romero and Rosopa (2004, 2008, 2010, 2011) and Spencer et al. (2005) have promoted the experimental causal chain approach to assessing mediation. In the causal chain approach, x must be manipulated in one study where m and y are observed, and in a second study, m must be manipulated and y observed. As examples of the approach, Stone-Romero and Rosopa (2011) described two studies examining the mediating role of self-efficacy. In one study, leader expectations about their subordinates, which was the hypothesized x, were manipulated and self-efficacy (m) was measured (Davidson & Eden, 2000). In a second study, the self-efficacy of navy cadets for managing seasickness (m) was manipulated via verbal persuasion and modeling (Eden & Zuk, 1995) and seasickness (y ₁) and performance (y ₂) were measured. All the links across these studies were found to relate as hypothesized. Given these two studies, Stone-Romero and Rosopa (2011) concluded that leader expectations can influence subordinate performance via the subordinates’ self-efficacy, though the set of studies did not completely conform to their specifications for the causal chain approach (i.e., performance [y] was not measured in the Davidson & Eden, 2009, study). Indeed, they acknowledged (p. 14) that they knew of no set of published studies that conformed to the precise specifications of a valid causal chain set of studies.

One reason causal chain studies are rare is likely the strict requirements of the approach. For instance, the approach requires two manipulations: one manipulation of the exogenous variable (x) of interest and one of the mediator (m) using a method other than the one used to manipulate the exogenous variable. Preferably, individuals are randomly assigned to conditions in both manipulations to address selection threats to internal validity and reduce confounds with the independent variable (Shadish, Cook, & Campbell, 2002). Moreover, measurements of the mediator along with measurement of the dependent variable are required. In response to the strict prescriptions of the causal chain approach, James (2008) and Kenny (2008) have countered that manipulations of either the exogenous or mediator constructs are not always possible, can threaten their construct validity (e.g., training as a manipulation can also influence capacity), or can introduce other threats to internal and external validity (e.g., demand characteristics; Orne, 1962; but see Stone-Romero & Rosopa, 2011, for a response to these criticisms). In addition, Shadish et al. (2002) add the possibility that the causal processes in one study might not generalize to the context that is the focus of the mediational hypothesis. Indeed, the point of mediational analysis is often to determine if some known causal factor (i.e., the mediator) is a specific causal factor responsible for a particular exogenous variable’s (x) influence on an endogenous (y) variable. That is, showing that a mediator can cause an endogenous variable does not mean that it is the mechanism by which a particular exogenous variable is affecting the endogenous variable. For example, in the Eden and Zuk (1995) study, it may be that self-efficacy positively affected performance at sea because seasickness undermines performance at sea and one’s self-efficacy for seasickness influences seasickness. Yet, when performances are unaffected by seasickness, self-efficacy for seasickness (or for anything else) might not matter.

The debate over which design is best for drawing valid conclusions regarding mediation is likely resolved using the triangulating approach advocated by Runkel and McGrath (1972). That is, no one design can maximize all the criteria sought by researchers and practitioners, and therefore, multiple methods ought to be employed. What we find interesting is that a design method, called moderation-of-process (Spencer et al., 2005) or enhancement/blockage (MacKinnon & Fairchild, 2009), could be a third leg in this triangulation process. Yet, the moderation-of-process approach has not received much attention in the organizational and management literature. The purpose of this article is to explain the moderation-of-process approach and how it can be and has been applied to address mediational hypotheses.

In brief, the moderation-of-process design involves measuring or manipulating a construct thought to affect the workings of, or links to or from, a mediating mechanism (e.g., blocking, weakening, or enhancing the mechanism or links). Although not perfect—hence the need for triangulation—the moderation-of-process design has several potential advantages over the statistical mediational analysis and causal chain approaches. These include (a) examining a pattern of findings that is likely subject to fewer alternative explanations; (b) the possibility of examining mediating mechanisms or processes that are not observable; (c) the possibility of making reasonably valid causal conclusions without resorting to manipulations of the exogenous, moderating, or mediating variables; and (d) the possibility of assessing an intervention implied by the theory of mediation. Moreover, to apply the moderation-of-process approach, one must clearly articulate the mediating mechanism or process, and thus it encourages a detailed explication of the theory being tested. In the sections to follow, we first define mediation and moderation. We then describe variations of the moderation-of-process approach to assessing mediation. Next, the advantages and disadvantages of the approach are described. Our purpose is not to suggest that moderation-of-process replace either of the other approaches, but that it should be considered an additional design in the researcher’s repertoire. Indeed, in the final section, we describe the steps a researcher might follow to develop a program of research designed to assess the mediating mechanisms of an important x-y relationship. This program is likely to include numerous approaches or combinations of approaches. Within this section, several examples from the literature are provided as a means for further illustrating our points.

Defining Mediation and Moderation

As implied by the label moderation-of-process, the approach involves using the moderator concept to test a mediational hypothesis. Confusion between mediation and moderation prompted the original papers on tests for mediation (Baron & Kenny, 1986; James & Brett, 1984) and continues to haunt the literature (Holmbeck, 1997); thus, it is important to distinguish the concepts. Typically, mediator and moderator refer to two different types of variables. For instance, Baron and Kenny (1986) define a mediator as a “variable, which represents the generative mechanism through which a focal independent variable is able to influence the dependent variable of interest” (p. 1173). Self-efficacy as the mediator between training and performance is one, albeit questionable, example (Mathieu & Taylor, 2006). In contrast, Baron and Kenny define a moderator as “a qualitative (e.g., sex, race, class) or quantitative (e.g., level of reward) variable that affects the direction and/or strength of the relation between an independent or predictor variable and a dependent or criterion variable” (p. 1174). For example, cognitive ability is considered a moderator that affects the degree to which training leads to performance (i.e., training’s influence on performance is stronger for those with higher cognitive ability; Colquitt et al., 2000). Visually, these two concepts are typically represented in the boxes and arrows diagrams shown in Figure 1, where the top diagram represents mediation and the mediator, and the bottom diagram represents moderation and the moderator.

Figure 1.

Mediation and moderation models.

An important, often overlooked, point is that variables can be contrasted with processes. This distinction is implicit in Baron and Kenny’s (1986) definition of the mediator. That is, a process refers to a mechanism or explanation. Baron and Kenny noted that the mediator variable merely represents a mechanism for explaining a known (or presumed) relationship between a cause (x) and effect (y). The key issue is the mechanism, process, or explanation. For example, in old-fashioned, rear-wheel drive automobiles, the drive train is an important mechanism that mediates the relationship between gas pedal depression and the speed of the car. A key variable representing this mechanism is the speed of rotation of the drive train. Speed of rotation is not the mechanism; it is merely a property of the mechanism that represents its level of operation.

The distinction between process and variable was an important issue in psychology in the middle of the past century. In particular, MacCorquodale and Meehl (1948) noted that mediating variables (called “intervening variables”) could be processes (called “hypothetical constructs”) that “involve terms which are not wholly reducible to empirical terms; they refer to processes or entities that are not directly observed (although they need not be in principle unobservable)” (p. 104). This statement created a great deal of backlash from the behaviorists of the day due to the behaviorists’ emphasis on observables, an emphasis that for mediational analysis appears to have continued to today (Mathieu et al., 2008). To be sure, to test a mediating hypothesis, some empirical prediction must be made and observations obtained, but empirical operationalizations implicating the hypothetical constructs can vary.² In this article, we describe alternative methods for operationalizing hypothetical processes in terms of empirical predictions, specifically, a finding of moderation not for the mediator but for a variable (z) that implicates a mediator.

Before describing how a moderator might implicate a mediating process, we need to note one final possible issue. That is, mediator and moderator are not only used as labels for variables or processes but also as adjectives applied to design and analysis procedures. This application is not inherently problematic. Rather, a problem has arisen because the mediator qualifier has typically been conceptualized only in terms of the SMAA (Wood et al., 2008) and the moderator qualifier has typically been conceptualized only in terms of determining boundaries for relationships (Aguinis, Beaty, Boik, & Pierce, 2005). The point of our article is to expand the repertoire of designs and analysis procedures that one can use to assess mediational hypotheses and to the interpretations that one might make regarding findings of moderation. Generally, the data emerging from these designs can be analyzed with some type of moderator (see Aguinis et al., 2005) or integrated moderator and mediator analysis (Edwards & Lambert, 2007), depending on the specifics of the design (e.g., includes a measure of the mediator).

Moderation-of-Process Logic

The basic idea for the moderation-of-process approach arises from the definition of mediation as a mechanism or process as opposed to a variable. Specifically, the statistical mediation analysis and causal chain approaches require identifying a key, measurable property of the mechanism or process and assessing the covariance among the observed or manipulated hypothetical causal variable, x, this measurable property, m, and the presumed effect, y. The SMAA further requires examining partialled coefficients or combinations of coefficients (i.e., differences or products). This allows the researcher to condition on the levels of the mediator and thus to differentiate direct and indirect effects between the x and y variables (Morgan & Winship, 2007). In contrast, the moderation approach requires identifying (a) ways to influence the effectiveness, rate, or operation of a mechanism; (b) methods that undermine (or enhance) links to or from the mechanism; or (c) measurable properties (i.e., moderator variables) that represent “a” or “b.” The first two approaches describe an experiment where the moderator (z) is manipulated (and possibly observed; i.e., use of a manipulation check); the last approach involves observation of the moderator. Note that when using the moderation-of-process approach, the x variable may be either manipulated, measured, or both. All cases involve assessing the moderating effect of z on the relationship between x and y and possibly m, depending on whether m can be measured and where in the process the moderator is presumed to have its effect. For example, if the moderator affects the link from x to m, z should moderate x’s effect on m and y. We call this case Type 1. If the moderator affects the operation of the mediator, z should also moderate x’s effect on m and y. We call this case Type 2. Finally, if the moderator affects the link from the mediator to y, z should moderate x’s effect on y but not m. We call this case Type 3.

To illustrate the logic of the moderation-of-process approach, we first use our car drive train example—because it is a known system—plus a little imagination. Specifically, imagine all human life on the planet is destroyed but many of humans’ machines have been left intact. Aliens visit the planet and begin to try to understand some of these machines. Automobiles are particularly intriguing (especially older model muscle cars), and a discipline devoted to understanding the automobile emerges. The scientists within this discipline learn how to make the older model cars run but are puzzled over the processes involved. Within the discipline arise two schools of thought regarding the operation of the automobile. In one school, the central explanation for the relationship between gas pedal depression and the speed of the car is the drive train (as mentioned previously). In the other school, the central explanation for the relationship between gas depression and car speed is the alternator.³ Both schools have identified key mediator variables (i.e., observable properties of the mechanism) to represent their respective mechanisms. In the case of the drive train, it is the speed of rotation of the drive train. In the case of the alternator, it is the voltage of electricity leaving the alternator. Both schools have also published papers demonstrating that when the measure of their respective mediator is controlled statistically, the observed relationship between gas pedal depression and car speed becomes nonsignificant and drops significantly from the relationship found prior to controlling for the respective mediator variables. Moreover, the product of the coefficients for x to each m and each m to y are significant (these are the criteria for the causal step, difference in coefficients, and product of coefficients varieties of the SMAA, respectively; MacKinnon et al., 2007). Thus, the SMAA supports both theories indiscriminately.

To resolve the debate, a moderation-of-process methodologist suggests that moderator variables can be manipulated for each hypothesized process and can be used to assess the mediational hypotheses. Specifically, for the drive train hypothesis the methodologist suggests that a wrench can be inserted into the drive train. A wrench in the works should effectively, and literally, disrupt its operation. It represents a Type 2 case of moderation-of-process. Likewise, removing the belt that operates the alternator or cutting the wire that carries the electricity from the alternator should effectively disrupt the alternator’s influence. The first situation (i.e., removing the belt) represents a Type 1 case because it would sever the link from the exogenous variable (i.e., petal depression) to the hypothesized mediator (i.e., the alternator). The second situation (i.e., cutting the wire) represents a Type 3 case because it would sever the link from the hypothesized mediator (i.e., the alternator) and the endogenous variable (i.e., speed of the car). Two experiments are conducted to test the theories. In one, a wrench is or is not inserted in the drive train and the gas pedal is depressed to two levels (i.e., a 2 × 2 design). In the other, the wire from the alternator is or is not cut and the gas pedal is depressed to the same two levels (i.e., another 2 × 2 design). In both experiments, the speed of the cars is recorded.

The findings are clear. The relationship between gas pedal depression and car speed is only different when the wrench is inserted in the drive train. This finding supports the inference that the drive train, not the alternator, is involved in determining the relationship between gas pedal depression and car speed.⁴ Emboldened with the design’s success, the researchers used the presence or absence of gasoline (i.e., a Type 2 case) to confirm the engine’s mediating role in making cars run.

Given the confusion regarding mediators and moderators (Baron & Kenny, 1986), it is important to highlight that the moderator variable (e.g., the presence or absence of the wrench) is not a mediator variable. It is a separate variable (z) that affects the workings of the mediator. The variable representing the mediator (m) is still the rotation speed of the drive train. This variable could still be measured and included in a moderated mediation analysis (e.g., Edwards & Lambert, 2007; Muller, Judd, & Yzerbyt, 2005), further confirming the role of the drive train as well as the effect of the moderator on its operation.

In sum, the moderation-of-process approach requires a description of the hypothesized process—a presumed default for any mediational hypothesis—as well as a description of how the moderator is presumed to affect that process. At an abstract level there are three possibilities, which we call Type 1, Type 2, and Type 3. The first possibility (i.e., Type 1) is that the moderator decouples, weakens, or enhances the link between the exogenous causal variable (x) and the mediating mechanism. Disconnecting (or not) the engine from the drive train or the belt that operates the alternator would be examples of this possibility, though for two different mediators. The second is that the moderator affects the process directly (i.e., Type 2). The presence (or absence) of a wrench in the drive train and the presence (or absence) of gasoline in the tank are examples of this type of moderator. The third possibility (i.e., Type 3) is that the moderator influences the link from the mediator to the endogenous effect variable (y). Cutting the wire (or not) emerging from the alternator is this type of moderator.

Advantages of the Moderation-of-Process Approach

Although the moderation-of-process approach cannot guarantee internally valid inferences—a point we make more directly in the next section—it can have several advantages over the statistical mediation analysis and causal chain approaches.

Unique Pattern of Results

The most important potential advantage is that articulated predictions of moderation might be “consistent with one mediation theory and inconsistent with another theory” (MacKinnon & Fairchild, 2009, p. 18). This was the case for the automobile example. In an example closer to the heart of organizational researchers, however, consider the job satisfaction–performance relationship. For decades, researchers have tended to find a positive but often weak correlation between job satisfaction and performance. Likewise, for decades researchers have argued over the direction of the relationship. A particularly intriguing argument by Porter and Lawler (1968) was that job performance causes job satisfaction via rewards (see Figure 2). An assumption of, or condition key to, this argument is that rewards are contingent on performance. To assess this theory, Cherrington, Reitz, and Scott (1971) designed a study that manipulated the contingency between performance and rewards (i.e., they manipulated the reward system, which was the moderator, z) and showed that the relationship between performance and satisfaction was only positive when rewards were positively contingent on performance. The relationship was negative when rewards were inversely related to performance. This pattern of results provided strong support for Porter and Lawler’s (1968) contention that performance causes job satisfaction via rewards.⁵ In terms of the type of moderator examined, this example is closest to Type 1 (see Figure 2). That is, the reward system (z) affected the link between performance (x) and rewards (m).

Figure 2.

Job satisfaction function of performance via rewards model.

Measurement of the Mediator Preferred but not Absolutely Required

Although measuring a property of a mediating mechanism can help when interpreting moderation-of-process studies and strengthen the validity of inferences, another key advantage of the approach is that it does not require a measure of the mediator. This can be helpful because hypothetical constructs presumed to mediate cause-effect relationships might not be “wholly reducible to empirical terms” (MacCorquodale & Meehl, 1948, p. 104). Indeed, a limitation of the causal chain approach is that empirical investigations of mediational hypotheses are restricted to mediators that can be manipulated and measured (James, 2008), and a limitation of SMAA is that it only can be applied to mediators that can be measured. In this way, the behaviorists’ edict that psychology can only deal with observable variables has continued to restrict the field.

Instead, the moderator approach we are advocating opens up mediational research to a classic approach for assessing psychological explanations. In particular, in experimental research it is the venerable 2 × 2 design (i.e., the crossing of two levels of two variables) that is used to examine explanations. For example, Vancouver and Tischner (2004) tested a key part of Kluger and DeNisi’s (1996) feedback intervention theory (FIT) using two moderator-of-process elements. Specifically, FIT hypothesizes that negative task feedback creates discrepancies in goal systems that individuals are motivated to reduce (Lewin, 1954). One goal system is the self-concept system. FIT hypothesizes that negative feedback, as compared to positive feedback (i.e., feedback valence), can adversely affect performance because the discrepancy it creates in the self-concept goal steals attention resources from the task at hand. This description evokes a series of two mediational constructs (see straight arrows in Figure 3). Unfortunately, discrepancies from goals and attention resource availability are difficult to measure, as illustrated by the dashed boxes for these concepts. Thus, to assess the mediational hypotheses of the self-concept path, Vancouver and Tischner used a 2 × 2 × 2 experimental design. The first variable was feedback valence (positive or negative feedback). This manipulated the hypothesized exogenous variable (x). The second variable was a self-affirmation manipulation (z ₁), which was designed to affect the self-concept discrepancy, which was the first mediator (m ₁). That is, self-affirmations are opportunities to engage in activities or recall instances that signal one’s integrity, morality, and adequacy, and thus they reduce self-concept discrepancies independent of the original source of the discrepancy (Steele & Liu, 1983). The manipulation of self-affirmation is a Type 2 moderator; it directly affects the first mediator self-concept system. To illustrate this type of moderator graphically, we use a diamond-headed connector (see Figure 3) to convey the notion that the moderator affects the operation of the mechanism rather than causing changes in the level (i.e., value) of the mediator,⁶ though operations can affect levels.⁷ The result is that the self-affirmations should negate or at least mitigate the effect of feedback valence on the self-concept discrepancy and therefore the effect of self-concept discrepancy on attentional resource availability and performance. The third variable was the resource sensitivity of the task (z ₂), which affects the link between resources available (m ₂) and performance (y). That is, some tasks are more sensitive to attention resources than others (Kanfer & Ackerman, 1989), and the study involved one task low on resource sensitivity and another one high on resource sensitivity. It represents a Type 3 process moderator as indicated by the moderator pointing at the arrow from the mediator to the endogenous variable.

Figure 3.

Model of feedback valence’s effect on performance via goal discrepancies.

Consistent with FIT, Vancouver and Tischner (2004) found that feedback valence positively affected performance when individuals were not able to self-affirm and the task required attention resources (i.e., resource sensitivity was high). However, when individuals were able to self-affirm or the task did not require much attention, feedback was negatively related to performance (i.e., those in the negative feedback condition performed better than those in the positive feedback condition). This later effect is consistent with Kluger and DeNisi’s (1996) position that negative feedback will create a discrepancy in the task goal system that will motivate subsequent performance (see lower path in Figure 3), an effect that can be realized provided the self-concept system is not stealing precious attention resources needed for the task.

Manipulations Preferred but Not Required

The previous examples all involved manipulations of the moderator variable. However, a third advantage of the moderation-of-process approach is that it can be done with only passively observed variables, which is considered one of the advantages of the SMAA (James, 2008). For example, one might replicate the Cherrington et al. (1971) study via observations of organizational (or position-level) reward systems (z) and the correlation between job performance (x) and job satisfaction (y) within those units. That is, a finding that the correlation between job performance and job satisfaction was stronger in organizations where rewards (m) were based on performance would replicate the Cherrington et al. findings. Note that rewards might vary dramatically across positions and tenure, making inclusion of rewards as a measure of the mediator very noisy and thus likely to play an unpredictable role.

An example of the moderation-of-process approach using a nonmanipulated moderator can be found in a set of studies by Adam and Shirako (2013). These researchers examined the role of stereotypes about Asians’ emotional expressiveness on interpreting emotional expressions during negotiations. The researchers reasoned that because expressions of anger among Asians are considered out of character (i.e., violate the stereotype, m), such expressions would be taken more seriously and lead to greater cooperation with the negotiator (y) when that negotiator was Asian as opposed to European American (x, see Figure 4). They found x (i.e., negotiators ethnicity) related to y (i.e., cooperation with negotiator) in the first three studies they reported. However, most important for our purposes, they implicated the stereotyping mechanism by measuring stereotype belief (z) in Study 4. Stereotype belief represents a Type 1 case because the stereotype mechanism is presumably not engaged if no stereotype belief exists to be violated. They found the effect disappeared in the group that did not hold the belief.

Figure 4.

Model of cooperation with negotiator via violation of stereotype with anger expression.

The Adam and Shirako (2013) study also provides an example of how one might estimate direct and indirect effects despite no measure of the mediator. Specifically, one can assume that having no stereotype completely blocks the operation of the stereotype-violation mechanism; thus, the effect found when the participant held the stereotype represents both the total and indirect effect (i.e., there was no direct effect). In contrast, the Vancouver and Tischner (2004) case is more problematic because one cannot assume that the self-affirmation manipulation completely reduced the self-concept discrepancy or that the resource insensitive task was completely resource insensitive. Thus, it was not possible to know if the moderators completely blocked the top, positive path in Figure 3. If not, the magnitude of the negative effect of the bottom path would be an underestimate and thus could not be used in calculating the indirect effect. More generally, if the moderator is presumed to dampen or enhance, as opposed to completely block the mediator path, estimating direct and indirect effects is not possible without a measure of the mediator or a precise understanding of the degree of dampening or enhancement of the moderator.

Potential Bonus Finding

The final advantage of the moderation-of-process approach to assessing mediation is that it may be a simultaneous assessment of an intervention implied by the mediating mechanism. That is, explanations and understandings of mediational processes are useful because they suggest policies or interventions that will affect outcomes of interest to the individual or the organization. Consider the aforementioned performance to job satisfaction example. If job satisfaction is a key variable that affects turnover (Griffeth, Hom, & Gaertner, 2000), the findings of moderation imply that organizations can use pay-for-performance systems to strengthen the relationship between performance and job satisfaction, increasing the chances that organizations will keep their good workers. Indeed, the moderation-of-process approach to testing a mediating mechanism was advocated by Kraemer, Wilson, Fairburn, and Agras (2002) for its practicality. For them, a primary reason for identifying a mediating mechanism is to enhance the effectiveness of some causal agent (x) if the outcome (y) is desired or to undermine it if the outcome is undesired. The results of attempts to change effectiveness would be reflected in findings of moderation (or not, if the mechanism was not involved).

Disadvantages of Moderation-of-Process Approach

There are four primary disadvantages to the moderation-of-process approach. First, it requires an inferential step not required with the statistical analysis or causal chain approaches. Specifically, it requires inferring how some moderator construct can influence a link to or from a mediation process or how it can influence the operation of the process itself. In particular, if the mechanism is “inadequately explicated,” a threat to construct validity of z could occur (Shadish et al., 2002). This issue can be mitigated by measuring a key property of the mediating mechanism and conducting a mediated moderator analysis. In particular, the mediated moderator analysis discussed by Edwards and Lambert (2007) allows for fairly precise understanding of the moderation process (i.e., it can distinguish Type 3, which would result in what Edwards and Lambert call a second stage moderation model, from Types 1 and 2, which would result in a first stage moderation model). That said, we would advocate for testing both hypothesized and not hypothesized stages to avoid the logical fallacy of affirming the consequent (Wason, 1968).

A second disadvantage of the moderation-of-process approach is that the moderator construct (z) must be measured or manipulated. If z is manipulated, the cleanliness of the manipulation is of course an issue (e.g., whether more than z being manipulated). Fortunately, because the relationship of interest is more complex (i.e., an interaction), it seems less likely that demand characteristics and other construct-related threats that experiments evoke (e.g., compensatory rivalry) can threaten causal inferences (Shadish et al., 2002). If z is measured, the typical issue of confounds (i.e., unmeasured variables) is a concern because the unmeasured variables might affect alternative processes that are actually responsible for mediating the effect of x on y. For example, in the Adam and Shirako (2013) study previously mentioned, one cannot assume that individuals differing in terms of the stereotypes held about the emotional expressiveness of Asians do not also differ on other variables (e.g., awareness of emotional signals) that may be responsible for the moderating effect. Of course, ferreting out these alternative explanations prior to data collection can inform measurement and design decisions for the study. Another problem when the moderator is merely measured is reverse causality (i.e., the strength of a relationship influences the level of the moderator). For example, if one were to conduct the observational version of the Cherrington et al. (1971) study we suggested previously, one might argue that job satisfaction is more likely to motivate high performance when performance measures are uncontroversial, and this lack of controversy promotes (or does not undermine) pay-for-performance reward systems.

A third problem associated with the moderation-of-process approach is that a finding of moderation is typically based on a significant interaction, which is a statistical test that does not distinguish the moderator from the exogenous variable (i.e., x and z are interchangeable). For example, it could be that performance moderates the relationship between pay-for-performance reward systems and job satisfaction, not that reward systems moderate the performance to job satisfaction relationship. This might be the case because a pay-for-performance system, as opposed to not having such a system, only leads to job satisfaction if one is performing well. Indeed, it can be easy to mistake a moderator that affects a process with a cause of that process. In particular, if x is at some non-zero level, and z, which enhances the effect of x, is increased, it will appear as if z caused an increase in y. To understand this situation, consider the drive train example. That is, suppose the aliens were driving their cars around with the gas pedal at a certain degree of depression and decided to add grease to the rusting drive train shafts. If the grease improves the function of the drive train mechanism by reducing the friction within the drive train, the car would pick up speed upon the application of the grease. This increase in speed (y) might be misinterpreted as grease (z) causing the car to move.

The final problem with the moderation-of-process approach also relates to the statistical test and the aforementioned example. That is, much like with the SMAA, the moderation-of-process approach requires statistical tests that are often underpowered. Specially, interactions are often quite small and can be difficult to detect (Aguinis et al., 2005; Dawson, 2014; Shieh, 2009; Stone-Romero & Anderson, 1994). Part of the reason is because some of the effect of the moderator (z) appears as a main affect (Bobko, 1986; Strube & Bobko, 1989). That is, grease would increase the speed of the car even though it only serves to enhance the effect of the engine via the drive train. Elaborated mechanisms and carefully designed studies should help address this issue. For example, one might expect that good performance may be its own reward above and beyond any pay one might receive for performance. However, because Cherrington et al. (1971) went as far as reversing the contingency in their manipulation of z, rewards’ influence on satisfaction was clear in their study.

Implications

Developing and testing explanations for key relationships is a staple activity for the organizational researcher (James, 2008). Yet, the most commonly used design for testing these explanations (i.e., SMAA) is weak in terms of internal validity (e.g., Wood et al., 2008), and the other highly touted approach (i.e., experimental causal chain; Stone-Romero & Rosopa, 2008), though stronger in terms of internal validity, cannot guarantee causal inferences are correct (Kenny, 2008). Moreover, the causal chain approach is limited to cases where exogenous and mediator constructs can be manipulated (James, 2008). In addition, both approaches are limited to examining mediators that can be measured. The point of the current article is to expand the organizational researcher’s repertoire of designs used for testing theories of mediation. However, because no one study or design will likely put to rest all the issues related to internal, construct, and external validity (Runkel & McGrath, 1972; Shadish et al., 2002), we devote this section to describing a process for developing a program of research for assessing mediational hypotheses that includes all the possible designs.

To facilitate the communication of the process we advocate, we provide a list of steps when planning and conducting mediational research (see Table 1). These steps are at the level of theory and research development as opposed to steps in an analysis procedure to be used on a set of observations (cf. the causal step form of SMAA). They also do not assume one mediating mechanism. That is, in most cases multiple mediators (i.e., where each is a partial mediator) as well as chains of mediation (i.e., where an exogenous variable causes a mediator that causes another mediator that causes the endogenous variable) are likely (Mathieu et al., 2008; Taylor, MacKinnon, & Tein, 2008). Because of this, multiple iterations of the steps might be needed to fully understand an x-y relationship or all the variations of that relationship.

Table 1.

Steps for Creating a Program of Research to Assess Mediation.

Step	Description/Questions	If yes …
1	Critically evaluate x-y relationship Have studies only passively observed x? If experimental studies, could confounds threaten causal interpretations? Could conditionals threaten external validity? Could third variable[s] cause x and y? Could y cause x? Have covariates been examined? Have moderators been examined? Could the relationship be nonlinear?	Consider experiment Consider different operationalizations of x Consider moderator designs Consider controlling via measurement or experiment Consider experiment or longitudinal design If reduce relationship, consider covariates might be measure of mediator If moderation found, consider if clue regarding mediator Consider design for nonlinear analysis
2	Explicate mediating mechanism Can the mechanism be measured? Can the mechanism be manipulated independently of x? Can a link to the mechanism be enhanced or blocked? Can the mechanism be enhanced or blocked? Can a link from the mechanism be enhanced or blocked? Do naturally occurring measurable moderators affect operation of mechanism (or links to/from mechanism)? Does the process rely on assumptions of the state of variables that might vary?	SMAA likely applicable; potentially within other designs Develop causal chain design Develop MoP design Develop MoP design Develop MoP design Develop MoP design Develop MoP design
3	Consider alternative explanations for supportive findings from studies reviewed or designed in previous step. Could y cause x or could the relationship be reciprocal? Could unmeasured third variables be involved?	Avoid SMAA unless can control reciprocal process Avoid SMAA

Note: SMAA, statistical mediation analysis approach; MoP, moderation-of-process approach.

Step 1: Review the Literature

The first step in assessing the explanation for a relationship between two variables is to review the evidence of association between the x and y constructs. Presumably, for there to be an interest in explaining the relationship, evidence or theory exists suggesting that x causes y. However, one needs to be both skeptical of the interpretations made in the literature and vigilant for clues of mediation not described as such.

Be Skeptical

The evidence of x causing y is often based on a story (i.e., theory) and cross-sectional, observational designs. Because of this, one should also consider reverse causality (i.e., y may cause x), nonrecursive relationships (i.e., x may cause y and y may cause x), and third variables that cause both x and y. Moreover, dynamics, construct validity, and nonlinear functional relationships can also impede causal interpretations (James, Mulaik, & Brett, 1982). Thus, one should maintain a skeptical attitude and attempt to develop reasonable alternative explanations for observed association, including alternative mechanisms that might explain why x could cause y. Moreover, one might consider cases where a lack of a relationship between x and y might occur because the effect, though real, is weak, or dueling mechanisms obscure an overall effect (James, 2008).

To help with the aforementioned logical analysis, researchers should pay attention to (i.e., list) covariates and moderators examined in the extant literature. One interesting issue for the SMAA is that the test of an alternative explanation for x causing y in terms of a third variable examined as a covariate and the test of a mediator are identical (Cole & Maxwell, 2003; MacKinnon, Krull, & Lockwood, 2000; Rubin, 2004). That is, in both cases, the covariate/mediator should remove or, at least, significantly reduce the relationship between the x and y variables when included in the statistical model. However, in the case of a covariate analysis, this reduction is interpreted as indicative of a third variable being responsible for the covariance between the x and y variables. In the case of mediational analysis, this reduction is interpreted as indicative of mediation. Thus, one researcher’s covariate that challenges the interpretation of x causing y because the covariate causes both is another researcher’s mediator that confirms the interpretation of x causing y via m. Skeptical researchers should consider each of these possibilities. They might realize that what had been interpreted as undermining an x causing y theory was really revealing a process by which x causes y or that what was interpreted as a mediator was really revealing a third variable that led to a spurious interpretation that x caused y. Which interpretation is correct should not be based on how many times one interpretation has been made compared to another (i.e., the fallacy of consensus gentium) but on conceptual arguments and rigorous research.

Indeed, Wood et al. (2008) highlighted the problems with cumulating past empirical results due to the inconsistencies and ambiguities in studies reporting mediational effects. Specifically, they estimated that among the studies they reviewed, 280 (66%) of those claiming full mediation and 134 (77%) of those claiming partial mediation “were based on questionable grounds and therefore potentially invalid” (p. 288). This problem implies that what might be considered strong theory, particularly when developed using inductive approaches (Locke, 2007), might in fact be based on weak inferences. As Kenny (2008) notes, if the mediational theory is wrong, the results of the mediational analysis, particularly when using the SMAA, are likely misleading.

Compounding the third variable problem are dynamic issues (Cole & Maxwell, 2003; Maxwell & Cole, 2007). Specifically, processes unfold over time. Thus, using cross-sectional designs, even coupled with sophisticated analysis procedures like structural equation modeling (SEM), are likely to lead to reasonable inferences in only very restrictive circumstances (James et al., 1982). To correct the primary problem (i.e., autocorrelation due to a variable’s effect on itself over time), a multiwave, longitudinal design is needed with a minimum of three waves of data collection of all the variables across intervals that correspond to the lags in the hypothesized processes, which is a lot to ask. However, even longitudinal designs can be undermined by nonrecursive relationships and nonstationary processes (DeShon, 2012). Thus, a theory might articulate a certain causal sequence, but feedback processes might create spurious relationships (Mathieu & Taylor, 2006). For example, Bandura and Wood (1989) suggest that self-efficacy causes performance and that performance causes self-efficacy, leading to what Lindsey, Brass, and Thomas (1995) referred to as performance spirals (i.e., positive feedback loops of an increasing or decreasing nature). Meanwhile, Locke (1982) notes that individual performance will asymptote at an ability ceiling, suggesting that an upward spiral will eventually wind down. These kinds of nonrecursive, nonlinear relationships are inconsistent with assumptions required for typical analytic procedures used to examine mediational hypotheses (DeShon, 2012). All this is to say, researchers should eye the existing literature with skepticism.

Look for Clues of Mediation in Moderator Studies

Although the aforementioned discussion suggests that results are often overinterpreted, the opposite is likely true as well. That is, the existing literature provides a plethora of clues regarding mediation unrecognized because the field has not been cognizant of the idea that findings of moderation might hint at underlying mechanisms. Instead, moderators are often only considered in terms of bounding a theory or empirical relationship (Aguinis et al., 2005). Yet, as Baron and Kenny (1986) pointed out in a section on “Strategic Considerations” (p. 1178), findings of moderation might help identify a mediator. Thus, one important implication of the moderation-of-process conceptualization is that it can motivate a way of thinking about an interaction that is rarely contemplated. Moreover, if the moderation effect found identifies a mechanism, then a theory’s scope is likely expanded rather than bounded by the finding because it includes within it a description of the mechanism that can determine the degree of the relationship and a moderator construct that can affect the operation of the mechanism.

Consider, for example, the relationships between personality and performance. The field is beginning to develop a database of the job dimensions that predict personality-performance relationships (Hough & Schneider, 1996; Judge, Rodell, Klinger, Simon, & Crawford, 2013). The question is: Do these job dimensions inform researchers about the underlying mechanisms responsible for the relationships? The answer may be yes, if thought about as such. For instance, Judge et al. (2013) found that excitement-seeking, a component of extraversion, had a credibility interval of –.09 to .33 for predicting task performance (based on four studies). Excitement-seeking might positively affect performance via risk taking. This can be examined using all three moderation-of-process types (see Figure 5). Specifically, if the job allows no risk-taking opportunities, the effect of excitement-seeking should be null if risk taking is the primary mechanism (or negative given other processes). This would be a Type 1 moderator because it affects the link to risk taking. Alternatively, if risk taking within a job is algorithmic (i.e., determined by policy) versus clinical (i.e., determined by incumbents’ judgments), one should see stronger validities in the latter case if risk taking is the mediator. Degree of risk-taking policy would be a Type 2 moderator, illustrated by the direct effect of the moderator on the mediator. Finally, if risk taking is hedged (or not) or can affect job performance measures differentially (i.e., positively affect one, not affect another, or negatively affect a third), then evidence of moderation implicates risk taking as a mediator between excitement-seeking and job performance via Type 3 by affecting the link from risk taking to performance.

Figure 5.

Model of excitement-seeking on performance via risk taking.

Unfortunately, the suppositions one might draw from existing moderator studies are not likely to be internally valid in terms of mediational hypotheses. This is because most of these existing studies were not designed for the purpose of testing mediation. Instead, the research question most moderator studies seek to address is: Does this effect generalize across condition or level of the moderator? This is an important question, to be sure, yet, when moderation is found one might ask: Why does the effect differ? In particular, does the finding of moderation reveal something about the underlying mechanism responsible for the transfer of variance in x to y? When considering this question, one is likely to find that tests of generalizability (e.g., does gender matter) might provide clues but are not clean manipulations of a mediational process because alternative mediational explanations for the moderation pattern observed were not considered. This issue motivates Steps 2 and 3.

Step 2: Elaborate the Mechanism

Step 2 begins with an elaboration of the mediating mechanism under consideration. An initial issue here is: Can a measure or a manipulation of the mechanism be obtained? If so, some form of SMAA is likely to be viable for examining mediation. Moreover, if the mediator can be manipulated independently from a manipulation of x, then a causal chain design is viable. However, one should also consider what might affect the workings of a mechanism by either interfering or improving it. Moreover, a researcher should consider what might affect the links to or from the mechanism and the x and y variables, respectively. In addition, one should consider whether the mechanism or the moderators of the mechanism (and or links to/from) can be manipulated or measured. If some moderators can be conceived, moderation-of-process designs are possible. Ideally, a researcher should consider multiple operationalizations of all of the aforementioned and compile a list of the various designs that are possible, given the results of this step.

Step 3: Articulate Alternative Explanations

Step 3 involves considering alternative explanations for relationships found and predicted. For an example, consider the self-efficacy as mediator between training and performance hypothesis we mentioned previously. This causal ordering (see Figure 6a) has been questioned by several researchers (Mathieu & Taylor, 2006; Sitzmann & Yeo, 2013; Stone-Romero & Rosopa, 2008; Vancouver, 2005). For example, Mathieu and Taylor (2006) described arguments for why self-efficacy might cause training (i.e., high self-efficacy individuals expect to get more out of training and therefore are more likely to volunteer for it), which affects performance (see Figure 6b), perhaps via skill acquisition (i.e., capacity change). They also suggested that because performance is a cause of self-efficacy (Bandura, 1997), the order of the variables could be completely reversed (Figure 6c). However, perhaps the most reasonable view is that training causes positive changes in capacity (i.e., knowledge and skills), but it is the changes in capacity, not self-efficacy, that positively affect performance and self-efficacy (Heggestad & Kanfer, 2005; see Figure 6d).

Figure 6.

Alternative causal models describing the associations among training, self-efficacy, and performance.

Given these alternative explanations, a researcher might begin by randomly assigning individuals to training or placebo groups or use a pre-post design. These would create variance on the exogenous variable (x) while removing the possibility that self-efficacy causes training status in the study. We can then combine the hypothesized model and its primary remaining alternative into the path diagram illustrated in Figure 7. This model acknowledges that training can change capacity, self-efficacy directly (path “a”) and indirectly (paths “b” and “c”), and performance indirectly through self-efficacy (paths “a” and “d” or paths “b,” “c,” and “d”), and through capacity change only (paths “b” and “e”).

Figure 7.

Elaborated model of training’s effect on self-efficacy and performance.

To test the a-d path of training on self-efficacy and performance, one might create a sham training program that has no effect on capacity (i.e., path “b” is blocked). The training can be presented as real or sham to create a Type 1 moderator that severs the link between training and self-efficacy when training is presented as a sham (see Figure 7; Study 1). To test the b-c-d path, one might provide training that changes capacity (path “b”) but hide (or not) information of capacity change. This would also be a Type 1 moderator (see Figure 7; Study 2). It should lead half the participants to believe that they are maintaining, as opposed to increasing, their capacity over time (i.e., performance does not appear to be increasing and thus self-efficacy is not increasing), whereas the other half would believe they are getting stronger (i.e., self-efficacy is increasing). Meanwhile, both groups would get stronger via the training.

We should note that including the pretraining performance would be important for increasing the power of the tests described previously and prevent spurious conclusions if the SMAA were applied (i.e., inflating path “d” and thus the indirect effect derived from multiplying “c” times “d”). That is, the SMAA can be applied on the group where the level of the moderator is such that it allows the process to occur (i.e., the knowledge condition), or a moderated, mediational analysis can be used on the entire sample (Edwards & Lambert, 2007; Muller et al., 2005), but both of these procedures would likely produce biased indirect effect size estimates if pretest performance is not included in the models because pretraining capacity differences would likely be reflected in self-efficacy measures. Indeed, this is an example of the more general issue of backdoor paths described in the causal inferences literature (e.g., Morgan & Winship, 2007). Specifically, backdoor paths can inflate direct and indirect effect size estimates when not blocked via statistical (e.g., control or instrumental variables) or design methods (e.g., random assignment).

The self-efficacy example illustrates the potential complexity of the processes involved in determining relationships between causes and effects. Elaboration of mechanism is key to understanding findings that appear to support theories of mediation, giving clues for moderators that might be examined or understand why moderation was found in past studies, and determining the diagnostic value of designs and statistical tests. One excellent way to elaborate mechanism is via computational models of the processes hypothesized (Vancouver & Weinhardt, 2012; Weinhardt & Vancouver, 2012). Such models can validate the logic of the processes and help determine the designs needed to address alternative explanations developed in Step 3 (Vancouver, Tamanini, & Yoder, 2010). These types of models are likely particularly relevant when feedback processes are involved because they immediately evoke a potential alternative explanation (i.e., reverse causality). Moreover, many tests of causal paths assume simple, linear processes. Complex processes like redundancies and compensatory processes (e.g., multiple behaviors available to restore equitable treatment perceptions), opposing processes (e.g., self-concept and task discrepancies arising from feedback valence can positively and negative affect performance via different processes; Kluger & DeNisi, 1996), or nonlinear relationships (e.g., the arousal-performance relationship hypothesize by Yerkes & Dodson, 1908) can interfere with interpretations. Monte Carlo simulations of computational models can help highlight what patterns of results can distinguish alternative models.

Conclusion

The use of empirical methods to examine explanations of the processes by which one variable (x) causes another (y) is a mainstay of science. In recent years, the SMAA has dominated psychology and organizational science (Wood et al., 2008). Typically, a myopic approach is limiting in terms of precision, generalizability, or rigor (Runkel & McGrath, 1972). In this case, a primary limitation of the SMAA is the rigor of the designs in which it is used (Stone-Romero & Rosopa, 2004). Meanwhile, the causal chain approach, which maximizes rigor, can suffer in terms of precision and generalizability as well as practicality. Both approaches require that the explanations have some measureable property to assess. To address these limitations, we advocate using the moderation-of-process approach, which seems to have been lost in the shuffle but can complement the other approaches.

We have also suggested a few more changes to the way mediational processes are examined in the organizational literature. First, we have highlighted the notion that mediation is about an explanation, not merely a variable. Explanations often need elaboration—an elaboration that should provide a clear description of the processes or mechanism being hypothesized. Such elaboration seems rare in most studies of mediation. Second, we have emphasized that the processes might involve variables that are not easily measured. Such a condition should not prevent speculation on, or testing of, the mechanism. That is, the moderation-of-process approach provides a possible way to implicate a mechanism that cannot be investigated using the other methods because they require measuring a key property of the mechanism. That said, failing to measure a key property of the mechanism when one is available might undermine the confidence others would have for the claims one was trying to make.

A third change, often advocated by others (e.g., Edwards, 2010; Leavitt, Mitchell, & Peterson, 2010), is that researchers or groups of researchers conduct programs of study that examine theoretical concepts across multiple studies. Given that mediating mechanisms are gleaned from the discipline’s theoretical landscape, systematic examination of mediating mechanisms is likely to lead to a cumulative science. Moreover, multiple moderation-of-process designs, perhaps coupled with integrated moderated mediational analysis (Edwards & Lambert, 2007) and experimental causal chain studies might be operationalized to provide precision, rigor, and generalizability to specific explanations (Runkel & McGrath, 1972). Precision would be enhanced to the degree mechanisms are clearly elaborated and those elaborations facilitate understanding how measurable properties relate to the process, how moderators might affect the processes, or more specific predictions (Edwards & Berry, 2010). Rigor is especially important when alternative explanations might exist for a finding of mediation, which is likely given the kinds of alternative explanations (e.g., unmeasured third variables, reverse causality, order of direction; James & Brett, 1984) that might exist when mediational hypotheses are supported via the SMAA. More specifically, tests of mediating mechanisms require multiple studies, preferably from multiple researchers applying multiple paradigms and from multiple perspectives.

Alternatively, or additionally, one might pivot the program of research suggested previously. That is, generalizability, and a cumulative science, is more likely to occur if we realize that explanations for x-y relationships tend to arise from a relatively small set of theoretical perspectives. Thus, a program of research might use an effective set of moderators of a process across several x-y relationships that presume to involve the same basic process, instead of (or in addition to) using several studies to examine a specific x-y relationship. For example, the field has used a dual-task paradigm for examining attentional process’ role in many x-y relationships (Pashler, 1994). To the extent such a program of research is successful, one might have also tested a general, practical intervention that enhances or mitigates desired or undesired outcomes. In fact, we might be able to solve all the world’s problems just by … wait, perhaps such sentiments call for a little moderation. Let’s just conduct some studies and go from there.

Footnotes

Acknowledgments

We thank Charlie M. Thompson for help on an earlier version of this paper and Julie A. Suhr for a review of the completed manuscript. We dedicate this paper to the memory of Lawrence R. James, whose work on mediational analysis has been instrumental to the field.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Notes

References

Adam

Shirako

(2013). Not all anger is created equal: The impact of the expresser’s culture on the social effects of anger in negotiations. Journal of Applied Psychology, 98, 785–798.

Aguinis

Beaty

J. C.

Boik

R. J.

Pierce

C. A.

(2005). Effect size and power in assessing moderating effects of categorical variables using multiple regression: A 30-year review. Journal of Applied Psychology, 90, 94–107.

Bandura

(1986). Social foundations of thought and action: A social cognitive theory. Englewood Cliffs, NJ: Prentice-Hall.

Bandura

(1997). Self-efficacy: The exercise of control. New York, NY: W. H. Freeman & Co, Publishers.

Bandura

Wood

R. E.

(1989). Effect of perceived controllability and performance standards on self-regulation of complex decision making. Journal of Personality and Social Psychology, 56, 805–814.

Baron

R. M.

Kenny

D. A.

(1986). The moderator-mediator variable distinction in social psychological research: Conceptual, strategic, and statistical considerations. Journal of Personality and Social Psychology, 51, 1173–1182.

Bobko

(1986). A solution to some dilemmas when testing hypotheses about ordinal interactions. Journal of Applied Psychology, 71, 323–326.

Cherrington

D. J.

Reitz

H. J.

Scott

W. E.

(1971). Effects of contingent and noncontingent reward on the relationship between satisfaction and task performance. Journal of Applied Psychology, 55, 531–536.

Cole

D. A.

Maxwell

S. E.

(2003). Testing mediational models with longitudinal data: Questions and tips in the use of structural equation modeling. Journal of Abnormal Psychology, 112, 558–577.

10.

Colquitt

J. A.

LePine

J. A.

Noe

R. A.

(2000). Toward an integrative theory of training motivation: A meta-analytic path analysis of 20 years of research. Journal of Applied Psychology, 85, 678–707.

11.

Davidson

O. B.

Eden

(2000). Remedial self-fulfilling prophecy: Two field experiments to prevent Golem effects among disadvantaged women. Journal of Applied Psychology, 85, 386–398.

12.

Dawson

J. F.

(2014). Moderation in management research: What, why, when, and how. Journal of Business and Psychology, 29, 1–19.

13.

DeShon

R. P.

(2012). Multivariate dynamics in organizational science. In Kozlowski

S. W. J.

(Ed.), The Oxford handbook of industrial and organizational psychology. New York, NY: Oxford University Press.

14.

Eden

Zuk

(1995). Seasickness as a self-fulfilling prophecy: Raising self-efficacy to boost performance at sea. Journal of Applied Psychology, 80, 628–635.

15.

Edwards

J. R.

(2010). Reconsidering theoretical progress in organizational and management research. Organizational Research Methods, 13, 615–619.

16.

Edwards

J. R.

Berry

J. W.

(2010). The presence of something or the absence of nothing: Increasing theoretical precision in management research. Organizational Research Methods, 13 (4), 668–689.

17.

Edwards

J. R.

Lambert

L. S.

(2007). Methods for integration moderation and mediation: A general analytical framework using moderated path analysis. Psychological Methods, 12, 1–22.

18.

Freedman

D. A.

(2004). Graphical models for causation, and the identification problem. Evaluation Review, 28, 267–293.

19.

Griffeth

R. W.

Hom

P. W.

Gaertner

(2000). A meta-analysis of antecedents and correlates of employee turnover: Update, moderator tests, and research implications for the next millennium. Journal of Management, 26, 463–488.

20.

Heggestad

E. D.

Kanfer

(2005). The predictive validity of self-efficacy in training performance: Little more than past performance. Journal of Experimental Psychology: Applied, 11, 84–97.

21.

Holmbeck

G. N.

(1997). Toward terminological, conceptual, and statistical clarity in the study of mediators and moderators: Examples from the child-clinical and pediatric psychology literatures. Journal of Consulting and Clinical Psychology, 65, 599–610.

22.

Hough

L. M.

Schneider

R. J.

(1996). Personality traits, taxonomies, and applications in organizations. In Murphy

K. R.

(Ed.), Individual differences and behavior in organizations (pp. 31–88). San Francisco, CA: Jossey-Bass.

23.

James

L. R.

(2008). On the path to mediation. Organizational Research Methods, 11, 359–363.

24.

James

L. R.

Brett

J. M.

(1984). Mediators, moderators and tests of mediation. Journal of Applied Psychology, 69, 307–321.

25.

James

L. R.

Mulaik

S. A.

Brett

J. M.

(1982). Causal analysis: Assumptions, models, and data. Beverly Hills, CA: Sage.

26.

Judge

T. A.

Rodell

J. B.

Klinger

R. L.

Simon

L. S.

Crawford

E. R.

(2013). Hierarchal representations of the five-factor model of personality in predicting job performance: Integrating three organizing frameworks with two theoretical perspectives. Journal of Applied Psychology, 98, 875–925.

27.

Kanfer

Ackerman

P. L.

(1989). Motivation and cognitive abilities: An integration of developmental, differential, and cognitive perspectives. Journal of Applied Psychology, 74, 657–690.

28.

Kenny

D. A.

(2008). Reflections on mediation. Organizational Research Methods, 11, 353–358.

29.

Kluger

A. N.

DeNisi

(1996). The effects of feedback interventions on performance: A historical review, a meta-analysis, and a preliminary feedback intervention theory. Psychological Bulletin, 119, 254–284.

30.

Kraemer

H. C.

Wilson

T. G.

Fairburn

C. G.

Agras

W. S.

(2002). Mediators and moderators of treatment effects in randomized clinical trials. Archives of General Psychiatry, 59, 877–883.

31.

Lau

Cheung

G. W.

(2012). Estimating and comparing specific mediation effects in complex latent variable models. Organizational Research Methods, 15, 3–16.

32.

Leavitt

Mitchell

T. R.

Peterson

(2010). Theory pruning: Strategies to reduce our dense theoretical landscape. Organizational Research Methods, 13 (4), 644–667.

33.

Lewin

(1954). Behavior and development as a function of the total situation. In Carmichael

(Ed.), Manual of child psychology (pp. 918–970). New York, NY: Wiley.

34.

Lindsey

D. H.

Brass

D. J.

Thomas

J. B.

(1995). Efficacy-performance spirals: A multilevel perspective. Academy of Management Review, 20, 645–678.

35.

Locke

E. A.

(1982). Relation of goal level to performance with a short work period and multiple goal levels. Journal of Applied Psychology, 67, 512–514.

36.

Locke

E. A.

(2007). The case for inductive theory building. Journal of Management, 33, 867–890.

37.

MacCorquodale

Meehl

P. E.

(1948). On a distinction between hypothetical constructs and intervening variables. Psychological Review, 55, 95–107.

38.

MacKinnon

D. P.

Fairchild

A. J.

(2009). Current directions in mediation analysis. Current Directions in Psychological Science, 18, 16–20.

39.

MacKinnon

D. P.

Fairchild

A. J.

Fritz

M. S.

(2007). Mediation analysis. Annual Review of Psychology, 58, 593–614.

40.

MacKinnon

D. P.

Krull

J. L.

Lockwood

C. M.

(2000). Equivalence of the mediation, confounding and suppression effect. Prevention Science, 1, 173–181.

41.

MacKinnon

D. P.

Lockwood

C. M.

Hoffman

J. M.

West

S. G.

Sheets

(2002). A comparison of methods to test mediation and other intervening variable effects. Psychological Methods, 7, 83–104.

42.

Mathieu

J. E.

DeShon

R. P.

Bergh

D. D.

(Eds.) (2008). Mediational inferences in organizational research: Then, now and beyond [Special section]. Organizational Research Methods, 11, 203–223.

43.

Mathieu

J. E.

Taylor

S. R.

(2006). Clarifying conditions and decision points for mediational type inferences in organizational behavior. Journal of Organizational Behavior, 27, 1031–1056.

44.

Maxwell

S. E.

Cole

D. A.

(2007). Bias in cross-sectional analyses of longitudinal mediation. Psychological Methods, 12, 23–44.

45.

Morgan

S. L.

Winship

(2007). Counterfactuals and causal inference: Methods and principles for social research. Boston, MA: Cambridge University Press.

46.

Muller

Judd

C. M.

Yzerbyt

V. Y.

(2005). When moderation is mediated and mediation is moderated. Journal of Personality and Social Psychology, 89, 852–863.

47.

Orne

M. T.

(1962). On the social psychology of the psychological experiment: With particular reference to demand characteristics and the implications. American Psychologist, 17, 776–783.

48.

Pashler

(1994). Dual-task interference in simple tasks: Data and theory. Psychological Bulletin, 116, 220–244.

49.

Porter

L. W.

Lawler

E. E.,

III (1968). Managerial attitudes and performance. Homewood, IL: Irwin-Dorsey.

50.

Rubin

D. B.

(2004). Teaching statistical inference for causal effects in experiments and observational studies. Journal of Educational and Behavioral Statistics, 29, 343–367.

51.

Runkel

P. J.

McGrath

J. E.

(1972). Research on human behavior: A systematic guide to method. New York, NY: Holt, Rinehart, and Winston.

52.

Shadish

W. R.

Cook

T. D.

Campbell

D. T.

(2002). Experimental and quasi-experimentation designs for generalized causal inference. Boston, MA: Houghton Mifflin.

53.

Shaver

J. M.

(2005). Testing for mediating variables in management research: Concerns, implications, and alternative strategies. Journal of Management, 31, 330–353.

54.

Shieh

(2009). Detecting interaction effects in moderated multiple regression with continuous variables: Power and sample size considerations. Organizational Research Methods, 12, 510–528.

55.

Sitzmann

Yeo

(2013). A meta-analytic investigation of the within-person self-efficacy domain: Is self-efficacy a product of past performance or a driver of future performance? Personnel Psychology, 66, 531–568.

56.

Spencer

S. J.

Zanna

M. P.

Fong

G. T.

(2005). Establishing a causal chain: Why experiments are often more effective than mediational analyses in examining psychological processes. Journal of Personality and Social Psychology, 89, 845–851.

57.

Steele

C. M.

Liu

T. J.

(1983). Dissonance processes as self-affirmation. Journal of Personality and Social Psychology, 45, 5–19.

58.

Stone-Romero

E. F.

Anderson

L. E.

(1994). Relative power of moderated multiple regression and the comparison of subgroup correlation coefficients for detecting moderating effects. Journal of Applied Psychology, 79, 354–359.

59.

Stone-Romero

E. F.

Rosopa

P. J.

(2004). Inference problems with hierarchical multiple regression-based tests of mediating effects. Research in Personnel and Human Resource Management, 23, 249–290.

60.

Stone-Romero

E. F.

Rosopa

P. J.

(2008). The relative validity of inferences about mediation as a function of research design characteristics. Organizational Research Methods, 11, 326–352.

61.

Stone-Romero

E. F.

Rosopa

P. J.

(2010). Research design options for testing mediation models and their implications for facets of validity. Journal of Managerial Psychology, 25, 697–712.

62.

Stone-Romero

E. F.

Rosopa

P. J.

(2011). Experimental tests of mediation: Prospects, problems, and some solutions. Organizational Research Methods, 14, 631–646.

63.

Strube

M. J.

Bobko

(1989). Testing hypotheses about ordinal interactions: Simulations and further comments. Journal of Applied Psychology, 74, 247–252.

64.

Taylor

A. B.

MacKinnon

D. P.

Tein

(2008). Tests of the three-path mediated effect. Organizational Research Methods, 11, 241–269.

65.

Vancouver

J. B.

(2005). The depth of history and explanation as benefit and bane for psychological control theories. Journal of Applied Psychology, 90, 38–52.

66.

Vancouver

J. B.

Tamanini

K. B.

Yoder

R. J.

(2010). Using dynamic computational models to reconnect theory and research: Socialization by the proactive newcomer example. Journal of Management, 36, 764–793.

67.

Vancouver

J. B.

Tischner

E. C.

(2004). The effect of feedback sign on task performance depends on self-concept discrepancies. Journal of Applied Psychology, 89, 1092–1098.

68.

Vancouver

J. B.

Weinhardt

J. M.

(2012). Modeling the mind and the milieu: Computational modeling for micro-level organizational researchers. Organizational Research Methods, 15, 602–623.

69.

Wason

P. C.

(1968). Reasoning about a rule. Quarterly Journal of Experimental Psychology, 20, 273–281.

70.

Weinhardt

J. M.

Vancouver

J. B.

(2012). Intelligent interventions. Perspectives in Industrial-Organizational Psychology, 5, 179–182.

71.

Wood

R. E.

Goodman

J. S.

Beckmann

Cook

(2008). Mediation testing in management research: A review and proposals. Organizational Research Methods, 11, 270–295.

72.

Yerkes

R. M.

Dodson

J. D.

(1908). The relation of strength of stimulus to rapidity of habit-formation. Journal of Comparative Neurology and Psychology, 18, 459–482.