A Simple Technique Assessing Ordinal and Disordinal Interaction Effects

Abstract

Previous research explicates ordinal and disordinal interactions through the concept of the “crossover point.” This point is determined via simple regression models of a focal predictor at specific moderator values and signifies the intersection of these models. An interaction effect is labeled as disordinal (or ordinal) when the crossover point falls within (or outside) the observable range of the focal predictor. However, this approach might yield erroneous conclusions due to the crossover point’s intrinsic nature as a random variable defined by mean and variance. To statistically evaluate ordinal and disordinal interactions, a comparison between the observable range and the confidence interval (CI) of the crossover point is crucial. Numerous methods for establishing CIs, including reparameterization and bootstrap techniques, exist. Yet, these alternative methods are scarcely employed in social science journals for assessing ordinal and disordinal interactions. This note introduces a straightforward approach for calculating CIs, leveraging an extension of the Johnson–Neyman technique.

Keywords

interaction effect crossover point moderation effect ordinal interactions disordinal interactions

Moderation effects are commonly assessed through interaction effects between a focal predictor and a moderator in moderated regression models. If a significant interaction effect exists, it is recommended to subsequently conduct a “simple slopes analysis” (e.g., Irwin & McClelland, 2001; Jaccard & Turrisi, 2003; MacCallum et al., 2002; Spiller et al., 2013), also known as a “pick-a-point approach” (Rogosa, 1980, 1981) or a “conditional effect” (Darlington & Hayes, 2017; Hayes, 2022; Preacher et al., 2007).

A simple slopes analysis examines the “simple slopes” indicating slopes in simple regression models of a focal predictor at specific values of a moderator (Irwin & McClelland, 2001; Jaccard & Turrisi, 2003; MacCallum et al., 2002; Park & Yi, 2022). It assesses the statistical significance of these simple slopes with their confidence intervals (CIs) obtained by the Johnson–Neyman (JN) technique (Johnson & Neyman, 1936). Various statistical packages such as Mplus, the JN function in R, and the PROCESS macro (Hayes, 2022) provide CIs for simple slopes identified through the JN technique. Lin (2020) offers computer code examples.

As a subsequent analysis, researchers may explore ordinal and disordinal interaction effects. These effects are defined based on simple regression models of a focal predictor at specific values of a moderator (e.g., Cohen et al., 2003; Murphy & Russell, 2017; Park & Yi, 2022; Widaman et al., 2012). When the predicted simple regression models intersect at a specific value of the focal predictor, it is referred to as a “crossover point” of the focal predictor.

A disordinal (or ordinal) interaction indicates an interaction that leads to a crossover of predicted simple regression models within (or outside) the observable range of a focal predictor (e.g., Cohen et al., 2003; Darlington & Hayes, 2017; Hayes, 2022; Preacher et al., 2007). The crossover point is defined as the value of the focal predictor at which the simple slope of a moderator is zero. Given that a focal predictor is measured with its minimum and maximum points, its values are naturally confined within this range. Thus, it is logical to investigate a crossover point falling within this interval.

The crossover point is a random variable characterized with its mean and variance. However, unlike each regression coefficient in a moderated regression model, the crossover point cannot be assessed using a normal distribution. Regression analysis assumes that each regression coefficient follows a normal distribution. In contrast, the crossover point is represented as a ratio function of regression coefficients in the moderated regression model. The ratio function does not follow a normal distribution. As a result, previous research has introduced specialized methods for examining the crossover point (e.g., Lee et al., 2015; Widaman et al., 2012).

To statistically evaluate ordinal and disordinal interactions, it is required to compare the CI of a crossover point with the observable range of the focal predictor. Widaman et al. (2012) discuss the importance of the CI of the focal predictor and propose a “reparameterization technique” to establish the CI of the crossover point using a reparametrized moderated regression model. Lee et al. (2015) illustrate the feasibility of determining the CI through alternative approaches, including the reparameterization technique, the delta method, the bootstrap technique, and the Fieller method.

If the CI of a crossover point falls within the observable range of a focal predictor, one can conclude that the interaction effect is disordinal. In contrast, if the CI falls outside the observable range, the interaction effect is considered ordinal. However, when the CI spans both within and outside the observable range, it is not possible to statistically determine whether the interaction is ordinal or disordinal. This holds true even if the crossover point itself falls within the observable range while the CI is partially within and partially outside. Similarly, the presence of a crossover point outside the observable range with a CI spanning both sides does not allow for a statistical distinction between ordinal and disordinal interactions.

Nevertheless, it is not easy to find articles that test ordinal and disordinal interaction effects using the alternative approaches in social science journals. Many researchers tend to visually interpret these effects without employing formal tests (e.g., Andrews et al., 2021; Flurry et al., 2021; Hurtak et al., 2022; Nejad et al., 2016; Padgett et al., 2020; Vickery et al., 2016). This trend is likely influenced by the prevailing standard procedure recommended in the moderation literature.

The established procedure involves analyzing interaction effects and simple slopes through a combination of moderated regression analysis and simple slopes analysis. In this approach, moderated regression analysis is utilized to explore interaction effects, while simple slopes analysis is employed to examine simple slopes. Simple slopes analysis identifies the interval of significance of these slopes using the JN technique. Notably, the JN technique does not require addition estimation. It relies on summary statistics such as estimated regression coefficients and their variance–covariance matrix. Most statistical packages offer an option for obtaining these summary statistics in moderated regression analysis.

While the standard procedure for moderation analysis is well-known to most researchers, alternative methods for assessing ordinal and disordinal interactions tend to be unfamiliar. The purpose of this note is to introduce a “simple technique” aimed at determining CIs of crossover points, building on the JN technique commonly used in the standard procedure. Specifically, this simple technique involves an extension of the JN technique. In this note, we refer to the conventional JN technique and its extension as the “basic JN technique” and “extended JN technique,” respectively. The extended JN technique examines simple slopes of a moderator unlike the basic JN technique. It pinpoints the interval of nonsignificance for these slopes, indicating the range of the focal predictor that falls outside the interval of significant simple slopes. Then, the technique specifies the CI of a crossover point using the interval of nonsignificant simple slopes.

This note empirically compares the extended JN technique with previous approaches. As a result of this comparison, the extended JN technique is recommended as an alternative approach for finding CIs of crossover points. Similar to the basic JN technique, the extended technique avoids the need for additional estimation and relies on summary statistics. Thus, one can examine interaction effects and simple slopes through a combination of moderated regression analysis and simple slopes analysis. By combining the basic and extended JN techniques, researchers can effectively scrutinize ordinal and disordinal interactions as well as simple slopes.

Ordinal and Disordinal Interactions in the Literature

Simple Slopes

Let us consider a standard moderated regression model written as

y = β_{0} + β_{1} x_{1} + β_{2} x_{2} + β_{3} x_{1} x_{2} + ∊,

where x ₁ is a value of the focal predictor X ₁ (i.e., $x_{1} \in X_{1}$ ), x ₂ is a value of the moderator X ₂ (i.e., $x_{2} \in X_{2}$ ), y is a value of the dependent variable Y (i.e., $y \in Y$ ), $∊$ is an error term [i.e., $∊ \sim N (0, σ^{2})$ ], $β_{0}$ is the intercept, $β_{1}$ is the regression coefficient of X ₁, $β_{2}$ is the regression coefficient of X ₁, and $β_{3}$ is the regression coefficient of the interaction term between X ₁ and of X ₂.

It can be rewritten as

y = β_{0} + β_{2} x_{2} + (β_{1} + β_{3} x_{2}) x_{1} + ∊ .

The simple slope of the focal predictor (X ₁) is expressed as $β_{1} + β_{3} x_{2}$ at a specific value of the moderator (X ₂) in Equation 2. It represents the combined effect of the main effect of X ₁ and its interaction effect with X ₂. Simple slopes analysis assesses these simple slopes with their CIs obtained using the JN technique. The JN technique calculates the CIs of simple slopes at specific values of the moderator.

Ordinal and Disordinal Interactions

One can examine the crossover point of a focal predictor under three basic assumptions: linear relationships among variables, uniform measurement precision and equal intervals across the range of each variable, and the observed range of the focal predictor corresponding to its population range (Widaman et al., 2012). This note considers a crossover point under these assumptions.

Let us explain a crossover point using a moderated regression model (Equation 1), where the focal predictor (X ₁) is quantitative and the moderator (X ₂) is qualitative (coded with 0 and 1). The crossover point can be found by setting up two regression models at two levels of the moderator (i.e., $0 or 1$ ). It represents the value of the focal predictor at which two regression lines intersect.

More specifically, the crossover point can be represented as the specific value that satisfies the following equation:

β_{0} + β_{1} x_{1} + β_{2} (0) + β_{3} (0) = β_{0} + β_{1} x_{1} + β_{2} (1) + β_{3} (x_{1}) .

This equation is satisfied at the specific value of a focal predictor denoted as

x_{1} = - β_{2} / β_{3} = c_{12},

where $c_{12}$ indicates the crossover point of the focal predictor (Cohen et al., 2003; Murphy & Russell, 2017; Park & Yi, 2022; Widaman et al., 2012). Figure 1 illustrates the ordinal and disordinal interactions.

Figure 1.

Exemplary plots of simple slopes of X ₁ at values of X ₂.

Alternative Approaches

Widaman et al. (2012) assert that it is simple to identify the crossover point of a focal predictor with a moderated regression model. However, they underscore the challenge of computing the CI of a crossover point. The crossover point is expressed as a ratio of two regression coefficients, resembling the structure of an indirect effect in mediation analysis, which involves the product of two regression coefficients. Both the ratio and the product distributions exhibit nonnormal characteristics (Lee et al., 2015; Marsaglia, 1965, 2006).

Widaman et al. (2012) derive a reparameterized moderated regression model in which the crossover point is a regression coefficient. They demonstrate the feasibility of estimating both the crossover point and its CI through this reparameterized technique. The estimation process for the reparameterized model involves a nonlinear regression program.

Lee et al. (2015) extend the discussion by proposing four alternative techniques for determining CIs of crossover points: the reparameterization technique (Widaman et al., 2012), the bootstrap technique (Efron, 1979), the delta method (Cox, 1990; Rao, 1973), and the Fieller (1932, 1954) method. The bootstrap technique encompasses variations such as the normal-theory bootstrap, percentile bootstrap, and bias-corrected accelerated bootstrap (BCa; Fox, 2016, pp. 655–658). It is frequently used in mediation analysis. One can obtain a bootstrap distribution of the statistic represented as a composite function and then calculate the standard error (SE) of the statistic. Accordingly, one can calculate the CI of a crossover point with the SE obtained from the bootstrap technique. The delta method indicates a general approach approximating the asymptotic SE of a crossover point (expressed as a ratio of two regression coefficients), permitting the derivation of the CI through an approximate equation. The Fieller method involves the calculation of CI using a χ² distribution with 1 degree of freedom. This method yields two closed-form solutions, serving as lower and upper bounds of the CI.

Lee et al. (2015, p. 245) empirically compare these alternative approaches using Monte Carlo simulations and conclude that “statistical inference using CIs to distinguish ordinal and disordinal interaction requires sample sizes more than 500 to be able to provide sufficiently narrow CIs to identify the location of the crossover point.”

A Proposed Approach: Extended JN Technique

The Simple Slope of a Moderator

In this section, we introduce a user-friendly technique to determine the CI of a crossover point using the JN technique. Equation 1 can be rewritten as

y = β_{0} + β_{1} x_{1} + (β_{2} + β_{3} x_{1}) x_{2} + ∊,

where $β_{2} + β_{3} x_{1}$ indicates the simple slope of the moderator (X ₂) at a specific value of the focal predictor (X ₁). It is important to note that this differs from the simple slope of the focal predictor in simple slopes analysis (Equation 2).

We recognize that when the simple slope of a moderator is zero (i.e., $β_{2} + β_{3} x_{1} = 0$ ), the moderator cannot alter the simple slope of the focal predictor. This holds true regardless of whether the moderator is quantitative or qualitative. At $x_{1} = - β_{2} / β_{3}$ , the simple slope of the moderator becomes zero, and the simple slope of the focal predictor is simply $β_{1}$ .

Thus, one can conclude that

S (x_{1}) = {\hat{β}}_{2} + {\hat{β}}_{3} x_{1} = 0 and {\hat{c}}_{12} = - {\hat{β}}_{2} / {\hat{β}}_{3},

where $S (x_{1})$ denotes the interval of nonsignificance of simple slopes. This means that the CI of a crossover point is identical to the interval of nonsignificance of simple slopes. This interval represents the range of the focal predictor, excluding the interval of significant simple slopes.

Deriving CIs of Crossover Points

Assessing whether the simple slope of a moderator is zero involves a t-statistic:

t = \frac{S (x_{1})}{S E (S (x_{1}))} = \frac{{\hat{β}}_{2} + {\hat{β}}_{3} x_{1}}{\sqrt{V a r ({\hat{β}}_{2} + {\hat{β}}_{3} x_{1})}} .

The t-statistic becomes zero at the crossover point (i.e., $x_{1} = {\hat{c}}_{12}$ ) because $S (x_{1}) = {\hat{β}}_{2} + {\hat{β}}_{3} x_{1} = 0$ at $x_{1} = {\hat{c}}_{12} = - {\hat{β}}_{2} / {\hat{β}}_{3}$ . The CI of $S (x_{1}) = {\hat{β}}_{2} + {\hat{β}}_{3} x_{1} = 0$ can be specified with its lower and upper bounds, where the absolute t-statistics at the two bounds are set to be identical to the critical t value.

Specifically, the closed-form solutions for the CI can be derived through these steps. First, one can find lower and upper bounds of the $100 \times (1 - α)$ CI for the t-statistic (Equation 7), with $α$ as the significance level. The critical t value, $t_{α / 2}^{2} \equiv t_{(α / 2, d f)}^{2}$ , is determined by $d f = N - k - 1$ , where N is the sample size and k is the number of predictors in the moderated regression model. The squared t-statistics at the lower and upper bounds of the CI are identical to the squared critical t value:

t^{2} = \frac{{({\hat{β}}_{2} + {\hat{β}}_{3} x_{1})}^{2}}{V a r ({\hat{β}}_{2} + {\hat{β}}_{3} x_{1})} = t_{α / 2}^{2} .

Equation 8 can be rewritten as

{({\hat{β}}_{2} + {\hat{β}}_{3} x_{1})}^{2} = t_{α / 2}^{2} V a r ({\hat{β}}_{2} + {\hat{β}}_{3} x_{1}),

where ${({\hat{β}}_{2} + {\hat{β}}_{3} x_{1})}^{2} = {\hat{β}}_{2}^{2} + 2 {\hat{β}}_{2} {\hat{β}}_{3} x_{1} + {\hat{β}}_{3}^{2} x_{1}^{2}$ and

$V a r ({\hat{β}}_{2} + {\hat{β}}_{3} x_{1}) = V a r ({\hat{β}}_{2}) + 2 C o v ({\hat{β}}_{2}, {\hat{β}}_{3}) x_{1} + V a r ({\hat{β}}_{3}) x_{1}^{2} .$

Second, one can derive a quadratic function of the focal predictor from Equation 8. The quadratic function is written as

A x_{1}^{2} + B x_{1} + C = 0,

where $A = {\hat{β}}_{3}^{2} - t_{α / 2}^{2} V a r ({\hat{β}}_{3})$ , $B = 2 [{\hat{β}}_{2} {\hat{β}}_{3} - t_{α / 2}^{2} C o v ({\hat{β}}_{2}, {\hat{β}}_{3})]$ , and $C = {\hat{β}}_{2}^{2} - t_{α / 2}^{2} V a r ({\hat{β}}_{2})$ .

One can see that A, B, and C are constants. Thus, the solutions for Equation 10 are expressed as the lower and upper bounds of the CI of a crossover point, where regression coefficients are given as their estimates and the critical t value is determined a priori. The lower and upper bounds of the CI are represented as

\frac{- B \pm \sqrt{B^{2} - 4 A C}}{2 A} .

Accordingly, Equation 11 serves as the $100 \times (1 - α)$ CI of a crossover point. In other words, one can find the crossover point of the focal predictor with Equation 6 and its CI with Equation 11. This approach connects the extended JN technique with the interval of nonsignificance of moderator simple slopes. Estimates of regression coefficients and their variance–covariance in Equation 11 can be obtained from commercial statistical packages. This implies that it is possible to obtain the CI of a crossover point without extra efforts.

It is notable that the extended JN technique and alternative approaches may not yield CIs of crossover points when ${\hat{β}}_{3}^{2} = 0$ and $V a r ({\hat{β}}_{3}) = 0$ . Lee et al. (2015, p. 249) point out that “this characteristic is well illustrated by Gleser–Hwang theorem” (Gleser & Hwang, 1987). This specific scenario can be excluded by examining crossover points in line with the standard procedure used in the moderation literature. Ordinarily, crossover points are analyzed when the interaction effect is significant.

If the interaction effect is significant, one may expect that simple regression models of the focal predictor will intersect within or outside the observed range of the focal predictor. However, Appendix A shows that the extended JN technique evaluates the crossover point of simple regression models while reflecting the correlation between the interaction effect and the moderator’s main effect. As such, the extended JN technique may conclude that a potential crossover point does not exist, even when the interaction effect is significant.

Specifically, it concludes that a crossover point does not exist if $B^{2} < 4 A C$ in Equation 11. In this case, one can statistically conclude that the simple regression models are parallel, even when the interaction effect is significant. Alternatively, it is possible to interpret that such an interaction is at least not disordinal, in view that the crossover point does not locate within the observable range of the focal predictor.

This interpretation is supported by Park and Yi (2023), who emphasize that simple slopes of a focal predictor at various moderator levels may not be statistically distinct, even with a significant interaction effect. If these simple slopes are statistically indistinct across the full range of the moderator, it signifies that the simple regression models of the focal predictor are parallel throughout the moderator’s range, even with a significant interaction effect. The term “simple slopes” here refers to the slopes of the focal predictor in simple regression models.

Multiple crossover points can be observed when a moderator is expressed as multiple dummy variables. The formulae in Equations 6 and 11 can be used to find and compute the CIs for multiple crossover points. Further details can be found in Supplemental Appendix B.

Differences With the Fieller Method

The Fieller method, as outlined in Lee et al. (2015), shares commonalities with the extended JN technique. Both approaches examine the simple slopes of a moderator. Specifically, the Fieller method uses the simple slopes in Equation 6, that is, $S (x_{1}) = {\hat{β}}_{2} + {\hat{β}}_{3} x_{1}$ .

However, a divergence arises in how the CI of a crossover point is derived. Unlike the extended JN technique, which employs a t-distribution, the Fieller method utilizes a χ² distribution specified as

t^{2} = \frac{S {(x_{1})}^{2}}{V a r (S (x_{1}))} = \frac{{({\hat{β}}_{2} + {\hat{β}}_{3} x_{1})}^{2}}{V a r ({\hat{β}}_{2} + {\hat{β}}_{3} x_{1})} \overset{χ^{2}}{\sim} (1),

where $S {(x_{1})}^{2} = {({\hat{β}}_{2} + {\hat{β}}_{3} x_{1})}^{2} = {\hat{β}}_{2}^{2} + 2 {\hat{β}}_{2} {\hat{β}}_{3} x_{1} + {\hat{β}}_{3}^{2} x_{1}^{2}$ and $V a r (S (x_{1})) = V a r ({\hat{β}}_{2} + {\hat{β}}_{3} x_{1}) = V a r ({\hat{β}}_{2}) + 2 C o v ({\hat{β}}_{2}, {\hat{β}}_{3}) x_{1} + V a r ({\hat{β}}_{3}) x_{1}^{2}$ .

From Equation 12, the Fieller method derives the following equation:

P r o b [\frac{{({\hat{β}}_{2} + {\hat{β}}_{3} x_{1})}^{2}}{V a r ({\hat{β}}_{2} + {\hat{β}}_{3} x_{1})} \leq k_{1 - α}] = 1 - α,

where $k_{1 - α}$ is a critical χ² value indicating the $(1 - α)$ percentile of a χ² distribution with 1 degree of freedom. The inequality equation in Equation 13 can be simplified to

A^{'} x_{1}^{2} + B^{'} x_{1} + C^{'} \leq 0,

where $A^{'} = {\hat{β}}_{3}^{2} - k_{1 - α} V a r ({\hat{β}}_{3})$ , $B^{'} = 2 [{\hat{β}}_{2} {\hat{β}}_{3} - k_{1 - α} C o v ({\hat{β}}_{2}, {\hat{β}}_{3})]$ , and $C^{'} = {\hat{β}}_{2}^{2} - k_{1 - α} V a r ({\hat{β}}_{2})$ . This inequality equation used in the Fieller method is very similar to the equality equation used in the extended JN technique (compare Equation 14 with Equation 10).

The solutions for Equation 14 denote the lower and upper bounds of the CI of a crossover point, with regression coefficients as estimates and a predetermined critical χ² value. Specifically, the two bounds of the CI in the Fieller method are represented as

\frac{- B^{′ 2} \pm \sqrt{B^{′ 2} - 4 A^{'} C^{'}}}{2 A^{'}} where A^{'} > 0.

However, unlike the extended JN technique, the Fieller method can lead to unbounded CIs of crossover points when $A^{'}$ is negative (Lee et al., 2015). Additionally, the squared t-statistic (t ²) in Equation 12 does not exactly follow a χ² distribution with 1 degree of freedom, in contrast to the Fieller method assumption.

Nevertheless, under the assumption of a sufficiently large sample size, the squared t-statistic can be approximated by the squared z-statistic, which does adhere to a χ² distribution with 1 degree of freedom. Consequently, the Fieller method can be justified with ample sample sizes. Notably, the Fieller method CIs for crossover points closely align with those from the extended JN technique when the interaction effect is significant and the sample size is adequately large. We explain these points in Supplemental Appendix C.

Lee et al. (2015, p. 248) underscore that “unbounded confidence intervals are not desirable because they are not comparable with the bounded confidence intervals from other methods.” However, regarding the examination of crossover points, the implications of unbounded CIs differ. Unbounded CIs signify that solutions for CIs are not provided by these methods, rather than unbounded CIs inherently undesirable. This limitation applies to both the Fieller method and the extended JN technique. Notably, the Fieller method fails to provide solutions for CIs when $A^{'} < 0$ or $B^{' 2} < 4 A^{'} C^{'}$ in Equation 15. In situations where the interaction effect is significant, the case of $A^{'} < 0$ is eliminated (see Supplemental Appendix C). Therefore, similar to the extended JN technique, if $B^{' 2} < 4 A^{'} C^{'}$ in Equation 15, the Fieller method does not yield solutions for CIs. In such instances, akin to the extended JN technique, the Fieller method concludes that simple regression models are parallel within and beyond the focal predictor range, even when the interaction effect is significant. Alternatively, one might interpret that such an interaction is, at least, not disordinal, resembling the stance of the extended JN technique.

Comparing Alternative Approaches

Table 1 provides a comprehensive comparison of alternative approaches based on three key criteria.

Criteria 1: Requirement for additional estimation

The alternative approaches are categorized according to whether they necessitate supplementary estimation efforts. The reparameterization technique and the bootstrap technique mandate additional estimation steps, such as nonlinear least estimation and bootstrap estimation. In contrast, the delta method, the Fieller method, and the extended JN technique circumvent the need for extra estimation.

Criteria 2: Statistical estimation versus mathematical calculation

The classification hinges on whether the approaches derive CIs through statistical estimation or mathematical calculation of summary statistics generated from moderated regression analysis. While the delta method, the Fieller method, and the extended JN technique involve mathematical calculation, the delta method’s calculation relies on approximations. Lee et al. (2015) observe the reparameterization technique as an approximation approach as well, with a conjecture that the two approximation techniques yield highly similar CIs with large sample sizes. In contrast, the Fieller method and the extended JN technique use closed-form solutions in calculation.

Criteria 3: Availability of closed-form solutions

The alternative approaches are categorized according to whether closed-form solutions for CIs can be derived. The Fieller method delivers CIs remarkably akin to those from the extended JN technique when ample sample sizes are present. However, the Fieller method lacks justification without sufficiently large sample sizes. Lee et al. (2015) underscore the significance of the Fieller method due to its advantage of employing closed-form solutions. Their Monte Carlo simulation concludes that sample sizes exceeding 500 are requisite for distinguishing ordinal and disordinal interactions. The bootstrap technique, which also warrants consideration, is recommended by Lee et al. for CIs of crossover points.

Table 1.

Comparison of Alternative Approaches

Method	Extra Estimation	Estimation or Calculation	Closed-Form Solutions
Reparameterization	Yes	Statistical estimation	No
Delta method	No	Mathematical calculation	No
Bootstrap technique	Yes	Statistical estimation	No
Fieller method	No	Mathematical calculation	Yes
Extended JN technique	No	Mathematical calculation	Yes

Notes. The delta method, the Fieller method, and the extended Johnson–Neyman (JN) technique find confidence intervals (CIs) of crossover points mathematically with summary statistics provided by moderated regression analysis. The delta method uses approximated solutions. The Fieller method and the extended JN technique use closed-form solutions. The solution by the extended JN technique calculates CIs while reflecting the size of samples used to estimate moderated regression models. In contrast, the solution by the Fieller method calculates CIs without reflecting the sample size. The two solutions become close to each other as the sample size increases. The Fieller method cannot be justified when the sample size is not sufficiently large.

The extended JN technique not only provides CIs of crossover points through mathematical calculation based on closed-form solutions, akin to the Fieller method, but also maintains validity with smaller sample sizes, an aspect where the Fieller method falls short. This distinction positions the extended JN technique as an attractive option. Nevertheless, the alternative approaches, including the ones mentioned, are not discouraged for finding CIs of crossover points. The extended JN technique is proposed as a viable avenue that aligns with the standard procedure in the moderation literature.

It is a standard practice to examine interaction effects and simple slopes through a blend of moderated regression analysis and simple slopes analysis. Notably, the extended JN technique is consistent with this conventional approach. Embracing both the basic and extended JN techniques facilitates the exploration of ordinal and disordinal interactions and simple slopes.

Empirical Illustration

In this section, we empirically compare alternative CIs of crossover points, focusing on scenarios with sufficiently large sample sizes, as recommended by prior researchers (Lee et al., 2015). We design this empirical study with three purposes. First, we empirically confirm whether the Fieller method and the extended JN technique yield similar CIs of crossover points when the sample size is sufficiently large, as expected. Second, we empirically compare the CIs generated by the bootstrap technique with those offered by the extended JN technique. Third, we empirically compare the widths of CIs resulting from different approaches when the sample size is substantially large.

The alternative approaches include the reparameterization technique, the delta method, the bootstrap technique (encompassing both normal theory and percentile approaches), the Fieller method, and the extended JN technique. Notably, two types are used for the bootstrap technique: the normal-theory bootstrap supported by the central limit theorem and the percentile bootstrap recommended as a new standard in this domain (e.g., Fritz et al., 2012; Hayes & Scharkow, 2013; Igartua & Hayes, 2021). Igartua and Hayes (2021) highlight that the BCa CI might not universally enhance outcomes, occasionally even diminishing the bootstrap technique’s performance. Moreover, the calculation of BCa with prevalent software packages is nontrivial.

Data and Model

We employed the data utilized in Echambadi and Hess (2007), accessible at https://doi.org/10.1287/mksc.1060.0263. This data set encompasses a substantial n = 10,203 observations. The dependent variable is perceived overall quality of an extension brand. The focal predictor (X ₁) is perceived substitutability between a parent product and an extension product, while the moderator (X ₂) is perceived quality of the parent brand. Both the focal predictor and the moderator were measured on a “1–7” scale, making the observable range of the focal predictor evident from the measure.

In their research, Echambadi and Hess (2007) estimated a moderated regression model with three control variables but reported the estimation results excluding statistics related to these control variables. We re-estimated the moderated regression model while accounting for the three control variables. SPSS was used in this estimation. It is worth noting that control variables did not influence simple slopes and crossover point. Hence, we opted not to incorporate their effects in examining the CI of a crossover point. Table 2 shows the summary statistics.

Table 2.

The Results Analyzed by Moderated Regression Analysis

	$β_{1}$	$β_{2}$	$β_{3}$	# of Observations
Estimate	−0.04655	0.25021	0.03426	10,203
Standard error	0.02562	0.01549	0.00454
t-statistic	−1.81716	16.15085	7.53215
p value	.0692	<.001	<.001
VAR-COV
$β_{1}$	0.00070	—	—
$β_{2}$	0.00033	0.00027	—
$β_{3}$	−0.00012	−0.00006	0.00002

Notes. $β_{1}$ , $β_{2}$ , and $β_{3}$ indicate the effect of the focal predictor (X ₁), the effect of the moderator (X ₂), and the interaction effect. We estimated the moderated regression model with three control variables in the data set. However, we reported the results excluding the statistics for the effects of control variables.

Software Packages for Alternative Approaches

As illustrated in Table 1, the reparameterization technique and the bootstrap technique require additional estimations such as bootstrap estimation and nonlinear least estimation to compute the CI of a crossover point. For this purpose, we employed syntax codes using SPSS and PROCESS for SPSS to execute the reparameterization technique and the bootstrap technique. These syntax codes are outlined in Supplemental Appendix D.

In contrast, the delta method, the Fieller method, and the extended JN technique entail mathematical calculations based on the summary statistics reported in Table 2 to determine the CI. We accomplished this calculation through SPSS syntax codes, which are detailed in Supplemental Appendix D. It is crucial to recognize that these three approaches facilitate the derivation of CIs without necessitating specialized computer codes, given that the CIs are computed mathematically using summary statistics.

Results

Table 3 shows the CIs of crossover points as well as their widths. The observable range of the focal predictor was measured on a “1–7” scale, yielding a calculated crossover point of −7.3040. Key findings are as follows. First, all alternative approaches indicated an ordinal interaction effect, because the CIs obtained by these methods extended beyond the observable range of the focal predictor. Second, the CIs obtained by the two approximation techniques (the reparameterization technique and the delta method) exhibited resemblance and were narrower than those generated by other techniques (the bootstrap technique, the Fieller method, and the extended JN technique). Third, the anticipated similarity between the CIs from the Fieller method and the extended JN technique was substantiated, although minor distinctions were noted. Fourth, both versions of the bootstrap technique (normal theory and percentile) yielded CIs closely aligned with the CI determined by the extended JN technique. In particular, the CI from the percentile bootstrap closely resembled the CI from the extended JN technique.

Table 3.

Confidence Intervals for Crossover Points in Alternative Approaches

Approach	Crossover Point	95% Confidence Interval (CI)	Width of CI
Reparameterization	−7.3040	[−9.9688, −4.6391]	5.3296
Delta method	−7.3040	[−9.5207, −5.0872]	4.4335
Normal-theory bootstrap	−7.3040	[−10.3539, −4.2540]	6.0999
Percentile bootstrap	−7.3040	[−11.2484, −5.1127]	6.1358
Fieller method	−7.3040	[−11.1561, −5.0861]	6.0700
Extended Johnson–Neyman (JN) technique	−7.3040	[−11.1561, −5.0861]	6.0700

Notes. We used 5,000 bootstrap samples in the normal-theory bootstrap and the percentile bootstrap. We used the second-order delta method. We calculated the normal-theory bootstrap CI according to the procedure proposed by Fox (2016, p. 656). The Fieller method and the extended JN technique yielded the identical CIs of the crossover point when the CIs were represented with 4 numbers behind the decimal point. However, this result does not mean that they are exactly identical to each other. There exists a marginal difference between them even when the sample size is sufficiently large.

Conclusions

The estimation of simple slopes and the calculation of crossover points of a focal predictor in a moderated regression model is a common analytic approach. This involves determining the crossover point using the effect of a moderator and the interaction effect between the focal predictor and the moderator in a moderated regression model. Then, researchers may conclude that the interaction is disordinal (or ordinal) if the crossover point falls within (or outside) the observable range of the focal predictor. However, caution is warranted, as this approach might yield incorrect conclusions about the nature of ordinal and disordinal interactions. This is due to the fact that the crossover point is inherently a random variable defined by its mean and variance.

To mitigate potential errors in deducing ordinal and disordinal interactions, it becomes imperative to scrutinize the interaction effect’s nature by comparing the observable range of the focal predictor with the CI of the focal predictor’s crossover point. If the CI falls within (or outside) the observable range, a clear determination of disordinal (or ordinal) interaction can be made. In cases where the CI intersects both within and outside the observable range, it becomes challenging to conclusively determine the nature of the interaction.

Although various alternative approaches, such as the reparameterization technique, the delta method, the bootstrap technique, and the Fieller method, enable CI estimation, these methods remain scarcely utilized in social science journals for assessing ordinal and disordinal interactions. Instead, graphical interpretations of these interactions are presented without a formal test. This inclination aligns with the standard practice in the literature, employing moderated regression analysis for interaction effects and subsequently employing simple slopes analysis for simple slopes. Simple slopes analysis is facilitated by the basic JN technique to ascertain the range of moderator values, where the focal predictor simple slopes are statistically significant. The familiarity with this standard procedure likely contributes to its preference over alternative approaches for interaction assessment.

In this context, this note introduces the extended JN technique, an extension of the basic JN technique, to compute the CI for crossover points. Unlike other techniques that necessitate additional estimations or calculations, the extended JN technique employs existing summary statistics to mathematically compute CIs. Implementing the extended JN technique via statistical software packages (e.g., Mplus, the JN function in R, and PROCESS) requires reversing the roles of the focal predictor and the moderator and determining the interval of nonsignificance of simple slopes.

Notably, the Fieller method, which relies on a χ² distribution, closely parallels the extended JN technique, which relies on a t-distribution, in providing closed-form solutions for crossover points. When sample size is sufficiently large, the Fieller method’s CIs align precisely with those from the extended JN technique. Nonetheless, it is important to acknowledge the Fieller method’s limitation, unreliable CIs of crossover points for small sample sizes due to assumptions about the χ² distribution being violated.

While previous research recommends the bootstrap technique for finding CIs of crossover points, we advocate for the extended JN technique as a compelling alternative. The extended JN technique aligns with the conventional methodology in the moderation literature. Utilizing a combination of the basic and extended JN techniques allows for the examination of ordinal and disordinal interactions as well as simple slopes through simple slopes analysis.

Supplemental Material

Supplemental Material, sj-docx-1-jeb-10.3102_10769986231217472 - A Simple Technique Assessing Ordinal and Disordinal Interaction Effects

Supplemental Material, sj-docx-1-jeb-10.3102_10769986231217472 for A Simple Technique Assessing Ordinal and Disordinal Interaction Effects by Sang-June Park and Youjae Yi in Journal of Educational and Behavioral Statistics

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research and/or authorship of this article: This research is supported by the Institute of Management Research, Seoul National University.

ORCID iD

Youjae Yi

References

Andrews

J. C.

Netemeyer

Burton

Kees

(2021). What consumers actually know: The role of objective nutrition knowledge in processing stop sign and traffic light front-of-pack nutrition labels. Journal of Business Research, 128, 140–155.

Cohen

West

S. G.

Aiken

L. S.

(2003). Applied multiple regression/correlation analysis for the behavioral sciences (3rd ed.). Lawrence Erlbaum.

Cox

(1990). Fieller’s theorem, the likelihood, and the delta method. Biometrics, 46, 709–718.

Darlington

R. B.

Hayes

A. F.

(2017). Regression analysis and linear models: Concepts, applications, and implementation. The Guilford Press.

Echambadi

Hess

J. D.

(2007). Mean-centering does not alleviate collinearity problems in moderated multiple regression models. Marketing Science, 26(3), 438–445.

Efron

(1979). Bootstrap methods: Another look at the jackknife. Annals of Statistics, 7(1), 1–26.

Fieller

E. C.

(1932). The distribution of the index in a normal bivariate population. Biometrika, 24, 428–440.

Fieller

E. C.

(1954). Some problems in interval estimation. Journal of the Royal Statistical Society Series B. (Methodological), 16, 175–185.

Flurry

L. A.

Swimberghe

Allen

(2021). Exposing the moderating impact of parent-child value congruence on the relationship between adolescents’ materialism and subjective well-being. Journal of Business Research, 128, 290–302.

10.

Fox

(2016). Applied regression analysis and generalized linear model (3rd ed.). Sage Publications.

11.

Fritz

M. S.

Taylor

A. B.

MacKinnon

D. P.

(2012). Explanation of two anomalous results in statistical mediation analysis. Multivariate Behavioral Research, 47(1), 61–87.

12.

Gleser

L. J.

Hwang

J. T

. (1987). The nonexistence of 100 (

1 - α

)% confidence sets of finite expected diameter in errors-in-variables and related models. Annals of Statisti cs, 15, 1351–1362.

13.

Hayes

A. F.

(2022). Introduction to mediation, moderation, and conditional process analysis: A regression-based approach. The Guilford Press.

14.

Hayes

A. F.

Scharkow

(2013). The relative trustworthiness of inferential tests of the indirect effect in statistical mediation analysis: Does method really matter? Psychological Science, 24(10), 1918–1927.

15.

Hurtak

Kashyap

Ehret

(2022). Lightening the dark side of customer participation—The mitigating role of relationship performance in business-to-business project contexts. Journal of Business Research, 140, 220–231.

16.

Igartua

J.-J.

Hayes

A. F.

(2021). Mediation, moderation, and conditional process analysis: Concepts, computations, and some common confusions. The Spanish Journal of Psychology, 24(e49), 1–23.

17.

Irwin

J. R.

McClelland

G. H.

(2001). Misleading heuristics and moderated multiple regression models. Journal of Marketing Research, 38(1), 100–109.

18.

Jaccard

Turrisi

(2003). Interaction effects in multiple regression. Sage Publications.

19.

Johnson

P. O.

Neyman

(1936). Tests of certain linear hypotheses and their application to some educational problems. Statistical Research Memoirs, 1, 57–93.

20.

Lee

Lei

M.-K.

Brody

G. H.

(2015). Confidence intervals for distinguishing ordinal and disordinal interactions in multiple regression. Psychological Methods, 20, 245–258.

21.

Lin

(2020). Probing two-way moderation effects: A review of software to easily plot Johnson-Neyman figures. Structural Equation Modelling: A Multidisciplinary Journal, 27(3), 494–402.

22.

MacCallum

R. C.

Zhang

S. Z.

Preacher

K. J.

Rucker

D. D.

(2002). On the practice of dichotomization of quantitative variables. Psychological Methods, 7(1), 19–40.

23.

Marsaglia

(1965). Ratios of normal variables and ratios of sums of uniform variables. Journal of the American Statistical Association, 60, 193–204.

24.

Marsaglia

(2006). Ratios of normal variables. Journal of Statistical Software, 16, 1–10.

25.

Murphy

K. R.

Russell

C. J.

(2017). Mend it or end it: Redirecting the search for interactions in the organizational science. Organizational Research Methods, 20(4), 549–573.

26.

Nejad

M. G.

Amini

Sherrell

D. L.

(2016). The profit impact of revenue heterogeneity and assortativity in the presence of negative word-of-mouth. International Journal of Research in Marketing, 33(3), 656–673.

27.

Padgett

Hopkins

C. D.

Williams

(2020). Buyer dependence in B2B relationships: The role of supplier investments, commitment form, and trust. Journal of Business Research, 119, 13–24.

28.

Park

S.-J.

(2022). Assessing moderator effects, main effects, and simple effects without collinearity problems in moderated regression analysis. Journal of Business Research, 145, 905–919.

29.

Park

S.-J.

(2023). Assessing differences between simple slopes in simple slopes analysis. Journal of Business Research, 168, 1140204.

30.

Preacher

K. J.

Rucker

D. D.

Hayes

A. F.

(2007). Addressing moderated mediation hypothesis: Theory, methods, and prescriptions. Multivariate Behavioral Research, 42(1), 185–227.

31.

Rao

C. R.

(1973). Linear statistical inference and its applications. Wiley.

32.

Rogosa

(1980). Comparing nonparallel regression lines. Psychological Bulletin, 88(2), 307–321.

33.

Rogosa

(1981). On the relationship between the Johnson–Neyman region of significance and statistical tests of parallel within group regressions. Educational and Psychological Measurement, 41, 73–84.

34.

Spiller

S. A.

Fitzsimons

G. J.

Lynch

J. G.

Jr. McClelland

G. H.

(2013). Spotlights, floodlights, and the magic number zero: Simple effects tests in moderated regression. Journal of Marketing Research, 50(2), 277–288.

35.

Vickery

S. K.

Koufteros

Dröge

Calantone

(2016). Product modularity, process modularity, and new product introduction performance: Does complexity matter? Production and Operations Management, 25(4), 751–770.

36.

Widaman

K. F.

Helm

J. L.

Castro-Schilo

Pluess

Stallings

M. C.

Belsky

(2012). Distinguishing ordinal and disordinal interactions. Psychological Methods, 17(4), 615–622.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.22 MB