Type II Error Problems in the Use of Moderated Multiple Regression for the Detection of Moderating Effects of Dichotomous Variables

Abstract

Monte Carlo simulation procedures were used to assess the power of moderated multiple regression (MMR) to detect the effects of a dichotomous moderator variable under conditions of: (1) between- group differences in within-group relationships between two variables (i.e., |ρ _XY ₍₁₎ -ρ _XY ₍₂₎ |= .20, .40, .60); (2) different combined sample sizes for the two groups (N ₁ + N ₂ = N_T = 30, 60, 90, 180, 300); and (3) differing proportions of cases (P_-i ) in the two groups (i.e., P ₁ = .10, .30, .50). Results showed that, consistent with our a priori predictions, the power of MMR increased as: (1) total sample size (NT) increased; (2) the difference between within-group correlation coefficients increased; and ( 3) the difference between the proportion of cases in each group decreased. Moreover, the simulation showed that these three variables had interactive effects on power. The major implication of our findings is that in cases where tests of moderating effects are conducted with MMR and the proportion of cases in each group differs greatly, inferences of no moderating effect may be erroneous: Such inferences may be the result of low statistical power rather than the absence of a moderating effect.

Get full access to this article

View all access options for this article.

References

Aguinis, H. (1993). The detection of moderator variables using moderated multiple regression: The impact of range restriction, unreliability, sample size, and the proportion of subjects in subgroups. Unpublished doctoral dissertation. University at Albany, State University of New York .

Alexander, R.A. , De Shon, R.P. , & Govern, D.M. (1993). Error variance heterogeneity and testing for regression slope differences in differential prediction research. Paper presented at the meeting of the Society for Industrial and Organizational Psychology. San Francisco, CA.

Cohen, J. & Cohen, P. (1983). Applied multiple regression/correlation for the behavioral sciences. Hillsdale, NJ: Erlbaum.

Cortina, J.M. , Doherty, M.L. , Schmitt, N. , Kaufman, G. , & Smith, R.G. (1992). The "big five" personality factors in the IPI and MMPI: Predictors of police performance. Personnel Psychology , 45: 119-140.

Hattrup, K. & Schmitt, N. (1990). Prediction of trades apprentices' performance on job sample criteria. Personnel Psychology, 43: 453-466.

Nunnally, J.C. (1978). Psychometric theory, 2nd ed. New York: McGraw-Hill.

Stone, E.F. (1988). Moderator variables in research: A review and analysis of conceptual and methodological issues. Pp. 191-229 in G. R. Ferris & K. M. Rowland (Eds.), Research in personnel and human resources management, Vol. 6. Greenwich, CT: JAI.

Stone-Romero, E.F. , & Anderson, L. (in press). Relative statistical power of moderated multiple regression and the comparison of subgroup correlation coefficients for the detection of moderating effects. Journal of Applied Psychology.