Ν2: A Proportional Variance-Accounted for Index for Some Cross-Level and Person-Situation Research Designs

Abstract

The authors present a new “relative effect size” statistic (n ² ) that indexes the proportion of criterion variance accounted for by (some set of) measured variable(s) relative to the proportion of criterion variance that could potentially be accounted for by the entire class of variables of which the measured variables are a subset. Statistical rationale, estimation procedures, inferential statistical tests, and research examples are presented for n ² for the cases of situational variables, person variables, and person situation interactions.

Get full access to this article

View all access options for this article.

References

Abelson, R. P. (1985). A variance explanation paradox: When a little is a lot. Psychological Bulletin, 97, 129-133.

Bartko, J. (1976). On various intraclass correlation reliability coefficients. Psychological Bulletin, 83, 762-765.

Bliese, P. D. , & Halverson, R. R. (1998). Group size and measures of group-level properties: An examination of eta-squared and ICC values. Journal of Management, 24, 157-172.

Bryk, A. S. , & Raudenbush, S. W. (1987). Application of hierarchical linear models to assessing change. Psychological Bulletin, 101, 147-158.

Bryk, A. S. , & Raudenbush, S. W. (1992). Hierarchical linear models: Applications and data analysis methods. Newbury Park, CA: Sage.

Burtch, L. D. , Lipscomb, M. S. , & Wissman, D. J. (1982). Aptitude requirements based on task difficulty: Methodology for evaluation (AFHRL-TR-81-34). Brooks AFB, TX: Air Force Human Resources Laboratory, Manpower and Personnel Division.

Carver, R. P. (1978). The case against statistical significance testing. Harvard Educational Review, 48, 378-399.

Cattell, R. B. (1966). The data box: Its ordering of total resources in terms of possible relational systems. In R. B. Cattell (Ed.), Handbook of multivariate experimental psychology (pp. 67-128). Chicago: Rand McNally.

Chou, C. -P. , Bentler, P. M. , & Pentz, M. A. (1998). Comparisons of two statistical approaches to study growth curves: The multilevel and the latent curve analysis. Structural Equation Modeling, 5, 247-266.

10.

Cohen, J. (1973). Eta-squared and partial eta-squared in fixed factor ANOVA designs. Educational and Psychological Measurement, 33, 107-112.

11.

Cohen, J. , & Cohen, P. (1975). Applied multiple regression/correlation analysis for the behavioral sciences. New York: John Wiley.

12.

Cohen, J. , & Cohen, P. (1983). Applied multiple regression/correlation analysis for the behavioral sciences. Hillsdale, NJ: Lawrence Erlbaum.

13.

Conley, J. J. (1984). Relation of temporal stability and cross-situational consistency in personality: Comment on the Mischel-Epstein debate. Psychological Review, 91, 491-496.

14.

Cooper, W. H. , & Richardson, A. J. (1986). Unfair comparisons. Journal of Applied Psychology, 71, 179-184.

15.

Craig, J. R. , Eison, C. L. , & Metze, L. P. (1976). Significance tests and their interpretation: An example utilizing published research and ω2. Bulletin of the Psychonomic Society, 7, 280-282.

16.

Dansereau, F. , Alutto, J. A. , & Yammarino, F. J. (1984). Theory testing in organizational behavior: The variant approach. Englewood Cliffs, NJ: Prentice Hall.

17.

Darley, J. M. , & Latane, B. (1968). Bystander intervention in emergencies: Diffusion of responsibility. Journal of Personality and Social Psychology, 27, 100-108.

18.

Department of Defense . (1984, July). Test manual for the armed services vocational aptitude battery. Chicago: U.S. Military Entrance Processing Command.

19.

Diener, E. , & Larsen, R. J. (1984). Temporal stability and cross-situational consistency of affective, behavioral, and cognitive responses. Journal of Personality and Social Psychology, 47, 871-883.

20.

Epstein, S. (1979). The stability of behavior: I. On predicting most of the people much of the time. Journal of Personality and Social Psychology, 37, 1097-1126.

21.

Epstein, S. , & O’Brien, E. J. (1985). The person-situation debate in historical and current perspective. Psychological Bulletin, 98, 513-537.

22.

Festinger, L. , & Carlsmith, J. M. (1959). Cognitive consequences of forced compliance. Journal of Abnormal and Social Psychology, 58, 203-210.

23.

Fiske, D. W. , & Rice, L. (1955). Intra-individual response variability. Psychological Bulletin, 52, 217-250.

24.

Fleiss, J. L. (1969). Estimating the magnitude of experimental effects. Psychological Bulletin, 72, 273-276.

25.

Fowler, R. L. (1985). Testing for substantive significance in applied research by specifying nonzero effect null hypotheses. Journal of Applied Psychology, 70, 215-218.

26.

Funder, D. C. , & Ozer, D. J. (1983). Behavior as a function of the situation. Journal of Personality and Social Psychology, 44, 107-112.

27.

Ghiselli, E. E. (1966). The validity of occupational aptitude tests. New York: John Wiley.

28.

Golding, S. L. (1975). Flies in the ointment: Methodological problems in the analysis of the percentage of variance due to persons and situations. Psychological Bulletin, 82, 278-288.

29.

Griffin, M. A. , & Hofmann, D. A. (in press). Hierarchical linear modeling in organizational research. In D. Dalton & L. Williams (Eds.), Advances in research methods for organizational studies. Greenwich, CT: JAI.

30.

Hays, W. L. (1988). Statistics (4th ed.). Orlando, FL: Holt, Rinehart & Winston.

31.

Hofmann, D. A. , Jacobs, R. , & Baratta, J. E. (1993). Dynamic criteria and the measurement of change. Journal of Applied Psychology, 78, 194-204.

32.

Hollenbeck, J. R. , Ilgen, D. R. , LePine, J. A. , Colquitt, J. A. , & Hedlund, J. (1998). Extending the multilevel theory of team decision making: Effects of feedback and experience in hierarchical teams. Academy of Management Journal, 41, 269-282.

33.

Hollenbeck, J. R. , Ilgen, D. R. , & Sego, D. J. (1994). Repeated measures regression: Enhancing the power of leadership research. Leadership Quarterly, 5, 3-23.

34.

Hollenbeck, J. R. , Ilgen, D. R. , Sego, D. J. , Hedlund, J. , Major, D. A. , & Phillips, J. (1995). Multilevel theory of team decision making: Decision performance in teams incorporating distributed expertise. Journal of Applied Psychology, 80, 292-316.

35.

House, R. , Rousseau, D. M. , & Thomas-Hunt, M. (1995). The meso paradigm: A framework for the integration of micro and macro organizational behavior. Research in Organizational Behavior, 17, 71-114.

36.

Hunter, J. E. , & Hunter, R. F. (1984). Validity and utility of alternative predictors of job performance. Psychological Bulletin, 96, 72-98.

37.

James, L. R. (1980). The unmeasured variables problem in path analysis. Journal of Applied Psychology, 65, 415-421.

38.

James, L. R. (1982). Aggregation bias in estimates of perceptual agreement. Journal of Applied Psychology, 67, 219-229.

39.

James, L. R. , Demaree, R. G. , & Hater, J. J. (1980). A statistical rationale for relating situational variables and individual differences. Organizational Behavior and Human Performance, 25, 354-364.

40.

James, L. R. , Demaree, R. G. , & Wolf, G. (1984). Estimating within-group interrater reliability with and without response bias. Journal of Applied Psychology, 69, 85-98.

41.

James, L. R. , Mulaik, S. A. , & Brett, J. (1982). Causal analysis: Assumptions, models and data. Beverly Hills, CA: Sage.

42.

Jensen, A. R. (1984). Test validity: g versus the specificity doctrine. Journal of Social Biological Structures, 7, 93-118.

43.

Kanfer, R. , & Ackerman, P. (1989). Motivation and cognitive abilities: An integrative/aptitudetreatment interaction approach to skill acquisition. Journal of Applied Psychology, 74, 657-690.

44.

Kendall, M. , & Stuart, A. (1977). The advanced theory of statistics (Vol. 2, 4th ed.). New York: Macmillan.

45.

Kendrick, D. T. , & Funder, D. C. (1988). Profiting from the controversy: Lessons from the person-situation debate. American Psychologist, 43, 23-34.

46.

Kennedy, J. J. (1970). The eta coefficient in complex ANOVA designs. Educational and Psychological Measurement, 30, 885-889.

47.

Kirk, R. E. (1982). Experimental design (2nd ed.). Monterey, CA: Brooks/Cole.

48.

Kirk, R. E. (1996). Practical significance: A concept whose time has come. Educational and Psychological Measurement, 56, 746-759.

49.

Klein, K. J. , Dansereau, F. , & Hall, R. J. (1994). Levels issues in theory development, data collection, and analysis. Academy of Management Review, 19, 195-229.

50.

Lahey, M. A. , Downey, R. G. , & Saal, F. E. (1983). Intraclass correlations: There’s more there than meets the eye. Psychological Bulletin, 93, 586-595.

51.

Lance, C. E. (1988). Residual centering, exploratory and confirmatory moderator analysis, and decomposition of effects in path models containing interactions. Applied Psychological Measurement, 12, 163-175.

52.

Lance, C. E. , Hedge, J. W. , & Alley, W. E. (1989). Joint relationships of task proficiency with aptitude, experience, and task difficulty: A cross-level, interactional study. Human Performance, 2, 249-272.

53.

Longford, N. T. (1995). Random coefficient models. In G. Arminger , C. C. Clogg , & M. E. Sobel (Eds.), Handbook of statistical modeling for the social and behavioral sciences (pp. 519-577). New York: Plenum.

54.

Lykken, D. T. (1968). Statistical significance in psychological research. Psychological Bulletin, 70, 151-159.

55.

McDaniel, M. A. , Schmidt, F. L. , & Hunter, J. E. (1988). Job experience correlates of performance. Journal of Applied Psychology, 73, 327-330.

56.

McGrath, J. E. (1976). Stress and behavior in organizations. In M. D. Dunnette (Ed.), Handbook of industrial and organizational psychology. Chicago: Rand McNally.

57.

McGraw, K. O. , & Wong, S. P. (1996). Forming inferences about some intraclass correlation coefficients. Psychological Methods, 1, 30-46.

58.

Milgram, S. (1975). Obedience to authority. New York: Harper/Colophon.

59.

Mossholder, K. W. , & Bedeian, A. G. (1983). Cross-level inference and organizational research: Perspectives on interpretation and application. Academy of Management Review, 8, 547-558.

60.

Mumford, M. D. , Weeks, J. L. , Harding, F. D. , & Fleishman, E. A. (1987). Measuring occupational difficulty: A construct validation against training criteria. Journal of Applied Psychology, 72, 578-587.

61.

O’Grady, K. E. (1982). Measures of explained variance: Cautions and limitations. Psychological Bulletin, 92, 766-777.

62.

Ozer, D. J. (1985). Correlation and the coefficient of determination. Psychological Bulletin, 97, 307-315.

63.

Prentice, D. A. , & Miller, D. T. (1992). When small effects are impressive. Psychological Bulletin, 112, 160-164.

64.

Quinones, M. A. , Ford, J. K. , & Teachout, M. S. (1995). The relationship between work experience and job performance: A conceptual and meta-analytic review. Personnel Psychology, 48, 887-910.

65.

Raju, N. S. , Burke, M. J. , & Normand, J. (1990). A new approach for utility analysis. Journal of Applied Psychology, 75, 3-12.

66.

Ramage, J. A. (1987). Task learning difficulty: Interrelationships among aptitude-specific benchmarked rating scales (AFHRL-TP-86-56). Brooks AFB, TX: Air Force Human Resources Laboratory, Manpower and Personnel Division.

67.

Rogosa, D. R. , & Willett, J. B. (1985). Understanding correlates of change by modeling individual differences in growth. Psychometrika, 50, 203-228.

68.

Rosenthal, R. , & Rubin, D. B. (1979). A note on percent variance explained as a measure of the importance of effects. Journal of Applied Social Psychology, 9, 395-396.

69.

Rousseau, D. M. (1985). Issues of level in organizational research: Multi-level and cross-level perspectives. Research in Organizational Behavior, 7, 1-37.

70.

Schmidt, F. L. , & Hunter, J. E. (1998). The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research. Psychological Bulletin, 124, 262-274.

71.

Schmidt, F. L. , Hunter, J. E. , McKenzie, R. C. , & Muldrow, T. W. (1979). Impact of valid selection procedures on work-force productivity. Journal of Applied Psychology, 64, 609-626.

72.

Schmidt, F. L. , Hunter, J. E. , Outerbridge, A. N. , & Goff, S. (1988). Joint relation of experience and ability with job performance: Test of three hypotheses. Journal of Applied Psychology, 73, 46-57.

73.

Schneider, B. , & Bowen, D. E. (1985). Employee and customer perceptions of service in banks: Replication and extension. Journal of Applied Psychology, 70, 423-433.

74.

Schwab, D. P. , & Cummings, L. L. (1976). A theoretical analysis of the impact of task scope on employee performance. Academy of Management Review, 1(2), 23-35.

75.

Sechrest, L. , & Yeaton, W. H. (1982). Magnitudes of experimental effects in social science research. Evaluation Review, 6, 579-600.

76.

Shrout, P. E. , & Fleiss, J. L. (1979). Intraclass correlations: Uses in assessing reliability. Psychological Bulletin, 86, 420-428.

77.

Southwood, K. E. (1978). Substantive theory and statistical interaction: Five models. American Journal of Sociology, 83, 1154-1203.

78.

Steiger, J. H. , & Ward, L. M. (1987). Factor analysis and the coefficient of determination. Psychological Bulletin, 101, 471-474.

79.

Ward, C. J. , & Jennings, E. (1973). Introduction to linear models. Englewood Cliffs, NJ: Prentice Hall.

80.

Wigdor, A. K. , & Green, B. F., Jr. (1986). Assessing the performance of enlisted personnel. Washington, DC: National Academy Press.

81.

Yammarino, F. J. , & Markham, S. E. (1992). On the application of within and between analysis: Are absence and affect really group-based phenomena? Journal of Applied Psychology, 77, 168-176.