Single-Item Reliability: A Replication and Extension

Abstract

The reliability of a single-item measure of student-rated college teaching effectiveness was estimated with two different methods and at two levels of analysis. The two methods are the correction for attenuation formula and factor analysis. The two levels of analysis are the group level (10, 682 classes) and the individual level (323, 262 students). Reliability estimates were higher using factor analysis (.88) than the correction for attenuation formula (.64), and they were higher using group-level data (.82) than individual-level data (.70). Based on the assumptions and limitations of each method used, the authors conclude that a minimum estimate of .80 for single-item reliability is reasonable for group-level data. The authors reaffirm a minimum reliability estimate of .70 for individual-level data, as previously concluded by Wanous, Reichers, and Hudy, who estimated single-item reliability for measures of overall job satisfaction using individual-level data.

Get full access to this article

View all access options for this article.

References

Arvey, R. D., Landon, T. E., Nutting, S. M., & Maxwell, S. E. (1992). Development of physical ability tests for police officers: A construct validation approach. Journal of Applied Psychology, 77, 996-1009.

Cranton, P., & Smith, R. A. (1990). Reconsidering the unit of analysis: A model of student ratings of instruction. Journal of Educational Psychology, 82, 207-212.

Harman, H. H. (1967). Modern factor analysis. Chicago: University of Chicago Press.

Hunter, J. E., & Schmidt, F. L. (1990). Methods of meta-analysis. Newbury Park, CA: Sage.

Nunnally, J. C. (1978). Psychometric theory (2nd ed.). New York: McGraw-Hill.

Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory (3rd ed.). New York: McGraw-Hill.

Rice, C. E. (1997). The scientist-practitioner split and the future of psychology. American Psychologist, 52, 1173-118l.

Sackett, P. R., & Larson, J. R. (1990). Research strategies and tactics in industrial and organizational psychology. In M. D. Dunnette & L. M. Hough (Eds.), Handbook of industrial and organizational psychology (2nd ed., Vol. 1, pp. 419-489). Palo Alto, CA: Consulting Psychologists.

Scarpello, V., & Campbell, J. P. (1983). Job satisfaction: Are all the parts there? Personnel Psychology, 36, 577-600.

10.

Schwarzer, R. (1989). Meta-analysis software. Berlin, Germany: Freie Universitat.

11.

Switzer, F. (1991). VGBOOT 2.0. Clemson, SC: Clemson University, Department of Psychology.

12.

Wanous, J. P., & Reichers, A. E. (1996). Estimating the reliability of a single-item measure. Psychological Reports, 78, 631-634.

13.

Wanous, J. P., Reichers, A. E., & Hudy, M. J. (1997). Overall job satisfaction: How good are single-item measures? Journal of Applied Psychology, 82, 247-252.

14.

Weiss, D. J. (1976). Multivariate procedures. In M. D. Dunnette (Ed.), Handbook of industrial and organizational psychology (pp. 327-362). Chicago: Rand McNally College.