This article gives equations for computing confidence interval estimates and conducting F tests for intraclass correlation coefficients (ICCs) defined using three-way random effects models for crossed and nested designs. The estimates and tests employ mean squares from analyses of variance. The equations are generalized so that data from an initial pilot study (G study) can be used to prophesy the confidence limits for ICCs in data with more or fewer factor levels. Equations also are included for determining the optimal number of factor levels to use in a design that will offer confidence of 1-[.alpha] that the population ICC is equal to or exceeds a fixed lower limit.
Get full access to this article
View all access options for this article.
References
1.
Brennan, R. L. (1983). Elements of generalizability theory. Iowa City, IA: ACT.
2.
Cronbach, L. J. , Gleser, G. C., Nanda, H., & Rajaratnam, N. (1972). The dependability of behavioral measurements: Theory of generalizability for scores and profiles. New York: John Wiley.
3.
Fleiss, J. L. , & Shrout, P. E. (1978). Approximate interval estimation for a certain intraclass coefficient. Psychometrika, 43, 259-262.
4.
Kirk, R. E. (1995). Experimental design: Procedures for the behavioral sciences. Pacific Grove, CA: Brooks/Cole.
5.
Maxwell, S. E. , & Delaney, H. D. (1990). Designing experiments and analyzing data: A model comparison perspective. Pacific Grove, CA: Brooks/Cole.
6.
McGraw, K. O. , & Wong, S. P. (1996). Forming inferences about some intraclass correlation coefficients. Psychological Methods, 1, 30-46.
7.
Satterthwaite, F. E. (1946). An approximate distribution of estimates of variance components. Biometrics, 2, 110-114.
8.
Schroeder, M. L. , & Hakstian, A. R. (1990). Inferential procedures for multifaceted coefficients of generalizability. Psychometrika, 55, 429-447.
9.
Shavelson, R. J. , & Webb, N. W. (1991). Generalizability theory: A primer. Newbury Park, CA: Sage.
10.
Shrout, P. E. , & Fleiss, J. L. (1979). Intraclass correlation: Uses in assessing rater reliability. Psychological Bulletin, 86, 420-428.
