Congeneric and (Essentially) Tau-Equivalent Estimates of Score Reliability

Abstract

Coefficient alpha, the most commonly used estimate of internal consistency, is often considered a lower bound estimate of reliability, though the extent of its underestimation is not typically known. Many researchers are unaware that coefficient alpha is based on the essentially tau-equivalent measurement model. It is the violation of the assumptions required by this measurement model that are often responsible for coefficient alpha's underestimation of reliability. This article presents a hierarchy of measurement models that can be used to estimate reliability and illustrates a procedure by which structural equation modeling can be used to test the fit of these models to a set of data. Test and data characteristics that can influence the extent to which the assumption of tau-equivalence is violated are discussed. Both heuristic and applied examples are used to augment the discussion.

Keywords

reliability structural equation modeling congeneric tau-equivalent

Get full access to this article

View all access options for this article.

References

Aiken, L. S. , West, S. G. , Sechrest, L. , & Reno, R. R. (with Roediger, H. L., Scarr, S., Kazdin, A. E., & Sherman, S. J.). (1990). The training in statistics, methodology, and measurement in psychology. American Psychologist, 45, 721-734.

Arbuckle, J. L. (2003). Amos (Version 5.0.1) [Computer software]. Spring House, PA: Amos Development Corporation.

Bentler, P. M. (1990).Comparative fit indices in structural models. Psychological Bulletin,107, 238-246.

Bentler, P. M. , & Bonett, D. G. (1980). Significance tests and goodness of fit in the analysis of covariance structures. Psychological Bulletin, 88, 588-606.

Feldt, L. S. , & Brennan, R. L. (1989). Reliability. In R. L. Linn (Ed.), Educational measurement (3rd ed., pp. 105-146). Phoenix, AZ: Ornyx.

Graham, J. M. , & Conoley, C. W. (2006). The role of marital attributions in the relationship between life stressors and marital quality. Personal Relationships, 13, 231-244.

Graham, J. M. , Liu, Y. J. , & Jeziorski, J. L. (in press). The Dyadic Adjustment Scale: A reliability generalizability meta-analysis. Journal of Marriage and Family.

Jöreskog, K. G. , & Sörbom, D. (1984). LISREL VI user's guide (3rd ed.). Mooresville, IN: Scientific Software.

Lord, F. M. , & Novick, M. R. (1968). Statistical theories of mental test scores. Reading, MA: Addison-Wesley.

10.

Miller, M. B. (1995). Coefficient alpha: A basic introduction from the perspectives of classical test theory and structural equation modeling. Structural Equation Modeling, 2, 255-273.

11.

Mulaik, S. A. , James, L. R. , van Alstine, J. , Bennett, N. , Lind, S. , & Stilwell, C. D. (1989). Evaluation of goodness-of-fit indices for structural equation models. Psychological Bulletin, 195, 430-445.

12.

Pedhazur, E. J. , & Schmelkin, L. P. (1991). Measurement, design, and analysis: An integrated approach. Hillsdale, NJ: Lawrence Erlbaum.

13.

Raykov, T. (1997a). Estimation of composite reliability for congeneric measures. Applied Psychological Measurement, 21, 173-184.

14.

Raykov, T. (1997b). Scale reliability, Cronbach's coefficient alpha, and violations of essential tauequivalence with fixed congeneric components. Multivariate Behavioral Research, 32, 329-353.

15.

Spanier, G. B. (1976). Measuring dyadic adjustment: New scales for assessing the quality of marriage and similar dyads. Journal of Marriage and Family, 38, 15-28.

16.

Steiger, J. H. , & Lind, J. C. (1980, June). Statistically based tests for the number of common factors. Paper presented at the annual meeting of the Psychonomic Society, Iowa City, IA.

17.

Thompson, B. (2004). Exploratory and confirmatory factor analysis: Understanding concepts and applications. Washington, DC: American Psychological Association.

18.

Thompson, B. , & Vacha-Haase, T. (2000). Psychometrics is datametrics: The test is not reliable. Educational and Psychological Measurement, 60, 174-195.

19.

Vacha-Haase, T. , Kogan, L. R. , & Thompson, B. (2000). Sample compositions and variabilities in published studies versus those in test manuals: Validity of score reliability inductions. Educationaland Psychological Measurement, 60, 509-522.

20.

Whittington, D. (1998). How well do researchers report their measures? An evaluation of measurement in published educational research. Educational and Psychological Measurement, 58, 21-37.

21.

Wilcox, R. R. (1998). How many discoveries have been lost by ignoring modern statistical methods? American Psychologist, 53, 300-314.

22.

Wilkinson, L. , & American Psychological Association (APA) Task Force on Statistical Inference . (1999). Statistical methods in psychology journals: Guidelines and explanations. American Psychologist, 54,594-604.