Two Prophecy Formulas for Assessing the Reliability of Item Response Theory-Based Ability Estimates

Abstract

Two new prophecy formulas for estimating item response theory (IRT)-based reliability of a shortened or lengthened test are proposed. Some of the relationships between the two formulas, one of which is identical to the well-known Spearman-Brown prophecy formula, are examined and illustrated. The major assumptions underlying these formulas are outlined and discussed. Both prophecy formulas appear to provide comparable estimates of reliability in the IRT context as well as in the classical test theory context.

Keywords

reliability Spearman-Brown prophecy formula item response theory classical test theory

Get full access to this article

View all access options for this article.

References

Divgi, D. R. (1989). Estimating reliabilities of computerized adaptive tests. Applied Psychological Measurement, 13, 145-149.

Green, B. F. , Bock, R. D. , Humphreys, L. G. , Linn, R. L. , & Reckase, M. D. (1984). Technical guidelines for assessing computerized adaptive tests. Journal of Educational Measurement, 21, 347-360.

Hambleton, R. K. , & Swaminathan, H. (1985). Item response theory: Principles and applications. Boston: Kluwer-Nijhoff.

Lord, F. M. (1980). Applications of item response theory to practical testing problems. Hillsdale, NJ: Lawrence Erlbaum.

Lord, F. M. (1983). Unbiased estimators of ability parameters, of their variance, and of their parallel-forms reliability. Psychometrika, 48, 233-245.

Lord, F. M. , & Novick, M. R. (1968). Statistical theories of mental test scores. Reading, MA: Addison-Wesley.

Mislevy, R. J. , & Bock, R. D. (1990). BILOG3: Item analysis and test scoring with binary logistic models. Mooresville, IN: Scientific Software.

Nicewander, W. A. , & Thomasson, G. L. (1999). Some reliability estimates for computerized adaptive tests. Applied Psychological Measurement, 23, 239-247.

Samejima, F. (1977). A use of information function in tailored testing. Applied Psychological Measurement, 1, 233-247.

10.

Samejima, F. (1994). Estimation of reliability coefficients using the test information function and its modifications. Applied Psychological Measurement, 18, 229-244.

11.

Sympson, J. B. (1980, April). Estimating the reliability of adaptive tests from a single administration. Paper presented at the annual meeting of the American Educational Research Association, Boston.

12.

Wright, B. D. , & Stone, M. H. (1979). Best test design. Chicago: Mesa.