Impact of Missing Data on Person—Model Fit and Person Trait Estimation

Abstract

The purpose of this research was to examine the effects of missing data on person—model fit and person trait estimation in tests with dichotomous items. Under the missing-completely-at-random framework, four missing data treatment techniques were investigated including pairwise deletion, coding missing responses as incorrect, hotdeck imputation, and model-based imputation. Person traits were estimated using the two-parameter item response model. Overall, missing data increased the difficulty in assessing person—model fit for both model-fitting and model-misfitting persons. The higher the proportion of missing data, the larger the number of persons incorrectly diagnosed. Among the four techniques, the pairwise deletion method performed best in recovering person—model fit and person trait level. Treating missing responses as incorrect caused the examinees with missing data to not fit the measurement model, thus invalidating the person trait estimates.

Keywords

person fit missing data pairwise deletion model-based imputation hotdeck imputation

Get full access to this article

View all access options for this article.

References

De Ayala, R.J. , Plake, B. , & Impara, J. (2001). The impact of omitted responses on the accuracy of ability estimation in item response theory. Journal of Educational Measurement, 38, 213-234.

Dempster, A.P. , Laird, D. , & Rubin, D. (1977). Maximum likelihood from incomplete data via the EM algorithm (with discussion). Journal of the Royal Statistical Society, Series B, 39, 1-38.

Ford, B.L. (1983). An overview of hot-deck procedures. In W. G. Madow , H. Nisselson , & I. Olkin (Eds.), Incomplete data in sample surveys (Vol. 2, pp. 185-207). New York: Academic Press.

Glasser, M. (1964). Linear regression analysis with missing observations among the independent variables. Journal of the American Statistical Association, 59, 834-844.

Huisman, M. , & Molenaar, W.I. (2001). Imputation of missing scale data with item response models. In A. Boomsma , M. A. J. van Duijn , & T. A. B. Snijders (Eds.), Essays on item response theory (pp. 221-244). New York: Springer-Verlag .

Karabatsos, G. (2003). Comparing the aberrant response detection performance of thirty-six person-fit statistics. Applied Measurement in Education, 16, 277-298.

Little, R.J.A. (1992). Regression with missing X's: A review. Journal of the American Statistical Association, 87, 1227-1237.

Little, R.J.A. , & Rubin, D.B. (1987). Statistical analysis with missing data. New York: John Wiley.

Lord, F.M. (1980). Applications of item response theory to practical testing problems. Hillsdale, NJ: Lawrence Erlbaum.

10.

Ludlow, L.H. , & O' Leary, M. (1999). Scoring omitted and not-reached items: Practical data analysis implications. Educational and Psychological Measurement, 59, 615-603.

11.

Meijer, R.R. (1996). Person-fit research: An introduction. Applied Measurement in Education, 9, 3-8.

12.

Meijer, R.R. , & Sijtsma, K. (2001). Methodology review: Evaluating person fit. Applied Psychological Measurement, 25, 107-135.

13.

Meijer, R.R. , & van Krimpen-Stoop, E.W.L.A. (2001). Person fit across subgroups: An achievement testing example. In A. Boomsma , M. A. J. van Duijn , & T. A. B. Snijders (Eds.), Essays on item response theory (pp. 377-390). New York: Springer-Verlag.

14.

Mislevy, R. , & Wu, P. (1988). Inferring examinee ability when some item responses are missing (RR-88-48-ONR). Princeton, NJ: Educational Testing Service.

15.

Mokken, R.J. , & Lewis, C. (1982). A nonparametric approach to the analysis of dichotomous item responses. Applied Psychological Measurement, 6, 417-430.

16.

Orlando, M. , & Thissen, D. (2003). Further investigation of the performance of S-X2: An item fit index for use with dichotomous item response theory models. Applied Psychological Measurement, 27, 289-298.

17.

Peugh, J.L. , & Enders, C.K. (2004). Missing data in educational research: A review of reporting practices and suggestions for improvement. Review of Educational Research, 74, 525-556.

18.

Raymond, M.R. , & Roberts, D.M. (1987). A comparison of methods for treating incomplete data in selection research. Educational and Psychological Measurement, 47, 13-26.

19.

Roth, P. (1994). Missing data: A conceptual review for applied psychologists. Personnel Psychology, 47, 537-560.

20.

Schafer, J.L. (1997). Analysis of incomplete multi-variate data. London: Chapman & Hall.

21.

Sijtsma, K. , & Meijer, R.R. (1992). A method for investigating the intersection of item response functions in Mokken's nonparametric IRT model. Applied Psychological Measurement, 16, 149-157.

22.

Thissen, D. (1991). MULTILOG: Multiple, categorical item analysis and test scoring using item response theory (Version 6) [Computer software and manual]. Mooresville, IN: Scientific Software International.

23.

Trabin, T.E. , & Weiss, D.J. (1983). The person response curve: Fit of individuals to item response theory models. In D. J. Weiss (Ed.), New horizons in testing (pp. 83-108). New York: Academic Press.