Type I error rates of the likelihood ratio test for the detection of differential item functioning (DIF) were investigated using monte carlo simulations. The graded response model with five ordered categories was used to generate datasets of a 30-item test for samples of 300 and 1,000 simulated examinees. All DIF comparisons were simulated by randomly pairing two groups of examinees. Three different sample size combinations of reference and focal groups were simulated under two ability matching conditions. For each of the six combinations of sample sizes by ability matching conditions, 100 replications of DIF detection comparisons were simulated. Item parameter estimates and likelihood values were obtained by marginal maximum likelihood estimation using the computer program MULTILOG. Irpe I eryor rates of the likelihood ratio test statistics for all six combinations of sample sizes and ability matching conditions were within theoretically expected values at each of the nominal alpha levels considered.
Get full access to this article
View all access options for this article.
References
1.
Ankenmann, R. D. , Witt, E. A., & Dunbar, S. B. (1996, April). An investigation of the power of the likelihood ratio goodness-of-fit statistic in detecting differential item functioning. Paper presented at the annual meeting of the American Educational Research Association, New York.
2.
Baker, F. B. (1988). GENIRV: A program to generate item response vectors [Computer program]. Madison: University of Wisconsin, Department of Educational Psychology, Laboratory of Experimental Design.
3.
Baker, F. B. (1992). Equating tests under the graded response model. Applied Psychological Measurement, 16, 87-96.
4.
Cohen, A. S. , Kim, S.-H., & Baker, F. B. (1993). Detection of differential item functioning in the graded response model. Applied Psychological Measurement, 17, 335-350.
5.
Cohen, A. S. , Kim, S.-H., & Wollack, J. A. (1996). An investigation of the likelihood ratio test for detection of differential item functioning. Applied Psychological Measurement, 20, 15-26.
6.
Flowers, C. P. , Oshima, T. C., & Raju, N. S. (1995, April). A Monte Carlo assessment of DFIT with polytomously scored unidimensional tests. Paper presented at the annual meeting of the American Educational Research Association, San Francisco.
7.
Kim, S.-H. , & Cohen, A. S. (1992). Effects of linking methods on detection of DIF. Journal of Educational Measurement, 29, 51-66.
8.
Kim, S.-H. , & Cohen, A. S. (1995). A comparison of Lord's chi-square, Raju's area measures, and the likelihood ratio test in detection of differential item functioning. Applied Measurement in Education, 8, 291-312.
9.
Lord, F. M. (1980). Applications of item response theory to practical testing problems. Hillsdale NJ: Erlbaum.
10.
Mantel, N. (1963). Chi-square tests with one degree of freedom; extensions of the Mantel-Haenszel procedure. Journal of the American Statistical Association, 58, 690-700.
11.
Mislevy, R. J. (1985). Estimation of latent group effects. Journal of the American Statistical Association, 80, 993-997.
12.
Neyman, J. , & Pearson, E. S. (1928). On the use and interpretation of certain test criteria for purposes of statistical inference: Part I and Part II. Biometrika, 20A, 174-240, 263-294.
13.
Pine, S. M. (1977). Application of item characteristic curve theory to the problem of test bias. In D. J. Weiss (Ed.), Application of computerized adaptive testing: Proceedings of a symposium presented at the 18th annual convention of the Military Testing Association (Research Rep. No. 77-1, pp. 37-43). Minneapolis: University of Minnesota, Department of Psychology, Psychometric Methods Program.
14.
Raju, N. S. (1988). The area between two item characteristic curves. Psychometrika, 53, 495-502.
15.
Raju, N. S. (1990). Determining the significance of estimated signed and unsigned areas between two item response functions. Applied Psychological Measurement, 14, 197-207.
16.
Reise, S. P. , & Yu, J. (1990). Parameter recovery in the graded response model using MULTILOG. Journal of Educational Measurement, 27, 133-144.
17.
Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores. Psychometrika Monograph Supplement, No. 17.
18.
Samejima, F. (1972). A general model for freeresponse data. Psychometrika Monograph Supplement, No. 18.
19.
Shepard, L. , Camilli, G., & Williams, D. M. (1984). Accounting for statistical artifacts in item bias research. Journal of Educational Statistics, 9, 93-128.
20.
Stocking, M. , & Lord, F. M. (1983). Developing a common metric in item response theory. Applied Psychological Measurement, 7, 201-210.
21.
Thissen, D. (1991). MULTILOG user's guide: Multiple, categorical item analysis and test scoring using item response theory [Computer program]. Chicago: Scientific Software.
22.
Thissen, D. , Steinberg, L., & Gerrard, M. (1986). Beyond group mean differences: The concept of item bias. Psychological Bulletin, 99, 118-128.
23.
Thissen, D. , Steinberg, L., & Wainer, H. (1988). Use of item response theory in the study of group differences in trace lines. In H. Wainer & H. I. Braun (Eds.), Test validity (pp. 147-169). Hillsdale NJ: Erlbaum.
24.
Thissen, D. , Steinberg, L., & Wainer, H. (1993). Detection of differential item functioning using the parameters of item response models. In P. W. Holland & H. Wainer (Eds.), Differential item functioning (pp. 67-113). Hillsdale NJ: Erlbaum.
25.
Webb, N. L. (1994). Wisconsin performance assessment development project: Analysis and technical report for fiscal year 1993-94. Madison: University of Wisconsin, Wisconsin Center for Educational Research.
