Differential item functioning (DIF) occurs when an item on a test, questionnaire, or interview has different measurement properties for one group of people versus another. One way to test items with ordinal response scales for DIF is likelihood ratio (LR) testing using item response theory (IRT), or IRT-LR-DIF. Despite the various advantages of IRT-LR-DIF, one disadvantage is that the latent variable is usually assumed to be normally distributed. If this normality assumption is violated, nonparametric alternatives such as the Mantel test, generalized Mantel—Haenszel (GMH) test, and poly-SIBTEST may be preferable. Simulations were carried out to compare IRT-LR-DIF to poly-SIBTEST and the GMH and Mantel tests when the latent density is nonnormal for both groups but presumed normal for IRT-LR-DIF. Results indicated that latent nonnormality detrimentally affected all three procedures, but IRT-LR-DIF was surprisingly more robust to latent nonnormality than all of the nonparametric approaches.
Ankenmann, R.D., Witt, E.A., & Dunbar, S.B. ( 1999). An investigation of the power of the likelihood ratio goodness-of-fit statistic in detecting differential item functioning. Journal of Educational Measurement, 36, 277-300.
2.
Bock, R.D., & Aitkin, M. ( 1981). Marginal maximum likelihood estimation of item parameters: Application of an EM algorithm. Psychometrika, 46, 443-459.
3.
Bock, R.D., Muraki, E., & Pfeiffenberger, W. (1988). Item pool maintenance in the presence of item parameter drift. Journal of Educational Measurement, 25, 275-285.
4.
Bolt, D. ( 2002). A Monte Carlo comparison of parametric and nonparametric polytomous DIF detection methods. Applied Measurement in Education , 15, 113-141.
5.
Bolt, D.M., Hare, R.D., Vitale, J.E., & Newman, J.P. ( 2004). A Multigroup item response theory analysis of the Psychopathy Checklist-Revised. Psychological Assessment, 16, 155-168.
6.
Camilli, G., & Congdon, P. ( 1999). Application of a method of estimating DIF for polytomous test items. Journal of Educational and Behavioral Statistics , 24, 323-341.
7.
Camilli, G., & Shepard, L.A. ( 1994). Methods for identifying biased test items. Thousand Oaks, CA: Sage.
8.
Chang, H.H., Mazzeo, J., & Roussos, L. ( 1996). Detecting DIF for polytomously scored items: An adaptation of the SIBTEST procedure. Journal of Educational Measurement , 33, 333-353.
9.
Cohen, A.S., Kim, S., & Wollack, J.A. ( 1996). An investigation of the likelihood ratio test for detection of differential item functioning. Applied Psychological Measurement , 20, 15-26.
10.
Dorans, N.J., & Holland, P.W. ( 1993). DIF detection and description: Mantel-Haenszel and standardization . In P. W. Holland & H. Wainer (Eds.), Differential item functioning (pp. 35-66). Hillsdale, NJ: Erlbaum.
11.
Hill, C.D. ( 2004). Precision of parameter estimates for the graded item response model (Unpublished master’s thesis). University of North Carolina at Chapel Hill.
12.
Holland, P.W., & Thayer, D.T. ( 1988). Differential item performance and the Mantel-Haenszel procedure. In H. Wainer & H. I. Braun (Eds.), Test validity (pp. 129-145). Hillsdale, NJ: Erlbaum.
13.
Kim, S., & Cohen, A.S. ( 1995). A comparison of Lord’s chi-square, Raju’s area measures, and the likelihood ratio test on detection of differential item functioning. Applied Measurement in Education, 8, 291-312.
14.
Kim, S., & Cohen, A.S. ( 1998). Detection of differential item functioning under the graded response model with the likelihood ratio test. Applied Psychological Measurement, 22, 345-355.
15.
Liu, I.-M., & Agresti, A. ( 1996). Mantel-Haenszel-type inference for cumulative odds ratios with a stratified ordinal response. Biometrics, 52, 1223-1234.
16.
Mantel, N. ( 1963). Chi-square tests with one degree of freedom: Extensions of the Mantel-Haenszel procedure. Journal of the American Statistical Association, 58, 690-700.
17.
Mantel, N., & Haenszel, W. ( 1959). Statistical aspects of the analysis of data from retrospective studies of disease. Journal of the National Cancer Institute , 22, 719-748.
18.
Masters, G.N. ( 1982). A Rasch model for partial credit scoring. Psychometrika, 47, 149-174.
19.
Mellenbergh, G.J. ( 1989). Item bias and item response theory. International Journal of Educational Research, 13, 127-143.
20.
Mellenbergh, G.J. ( 1995). Conceptual notes on models for discrete polytomous item responses. Applied Psychological Measurement, 19, 91-100.
21.
Miller, M.D., & Oshima, T.C. ( 1992). Effect of sample size, number of biased items and magnitude of bias on a two stage item bias estimation method. Applied Psychological Measurement, 16, 381-388.
22.
Muraki, E. ( 1992). A generalized partial credit model: Application of an EM algorithm. Applied Psychological Measurement, 16, 159-176.
23.
Muraki, E. ( 2003). 7.3 Models in PARSCALE. In M. du Toit (Ed.), IRT from SSI (pp. 544-566). Lincolnwood, IL: Scientific Software International .
24.
Muraki, E., & Bock, R.D. ( 2003). PARSCALE-4: IRT item analysis and test scoring for rating-scale data [Computer software]. Chicago, IL: Scientific Software International.
25.
Navas-Ara, M.J., & Gόmez-Benito, J. (2002). Effects of ability scale purification on the identification of DIF. European Journal of Psychological Assessment, 18, 9-15.
26.
Orlando, M., & Marshall, G.N. (2002). Differential item functioning in a Spanish translation of the PTSD checklist: Detection and evaluation of impact. Psychological Assessment, 14, 50-59.
27.
Park, D., & Lautenschlager, G.J. (1990). Improving IRT item bias detection with iterative linking and ability scale purification. Applied Psychological Measurement, 14, 163-173.
Penfield, R.D. ( 2007). Assessing differential step functioning in polytomous items using a common odds ratio estimator. Journal of Educational Measurement, 44, 187-210.
30.
Penfield, R.D., & Algina, J. ( 2003). Applying the Liu-Agresti estimator of the cumulative odds ratio to DIF detection in polytomous items. Journal of Educational Measurement, 40, 353-370.
31.
Reise, S.P., Widaman, K.F., & Pugh, R.H. ( 1993). Confirmatory factor analysis and item response theory: Two approaches for exploring measurement invariance. Psychological Bulletin, 114, 552-566.
32.
Roussos, L., & Stout, W. ( 2005). Dimensionality-based DIF package: Poly-SIBTEST (version 1.7) [Computer software]. Urbana-Champaign, IL: William Stout Institute for Measurement.
33.
Samejima, F. ( 1969). Estimation of latent ability using a response pattern of graded scores. Psychometrika Monograph, No. 17.
34.
Samejima, F. ( 1997). Graded response model. In W. J. van der Linden & R. K. Hambleton (Eds.), Handbook of modern item response theory (pp. 85-100). New York, NY: Springer.
35.
Shealy, R., & Stout, W. ( 1993). A model-based standardization approach that separates true bias/DIF from group ability differences and detects test bias/DTF as well as item bias/DIF. Psychometrika, 58, 159-194.
36.
Somes, G.W. ( 1986). The generalized Mantel-Haenszel statistic. The American Statistician, 40, 106-108.
37.
Stark, S., Chernyshenko, O.S., & Drasgow, F. ( 2006). Detecting differential item functioning with confirmatory factor analysis and item response theory: Toward a unified strategy. Journal of Applied Psychology, 91, 1291-1306.
38.
Steinberg, L., & Thissen, D. ( 2006). Using effect sizes for research reporting: Examples using item response theory to analyze differential item functioning. Psychological Methods, 11, 402-415.
39.
Su, Y., & Wang, W. ( 2005). Efficiency of the Mantel, generalized Mantel-Haenszel, and logistic discriminant function analysis methods in detecting differential item functioning for polytomous items. Applied Measurement in Education, 18, 313-350.
40.
Sweeney, K.P. ( 1996). A Monte-Carlo investigation of the likelihood-ratio procedure in the detection of differential item functioning (Unpublished doctoral dissertation) . Fordham University, New York.
41.
Thissen, D. ( 2001). IRTLRDIF v2.0b: Software for the computation of the statistics involved in item response theory likelihood-ratio tests for differential item functioning [Documentation for computer program]. L.L. Thurstone Psychometric Laboratory, University of North Carolina at Chapel Hill .
42.
Thissen, D., Steinberg, L., & Gerrard, M. ( 1986). Beyond group-mean differences: The concept of item bias . Psychological Bulletin, 99, 118-128.
43.
Thissen, D., Steinberg, L., & Wainer, H. ( 1988). Use of item response theory in the study of group difference in trace lines. In H. Wainer & H. Braun (Eds.), Test validity (pp. 147-169). Hillsdale, NJ: Erlbaum.
44.
Thissen, D., Steinberg, L., & Wainer, H. ( 1993). Detection of differential item functioning using the parameters of item response models. In P. W. Holland & H. Wainer (Eds.), Differential item functioning (pp. 67-113). Hillsdale, NJ: Erlbaum.
45.
Wainer, H. ( 1993). Model-based standardized measurement of an item’s differential impact. In P. W. Holland & H. Wainer (Eds.), Differential item functioning (pp. 123-135). Hillsdale, NJ: Erlbaum.
46.
Wainer, H., Sireci, S.G., & Thissen, D. ( 1991). Differential testlet functioning: Definitions and detection . Journal of Educational Measurement, 28, 197-219.
47.
Wang, W., & Su, Y. ( 2004). Factors influencing the Mantel and generalized Mantel-Haenszel methods for the assessment of differential item functioning in polytomous items. Applied Psychological Measurement, 28, 450-480.
48.
Wang, W., & Yeh, Y. ( 2003). Effects of anchor item methods on differential item functioning detection with the likelihood ratio test. Applied Psychological Measurement, 27, 479-498.
49.
Welch, C.J., & Hoover, H.D. ( 1993). Procedures for extending item bias techniques to polytomously scored items. Applied Measurement in Education, 6, 1-19.
50.
Woods, C.M. ( 2006). Ramsay-curve item response theory to detect and correct for nonnormal latent variables. Psychological Methods, 11, 253-270.
Woods, C.M. ( 2008b). IRT-LR-DIF with estimation of the focal-group density as an empirical histogram. Educational and Psychological Measurement , 68, 571-586.
53.
Woods, C.M. ( 2009). Empirical selection of anchors for tests of differential item functioning. Applied Psychological Measurement, 33, 42-57.
Woods, C.M. (in press). DIF testing with an empirical-histogram approximation of the latent density for each group. Applied Measurement in Education .
56.
Woods, C.M., & Thissen, D. ( 2006). Item response theory with estimation of the latent population distribution using spline-based densities. Psychometrika, 71, 281-301.
57.
Zwick, R., Donoghue, J., & Grima, A. ( 1993). Assessment of differential item functioning for performance tasks. Journal of Educational Measurement, 30, 233-251.
58.
Zwick, R., Thayer, D.T., & Mazzeo, J. ( 1997). Descriptive and inferential procedures for assessing differential item functioning in polytomous items. Applied Measurement in Education, 10, 321-344.
