An item with differential item functioning (DIF) displays different statistical properties, conditional on a matching variable. The presence of DIF in measures can invalidate the conclusions of medical outcome studies. Numerous approaches have been developed to examine DIF in many areas, including education and health-related quality of life. There is little consensus in the research community regarding selection of one best method, and most methods require large sample sizes. This article describes some approaches to examine DIF with small samples (e.g., less than 200).
Anastasi, A., & Urbina, S. (1997). Psychological testing. Upper Saddle River, NJ: Prentice Hall.
2.
Angoff, W. H., & Ford, S. F. (1973). Item-race interaction on a test of scholastic aptitude. Journal of Educational Measurement, 10, 95-105.
3.
Bergstrom, B. A., Gershon, R. C., & Brown, W. L. (1993, April). Differential item functioning vs. differential test functioning. Paper presented at the annual meeting of the American Educational Research Association, Atlanta. (ERIC Document Reproduction No. TMO22402)
4.
Bergstrom, B. A., Gershon, R. C., & Brown, W. L. (1994, April). The effect of ability on differential item functioning. Paper presented at the annual meeting of the American Educational Research Association, New Orleans.
5.
Camilli, G., & Shepard, L. A. (1994). Methods for identifying biased test items. Thousand Oaks, CA: Sage.
6.
Chang, C.-H. & Reeve, B. B. (2005). Item Response Theory and Its applications to patient-reported outcomes measurement. Evaluation & the Health Professions, 28(3), 264-282.
7.
Cole, N. S., & Moss, P. A. (1989). Bias in test use. In R. L. Linn (Ed.), Educational measurement (3rd ed., pp. 201-219). New York: American Council on Education.
8.
Crane, P. K., van Belle, G., & Larson, E. G. (2004). Test bias in a cognitive test: Differential item functioning in the CASI. Statistics in Medicine, 23, 241-256.
9.
Dorans, N. J., & Holland, P. W. (1993). DIF detection and description: Mantel-Haenszel and standardization. In P. W. Holland & H. Wainer (Eds.), Differential item functioning (pp. 35-66). Hillsdale, NJ: Lawrence Erlbaum.
10.
Embretson, S. E., & Reise, S. P. (2000). Item response theory for psychologists. Mahwah, NJ: Lawrence Erlbaum.
11.
Fidalgo, A. M., Ferreres, D., & Muniz, J. (2004). Utility of the Mantel-Haenszel procedure for detecting differential item functioning in small samples. Educational and Psychological Measurement, 64, 925-936.
12.
Fleishman, J. A., & Lawrence, W. F. (2003). Demographic variation in SF-12 scores: True differences or differential item functioning. Medical Care, 41, III-75-III-86.
13.
Flower, C. P., Oshima, T. C., & Raju, N. S. (1999). A description and demonstration of the polytomous DFIT framework. Applied Psychology Measurement, 23, 309-326.
14.
Hambleton, R. K. (1989). Principles and selected applications of item response theory. In R. L. Linn (Ed.), Educational measurement (3rd ed., pp. 147-200). New York: American Council on Education.
15.
Hidalgo, M. D., & Lopez-Pina, J. A. (2004). Differential item functioning detection and effect size: A comparison between logistic regression and Mantel-Haenszel procedures. Educational and Psychological Measurement, 6, 903-915.
16.
Holland, P. W., & Thayer, D. T. (1988). Differential item functioning and the Mantel-Haenszel procedures. In H. Wainer& H. Brain (Eds.), Test validity. Hillsdale, NJ: Lawrence Erlbaum.
17.
Jodoin, M. G., & Gierl, M. J. (2001). Evaluating type I error and power rates using an effect size measure with the logistic regression procedure for DIF detection. Applied Measurement in Education, 14, 329-349.
18.
Lai, J.-S., Cella, D., Chang, C.-H., Bode, R., & Heinemann, A. (2003). Item banking to improve, shorten and computerize self-reported fatigue. Quality of Life Research, 12, 485-501.
19.
Mantel, N., & Haenszel, W. (1959). Statistical aspects of the analysis of data from retrospective studies of disease. Journal of the National Cancer Institute, 22, 719-748.
20.
Mazor, K. M., Clauser, B. E., & Hambleton, R. K. (1992). The effect of sample size on the functioning of the Mantel-Haenszel statistic. Educational & Psychological Measurement, 52(2), 443-451.
21.
Mellenbergh, G. J. (1982). Contingency table models for assessing item bias. Journal of Educational Statistics, 7, 105-118.
22.
Osterlind, S. J. (1983). Test item bias. Newbury Park, CA: Sage.
23.
Raju, N. S. (1988). The area between two item characteristic curves. Psychometrika, 53(4), 495-502.
24.
Raju, N. S. (1990). Determining the significance of estimated signed and unsigned areas between two item response functions. Applied Psychological Measurement, 14(2), 197-207.
25.
Raju, N. S. (1991). Determining the significance of estimated signed and unsigned areas between two item response functions: Correction. Applied Psychological Measurement, 15(4), 352.
26.
Raju, N. S. (2000). DFITPS6: A computer program for analyzing differential item functioning [Computer software]. Chicago: Illinois Institute of Technology.
27.
Raju, N. S., Laffitte, L. J., & Byrne, B. M. (2002). Measurement equivalence: A comparison of methods based on confirmatory factor analysis and item response theory. Journal of Applied Psychology, 87, 517-529.
28.
Raju, N. S., van der Linden, W., & Fleer, P. (1995). An IRT-based internal measure of test bias with applications for differential item functioning. Applied Psychological Measurement, 19, 353-368.
29.
Scheuneman, J. D. (1979). A new method of assessing bias in test items. Journal of Educational Measurement, 16, 143-152.
30.
Shtatland, E. S., Moore, S., & Barton, M. B. (2000, April 9-12). Why we need an R2 measure of fit (and not only one) in PROC LOGISTIC and PROC GENMOD. Proceedings of the 25th annual SAS users group international conference, Indianapolis, Indiana.
31.
Swaminathan, H., & Rogers, H. J. (1990). Detecting differential item functioning using logistic regression procedures. Journal of Educational Measurement, 27(4), 361-370.
32.
Teresi, J. A. (2001). Statistical methods for examination of Differential Item Functioning (DIF) with applications to cross-cultural measurement of functional, physical, and mental health. Journal of Mental Health and Aging, 7, 31-40.
33.
Teresi, J. A. (2004). Differential item functioning and health assessment. Retrieved December, 2004, from http://outcomes.cancer.gov/conference/irt/teresi.pdf
34.
Teresi, J. A., & Holmes, D. (1994). Overview of methodological issues in gerontological and geriatric measurement. In M. Lawton & J. Teresi (Eds.), Annual review of gerontology and geriatrics: Focus on assessment techniques (Vol. 14, pp. 1-22). New York: Springer.
35.
Teresi, J. A., Kleinman, M., & Ocepek-Welikson, K. (2000). Modern psychometric methods for detection of differential item functioning: Application to cognitive assessment measures. Statistics in Medicine, 19, 1651-1683.
36.
Thissen, D. (2001). IRTLRDIF v2.0b: Software for the computation of the statistics involved in item response theory likelihood-ratio tests for differential item functioning. Retrieved December, 2004, from http://www.unc.edu/%7Edthissen/
37.
Wright, B. D., & Panchapakesan, N. (1969). A procedure for sample-free item analysis (MESA Research Memoranda No. 46). Chicago: MESA Press.
38.
Wright, B. D., & Stone, M. (1979). Best test design. Chicago: MESA Press.
39.
Zumbo, B. D. (1999). A handbook on the theory and methods of differential item functioning (DIF): Logistic regression modeling as a unitary framework for binary and Likert-type (ordinal) item scores. Ottawa, Ontario, Canada: Directorate of Human Resources Research and Evaluation, Department of National Defense.
