Internal measures of differential functioning of items and tests (DHFIT) based on item response theory (IRT) are proposed. Within the DFIT context, the new differential test functioning (DTF) index leads to two new measures of differential item functioning (DIF) with the following properties: (1) The compensatory DIF (CDIF) indexes for all items in a test sum to the DTF index for that test and, unlike current DIF procedures, the CDIF index for an item does not assume that the other items in the test are unbi ased ; (2) the noncompensatory DIF (NCDIF) index, which assumes that the other items in the test are unbiased, is comparable to some of the IRT-based DIP indexes; and (3) COIF and NCDIF, as well as DTF, are equally valid for polytomous and multidimensional IRT models. Monte carlo study results, comparing these indexes with Lord's χ2 test, the signed area measure, and the unsigned area measure, demonstrate that the DFIT framework is accu rate in assessing DTF, COIF, and NCDIF.
Angoff, W.H., & Ford, S.F. (1973). Item-race interaction on a test of scholastic aptitude. Journal of Educational Measurement, 10, 95-105.
2.
Baker, F.B. (1990). Some observations on the metric of PC-BILOG results . Applied Psychological Measurement, 14, 139-150.
3.
Baker, F.B., Al-Kami, A., & Al-Dosary, I.M. (1991). EQUATE: A computer program for the test characteristic method of IRT equating. Applied Psychological Measurement, 15, 78.
4.
Candell, G.L., & Drasgow, F. (1988). An iterative procedure for linking metrics and assessing item bias in item response theory. Applied Psychological Measurement, 12, 253-260.
5.
Cohen, A.S., & Kim, S. (1993). A comparison of Lord's χ2 and Raju's area measures on detection of DIF. Applied Psychological Measurement , 17, 39-52.
6.
Dorans, N.J. (1986, April). Two new approaches to assessing unexpected differential item preference: Standardization and the Mantel-Haenszel methods . Paper presented at the annual meeting of the American Educational Research Association, San Francisco .
7.
Fleer, P.F. (1993). A Monte Carlo assessment of a new measure of item and test bias. (Doctoral dissertation, Illinois Institute of Technology. Dissertation Abstracts International, 54-04, 2266B.
8.
Fleer, P.F., Kiley, K.A., & Raju, N.S. (1991).RANGEN [Computer program]. Unpublished computer program, Illinois Institute of Technology, Chicago.
9.
Flowers, C.P., Oshima, T.C., & Raju, N.S. (1995, April). A monte carlo assessment of DFIT with polytomously scored unidimensional tests. Paper presented at the annual meeting of the American Educational Research Association, San Francisco.
10.
Holland, P.W., & Thayer, D.T. (1988). Differential item performance and the Mantel-Haenszel procedure. In H. Wainer & H. I. Braun (Eds.), Test validity (pp. 129-145). Hillsdale NJ: Erlbaum.
11.
Kim, S.-H., & Cohen, A.S. (1991). A comparison of two area measures for detecting differential item functioning. Applied Psychological Measurement , 15, 269-278.
12.
Kim, S.-H., & Cohen, A.S. (1992). Effects of linking methods on detection of DIP . Journal of Educational Measurement, 29, 51-66.
13.
Lim, R.G., & Drasgow, F. (1990). Evaluation of two methods for estimating item response theory parameters when assessing differential item functioning. Journal of Applied Psychology, 75, 164-174.
14.
Lord, F.M. (1980). Applications of item response theory to practical testing problems. Hillsdale NJ: Erlbaum.
15.
Mellenbergh, G.J. (1982). Contingency table models for assessing item bias . Journal of Educational Measurement, 7, 105-108.
Mislevy, R.J., & Bock, R.D. (1990). BILOG: Item analysis and test scoring with binary logistic models. Mooresville IN: Scientific Software.
18.
Oshima, T.C., Raju, N.S., & Flowers, C.P. (1993, April). Evaluation of a multidimensional IRT-based DIF/DTF index. Paper presented at the annual meeting of the American Educational Research Association, Atlanta .
19.
Oshima, T.C., Raju, N.S., Flowers, C.P., & Monaco, M. (1995, April). A monte carlo assessment of DFIT with dichotomously scored multidimensional tests. Paper presented at the annual meeting of the American Educational Research Association, San Francisco.
20.
Raju, N.S. (1988). The area between two item characteristic curves . Psychometrika, 53, 495-502.
21.
Raju, N.S. (1990). Determining the significance of estimated signed and unsigned areas between two item response functions. Applied Psychological Measurement, 14, 197-207.
22.
Raju, N.S., van der Linden, W.J., & Fleer, P.F. (1992, April). An IRT-based internal measure of test bias
23.
with applications for differential item functioning. Paper presented at the annual meeting of the American Educational Research Association, San Francisco.
Scheuneman, J.D. (1979). A method for assessing bias in test items. Journal of Educational Measurement, 16, 143-152.
26.
Shealy, R.T., & Stout, W.F. (1993). A model-based standardization approach that separates true bias/DIF from ability group differences and detects test bias/DTF as well as item bias/DIF. Psychometrika, 58, 159-194.
27.
Shepard, L.A., Camilli, G., & Averill, M. (1981). Comparison of procedures for detecting test-item bias with both internal and external ability criteria. Journal of Educational Statistics, 6, 317-375.
28.
Stocking, M.L., & Lord, F.M. (1983). Developing a common metric in item response theory . Applied Psychological Measurement, 7, 201-210.
29.
Swaminathan, H., & Rogers, H.J. (1990). Detecting differential item functioning using the logistic regression procedure. Journal of Educational Measurement, 27, 361-370.
30.
Thissen, D., Steinberg, L., & Wainer, H. (1988). Use of item response theory in the study of group differences in trace lines. In H. Wainer & H. I. Braun (Eds.), Test validity (pp. 147-169). Hillsdale NJ: Erlbaum.
31.
Wainer, H. (1993). Model-based standardized measurement of an item's differential impact. In P. W. Holland & H. Wainer (Eds.), Differential item functioning (pp. 123-135). Hillsdale NJ: Erlbaum.
