A Comparison Between Some Generalized Mantel-Haenszel Statistics for Detecting DIF in Data Simulated Under the Graded Response Model

Abstract

The main objective of this study was to establish the relative efficacy of the generalized Mantel-Haenszel test (GMH) and the Mantel test for detecting large numbers of differential item functioning (DIF) patterns. To this end this study considered a topic not dealt with in the literature to date: the possible differential effect of type of scores assigned to item-response categories on the power and Type I error rate of the Mantel test. For this purpose, a simulation study with data generated under the graded response model was carried out. The results showed that (a) the scoring system used to compute the Mantel test influences its power for detecting DIF and (b) for conditions comparable to those simulated, the GMH may be the best option, given that it is capable of detecting more complex patterns of association than the Mantel test, that is, more types of DIF.

Keywords

differential item functioning generalized Mantel-Haenszel statistics graded response model Mantel-Haenszel methods Mantel test polytomous items

Get full access to this article

View all access options for this article.

References

Aptech Systems. (1993). GAUSS (version 3.1.4) [Computer programming language]. Maple Valley, WA: Aptech Systems.

Fidalgo, A.M. (in press). GMHDIF: A computer program for detecting DIF in dichotomous and polytomous items using generalized Mantel-Haenszel statistics. Applied Psychological Measurement.

Fidalgo, A.M. , & Madeira, J.M. ( 2008). Generalized Mantel-Haenszel methods for DIF detection . Educational and Psychological Measurement, 68, 940-958.

Fidalgo, A.M. , & Scalon, J.D. ( 2010). Using generalized Mantel-Haenszel statistics to assess DIF among multiple groups. Journal of Psychoeducational Assessment , 28, 60-69.

Kristjansson, E. , Aylesworth, R. , McDowell, I. , & Zumbo, B.D. ( 2005). A comparison of four methods for detecting differential item functioning in ordered response items. Educational and Psychological Measurement, 65, 935-953.

Landis, J.R. , Heyman, E.R. , & Koch, G.G. ( 1978). Average partial association in three-way contingency tables: A review and discussion of alternative tests. International Statistical Review, 46, 237-254.

Mantel, N. ( 1963). Chi-square tests with one degree of freedom: Extension of the Mantel-Haenszel procedure. Journal of the American Statistical Association, 58, 690-700.

Mantel, N. , & Haenszel, W. ( 1959). Statistical aspects of the analysis of data from retrospective studies of disease. Journal of the National Cancer Institute , 22, 719-748.

Wang, W.-C. , & Su, Y.-H. ( 2004). Factors influencing the Mantel and generalized Mantel-Haenszel methods for the assessment of differential item functioning in polytomous items. Applied Psychological Measurement, 28, 450-480.