This paper presents a conceptual overview of the Mantel-Haenszel (MH) technique for evaluating item bias. It then briefly contrasts MH with Rasch methodology and provides an applied example wherein MH is used to examine possible racial/ethnic and gender bias on the Guide to the Assessment of Test Session Behavior. Results from the applied example show that among the 80 possible instances of bias, 10 were statistically significant. This number barely exceeded the chance rate of 9 significant comparisons. Results therefore align with findings from previous studies that show that test observations display few instances of bias. Implications are discussed.
Get full access to this article
View all access options for this article.
References
1.
Anastasi, A. (1988). Psychological testing (6th ed.). New York: Macmillan.
2.
Bracken, B. (1991). The clinical observation of preschool assessment behavior. In B. A. Bracken (Ed.), The psychoeducational assessment of preschool children (pp. 40-52). Boston: Allyn & Bacon.
3.
Camilli, G., & Smith, J. K. (1990). Comparison of the Mantel-Haenszel test with a randomized and a jacknife test for detecting item bias. Journal of Educational Statistics, 15, 53-67.
4.
Divgi, D. R. (1981, April). Potential pitfalls in application of item response theory. Paper accompanying discussant comments at the annual meeting of the National Council on Measurement in Education, Los Angeles.
5.
Dorans, N. J. (1989). Two new approaches to assessing differential item functioning: Standardization and the Mantel-Haenszel method. Applied Psychological Measurement, 2, 217-233.
6.
Elliott, C. D. (1990). The Differential Ability Scales. San Antonio, TX: Psychological Corporation.
7.
Vol. 42, No. 163.
8.
Glutting, J. J., & McDermott, P. A. (1988). Generality of test-session observations to kindergartners' classroom behavior. Journal of Abnormal Clinical Child Psychology, 16, 527-537.
9.
Glutting, J. J., & Oakland, T. (in press). Guide to the Assessment of Test-Session Behavior. San Antonio, TX: Psychological Corporation.
10.
Glutting, J. J., Oakland, T., & McDermott, P. A. (1989). Observing children's behavior during testing: Constructs, validity, and situational generality. Journal of School Psychology, 27, 155-164.
11.
Gutkin, T. B., & Reynolds, C. R. (1981). Factorial similarity of the WISC-R for Anglos and Chicanos referred for psychological services. Journal of Educational Psychology, 73, 227-231.
12.
Hambleton, R. K., & Rogers, H. J. (1989). Detecting potentially biased test items: Comparison of IRT area and Mantel-Haenszel methods. Applied Measurement in Education, 2, 313-314.
13.
Hambleton, R. K., & Swaminathan, H. (1985). Item response theory: Principles and applications. Boston: Kluwer-Nyjhoff.
14.
Heller, K., Holtsman, W., & Messick, S. (Eds.). (1982). Placing children in special education: A strategy for equality. Washington: National Academy Press.
15.
Hilliard, A. G. (1984). IQ testing as the emperor's new clothes: A critique of Jensen's Bias in mental testing. In C. R. Reynolds & R. T. Brown (Eds.), Perspectives on bias in mental testing (pp. 139-169). New York: Plenum Press.
16.
Hills, J. R. (1989). Screening for potentially biased items in testing programs. Educational Measurement: Issues and Practices, 20, 5-11.
17.
Holland, P. W. (1985, October). On the study of differential item performance without IRT. Paper presented at the meeting of the Military Testing Association, San Diego, CA.
18.
Holland, P. W., & Thayer, D. T. (1988). Differential item performance and the Mantel-Haenszel procedure. In H. Wainer & H. I. Braum (Eds.), Test validity (pp. 129-145). Hillsdale, NJ: Erlbaum.
19.
Hulin, C. L., Drasgow, F., & Parsons, L. K. (1983). Item response theory. Homewood, IL: Jones-Irwin.
20.
Ironson, G. H. (1982). Use of chi-square and latent trait approaches for detecting item bias. In R. A. Berk (Ed.), Handbook of methods for detecting item bias (pp. 117-160). Baltimore: Johns Hopkins University Press.
21.
Jensen, A. R. (1980). Bias in mental testing. New York: Free Press.
22.
Jensen, A. R. (1981). Straight talk about mental tests. New York: Free Press.
23.
Kamin, L. J. (1974). The science and politics of IQ. New York: Halsted Press.
24.
Kaufman, A. S., & DiCuio, R. (1975). Separate factor analyses of the McCarthy Scales for groups of black and white children. Journal of School Psychology, 13, 10-18.
25.
Kaufman, A. S., & Hollenbeck, G. P. (1974). Comparative structure of the WPPSI for Black and White children. Journal of Clinical Psychology, 30, 316-319.
26.
Kaufman, A. S., & Kaufman, N. L. (1983). Kaufman Assessment Battery for Children: Interpretive manual. Circle Pines, MN: American Guidance Service.
27.
Kaufman, A. S., & Reynolds, C. R. (1984). Intellectual and academic achievement tests. In T. H. Ollendick & M. Hersen (Eds.), Child behavioral assessment: Principles and procedures (pp. 195-220). New York: Pergamon Press.
28.
Larry P. vs. Riles, Supp. 926 (N.D. Cal. 1979) (decision on merits). Order modifying judgment, C-71-2270 RFP, September 25, 1986.
29.
Linacre, J. M. (1988, April). The practical realization of the standard error of the Mantel-Haenszel statistic. Paper presented at the annual meeting of the American Educational Research Association, New Orleans.
30.
Mantel, N., & Haenszel, W. (1959). Statistical aspects of the retrospective study of disease. Journal of the National Cancer Institute, 22, 719-748.
31.
Marshall et al. vs. Georgia. U.S. District Court for the Southern District of Georgia, CV482-233, June 28, 1984; Affd (1 Ith cir. no. 84-8771, October 19, 1985).
32.
Mercer, J. R. (1984). What is a racially and culturally nondiscriminatory test? A sociologial and pluralistic response. In C. R. Reynolds & R. T. Brown (Eds.), Perspectives on bias in mental testing (pp. 293-356). New York: Plenum Press.
33.
Mishra, S. P. (1981). Factor analysis of the McCarthy Scales for groups of White and Mexican-American children. Journal of School Psychology, 19, 178-182.
34.
Naglieri, J. A., & Hill, D. (1986). Comparison of WISC-R and K-ABC regression lines with Black and White children. Journal of Clinical Psychology, 15, 352-355.
35.
Naglieri, J. A., & Jensen, A. R. (1987). Comparison of Black-White differences on the WISC-R and the K-ABC: Spearman's hypothesis. Intelligence, 11, 21-43.
36.
Oakland, T. (Ed.). (1977). Psychological and educational assessment of minority children. New York: Brunner/Mazel.
37.
Oakland, T., & Feigenbaum, D. (1979). Multiple sources of test bias on the WISC-R and the Bender-Gestalt Test. Journal of Consulting and Clinical Psychology, 47, 968-974.
38.
Oakland, T., & Glutting, J. J. (1990). Examiner observations of children's WISC-R test-related behaviors: Possible SES, race, and gender effects. Psychological Assessment: A Journal of Consulting and Clinical Psychology, 2, 86-90.
39.
Raju, N. S., Bode, R. K., & Larsen, V. S. (1989). An empirical assessment of the Mantel-Haenszel statistic for studying differential item performance. Applied Measurement in Education, 2, 1-13.
40.
Reschly, D. J. (1978). WISC-R factor structures among Anglos, Blacks, Chicanos, and Native American Papagos. Journal of Consulting and Clinical Psychology, 46, 417-422.
41.
Reynolds, C. R. (1979). The invariance of the factorial validity of the Metropolitan Readiness Tests for blacks, whites, males and females. Educational and Psychological Measurement, 39, 1047-1052.
42.
Reynolds, C. R. (1980). Differential construct validity of a preschool battery for blacks, whites, males and females. Journal of School Psychology, 18, 112-125.
43.
Reynolds, C. R. (1982). Methods for detecting construct and predictive bias. In R. A. Berk (Ed.), Handbook of methods for detecting test bias (pp. 199-227). Baltimore: Johns Hopkins University Press.
44.
Reynolds, C. R., & Kaiser, S. M. (1990a). Bias in assessment of aptitude. In C. R. Reynolds & R. W. Kamphaus (Eds.), Handbook of psychological and educational assessment of children: Intelligence and achievement (pp. 611-653). New York: Guilford Press.
45.
Reynolds, C. R., & Kaiser, S. M. (1990b). Test bias in psychological assessment. In T. B. Gutkin & C. R. Reynolds (Eds.), The handbook of school psychology (2nd ed., pp. 487-525). New York: John Wiley.
46.
Reynolds, C. R., Willson, V. L., & Chatman, S. R. (1984). Item bias on the 1981 revision of the Peabody Picture Vocabulary Test using a new method of detecting bias. Journal of Psychoeducational Assessment, 2, 219-222.
47.
S-1 vs. Turlington, Trial on Merits, May 19-June 4, 1986. Order on Motion to Dismiss, No. 79-8020-Civ-Atkins, U.S. District Court, Southern District of Florida, October 9, 1986.
48.
Sandoval, J. (1979). The WISC-R and internal evidence of test bias with minority groups. Journal of Consulting and Clinical Psychology, 47, 919-927.
49.
Sattler, J. M. (1988). Assessment of children (3rd ed.). San Diego, CA: Author.
50.
Stone, B. J., & Gridley, B. E. (1991). Test bias of a kindergarten screening battery: Predicting achievement for white and Native American elementary students. School Psychology Review, 20, 132-139.
51.
Swaminathan, H., & Rogers, H. J. (1988). Detecting item bias using logistic regression. Paper presented at the annual meeting of the American Educational Research Association, New Orleans.
52.
Wechsler, D. (1989). Manual for the Wechsler Preschool and Primary Scale of Intelligence-Revised. San Antonio, TX: Psychological Corporation.
53.
Wechsler, D. (in press a). Wechsler Individual Achievement Test. San Antonio, TX: Psychological Corporation.
54.
Wechsler, D. (in press b). Wechsler Intelligence Scale for Children -third edition. San Antonio, TX: Psychological Corporation.
55.
Williams R. L. (1974). Scientific racism and IQ: The silent mugging of the Black community. Psychology Today, 32-41.
56.
Willson, V. L., Nolan, R. F., Reynolds, C. R., & Kamphaus, R. W. (1989). Race and gender effects on item functioning on the Kaufman Assessment Battery for Children. Journal of School Psychology, 27, 289-296.
57.
Zwick, R., & Ercikan, K. (1989). Analysis of differential item functions in the NAEP history assessment. Journal of Educational Measurement, 26, 55-66.
