This study is aimed to detect differential item functioning (DIF) items across gender groups, analyze item content for the possible sources of DIF, and eventually investigate the effect of DIF items on the criterion-related validity of the test scores in the quantitative section of the university entrance examination (UEE) in Turkey. The reason for DIF items in the UEE basically comes from subject matter related factors, cognitive skills measured, and item format characteristics. It seems that higher order cognitive skills and figural or graphical representations used in item content are the two sources of DIF for favoring male students, whereas routine algorithmic calculations could produce DIF against males. Among the factors considered, cognitive skills assessed by items seem the most effective factor in producing gender DIF. However, DIF items do not create a threat to the criterion-related validity of the quantitative section of the UEE. Consideration of the DIF items and gender differences as presented here serves to underscore the need to consider gender differences in item selection on any measure that is used for a similar purpose.
Allalouf, A., & Abramzon, A. ( 2008). The effect of differential item functioning on the reliability and validity of tests. Paper presented at the 6th Conference of the International Testing Commission, Liverpool, UK.
2.
Berberoğlu, G. ( 1996). The university entrance examinations in Turkey. Studies in Educational Evaluation, 22, 363-373.
3.
Berberoğlu, G., Dochy, F., & Moerkerke, G. ( 1996). Psychometric evaluation of entry assessment in higher education: A case study. European Journal of Psychology of Education, 11, 25-41.
4.
Braddy, P.W., Meade, A.W., & Johnson, E.C. ( 2006). Practical implications of using different tests of measurement invariance for polytomous measures. Paper presented at the 21st Annual Conference of the Society for Industrial and Organizational Psychology, Dallas, TX.
5.
Browne, M.W., & Cudeck, R. ( 1993). Alternative ways of assessing model fit. In K. A. Bollen & J. S. Long (Eds.), Testing structural equation models (pp. 136-162). Newbury Park, CA: SAGE.
6.
Camilli, G. ( 1993). The case against item bias detection techniques based on internal criteria: Do item bias procedures obscure test fairness issues? In P. W: Holland & H. Wainer (Eds.), Differential item functioning (pp. 397-413). Hillsdale, NJ: Lawrence Erlbaum.
7.
Camilli, G., & Shepard, L.A. ( 1994). Procedures for identifying biased test items (Vol. 4). Thousand Oaks, CA: SAGE.
8.
Dorans, N.J., & Holland, P.W. ( 1993). DIF detection and description: Mantel-Hanezsel and standardization . In P. W: Holland & H. Wainer (Eds.), Differential item functioning (pp. 35-66). Hillsdale, NJ: Lawrence Erlbaum.
9.
Ercikan, K. ( 2002). Disentangling sources of differential item functioning in multilanguage assessments. International Journal of Testing , 2, 199-215.
10.
Ercikan, K., Gierl, M.J., McCreith, T., Puhan, G., & Koh, K. ( 2004). Comparability of bilingual versions of assessments: Sources of incomparability of English and French versions of Canada’s national achievement tests. Applied Measurement in Education, 17, 301-321.
11.
Gomez-Benito, J., & Navas-Ara, M.J. (2000). A comparison of X2, RFA and IRT based procedures in the detection of DIF. Quality & Quantity, 34, 17-31.
12.
Harris, A.M., & Carlton, S.T. ( 1993). Patterns of gender differences on mathematics items on the scholastic aptitude test. Applied Measurement in Education , 6, 137-151.
13.
Hidalgo, M.D., & Lopez-Pina, J.A. (2004). Differential item functioning detection and effect size: A comparison between logistic regression and Mantel-Haenszel procedures. Educational and Psychological Measurement, 64, 903-915. doi: 10.1177/0013164403261769
14.
Holland, P.W., & Thayer, D.T. ( 1988). Differential item functioning and the Mantel-Haenszel procedure. In H. Wainer & H. I. Braun (Eds.), Test validity (pp. 129-145). Hillsdale, NJ: Lawrence Erlbaum.
15.
Jodoin, M.G., & Gierl, M.J. ( 2001). Evaluating type I error and power rates using an effect size measure with the logistic regression procedure for DIF detection. Applied Measurement in Education, 14, 329-349.
16.
Jöreskog, K.G., & Sörbom, D. (1996). LISREL 8: Structural equation modeling with the SIMPLIS command language. Chicago, IL : Scientific Software International.
17.
Kim, S., & Cohen, A.S. ( 1995). A comparison of Lord’s chi-square, Raju’s area measures, and the likelihood ratio test on detecting of differential item functioning. Applied Measurement in Education, 8, 291-312.
18.
Li, Z., & Zumbo, B.D. ( 2009). Impact of differential item functioning on subsequent statistical conclusions based on observed test score data. Psicologica, 30, 343-370.
19.
Mendes-Barnett, S., & Ercikan, K. ( 2006). Examining sources of gender DIF in mathematics assessments using a confirmatory multidimensional model approach. Applied Measurement in Education, 19, 289-304.
20.
Oort, F.J. ( 1992). Using restricted factor analysis to detect item bias . Methodika, 6, 150-166.
Teresi, J.A., & Fleishman, J.A. (2007). Differential item functioning and health assessment. Quality of Life Research, 16, 33-42. doi: 10.1007/s1136-007-9184-6
23.
Thissen, D. ( 2001). IRTLRDIF v.2.0b [Computer software]. Retrieved from http://www.unc.edu/~dthissen/dl.html
24.
University Entrance Examination Placement Statistics ( 2008). Student Selection and Placement Center, 2009. Retrieved from http://www.osym.gov.tr/
25.
Van de Vijver, F.J.R., & Tanzer, N.K. ( 1997). Bias and equivalence in cross-cultural assessment: An overview. European Review of Applied Psychology, 47, 263-279.
26.
Waller, N.G. ( 1998). EZDIF: Detection of uniform and nonuniform differential item functioning with the Mantel-Haenszel and logistic regression procedures . Applied Psychological Measurement, 22, 391.
27.
Waller, N.G., Thompson, J.S., & Wenk, E. ( 2000). Using IRT to separate measurement bias from true group differences on Homogeneous and Heterogeneous Scales: An illustration with the MMPI. Psychological Methods, 5, 125-146.
28.
Yildirim, H.H., & Berberoğlu, G. (2009). Judgmental and statistical DIF analyses of the PISA-2003 mathematics literacy items. International Journal of Testing, 9, 108-121. doi: 10.1080/15305050902880736
29.
Zenisky, A.L., Hambleton, R.K., & Robin, F. ( 2004). DIF detection and interpretation in large-scale science assessments: Informing item writing practices. Educational Assessment, 9, 61-78.
30.
Zieky, M. ( 1993). Practical questions in the use of DIF statistics in test development. In P. Holland & H. Wainer (Eds.), Differential item functioning (pp. 337-347). Hillsdale, NJ: Lawrence Erlbaum.
31.
Zumbo, B.D. ( 1999). A handbook on the theory and procedures of differential item functioning (DIF): Logistic regression modeling as a unitary framework for binary and Likert-type (ordinal) item scores. Ottawa, Ontario, Canada: Directorate of Human Resources Research and Evaluation, Department of National Defense.
