Bayesian Adaptive Lasso for the Detection of Differential Item Functioning in Graded Response Models

Abstract

The detection of differential item functioning (DIF) is important in psychological and behavioral sciences. Standard DIF detection methods perform an item-by-item test iteratively, often assuming that all items except the one under investigation are DIF-free. This article proposes a Bayesian adaptive Lasso method to detect DIF in graded response models (GRMs), where the DIF effects for all items can be identified simultaneously. The multiple-group GRMs are specified, and the possible DIF effects for each item are reparameterized using the increment components. Then, a Bayesian adaptive Lasso procedure is developed for parameter estimation, in which DIF effects can be automatically obtained. Our method is evaluated and compared with the commonly used likelihood ratio test method in a simulation study. The results show that our method can recover most model parameters well and has better control of false positive rates in almost all conditions. An application is presented using data from the National Longitudinal Study of Adolescent to Adult Health (Add Health).

Keywords

Bayesian adaptive Lasso differential item functioning graded response model regularization

Get full access to this article

View all access options for this article.

References

Alhamzawi

Benoit

D. F.

(2012). Bayesian adaptive Lasso quantile regression. Statistical Modelling, 12(3), 279–297.

Bauer

D. J.

(2017). A more general model for testing measurement invariance and differential item functioning. Psychological Methods, 22(3), 507–526.

Belzak

W. C. M.

Bauer

D. J.

(2020). Improving the assessment of measurement invariance: Using regularization to select anchor items and identify differential item functioning. Psychological Methods, 25(6), 673–690.

Benjamini

Hochberg

(1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. Journal of the Royal Statistical Society: Series B (Methodological), 57(1), 289–300.

Brandt

Chen

S. M.

Bauer

D. J.

(2023). Bayesian penalty methods for evaluating measurement invariance in moderated nonlinear factor analysis. Psychological Methods. https://doi.org/10.1037/met0000552

Candell

G. L.

Drasgow

(1988). An iterative procedure for linking metrics and assessing item bias in item response theory. Applied Psychological Measurement, 12(3), 253–260.

Carpenter

Gelman

Hoffman

M. D.

Lee

Goodrich

Betancourt

Brubaker

Guo

Riddell

(2017). Stan: A probabilistic programming language. Journal of Statistical Software, 76(1), 1–32.

Chalmers

R. P.

(2012). mirt: A multidimensional item response theory package for the R environment. Journal of Statistical Software, 48(6), 1–29.

Chang

H. H.

Mazzeo

Roussos

(1996). Detecting DIF for polytomously scored items: An adaptation of the SIBTEST procedure. Journal of Educational Measurement, 33(3), 333–353.

10.

Chen

(2022). A generalized partially confirmatory factor analysis framework with mixed Bayesian Lasso methods. Multivariate Behavioral Research, 57(6), 879–894.

11.

Chen

Guo

Zhang

Pan

(2021). A partially confirmatory approach to scale development with the Bayesian Lasso. Psychological Methods, 26(2), 210–235.

12.

Chen

S. M.

Bauer

D. J.

Belzak

W. M.

Brandt

(2022). Advantages of spike and slab priors for detecting differential item functioning relative to other Bayesian regularizing priors and frequentist lasso. Structural Equation Modeling: A Multidisciplinary Journal, 29(1), 122–139.

13.

Choi

Gibbons

Crane

(2011). Lordif: An R package for detecting differential item functioning using iterative hybrid ordinal logistic regression/item response theory and Monte Carlo simulations. Journal of Statistical Software, 39(1), 1–30.

14.

Cohen

A. S.

Kim

S. H.

Baker

F. B.

(1993). Detection of differential item functioning in the graded response model. Applied Psychological Measurement, 17(4), 335–350.

15.

Edelen

M. O.

Stucky

B. D.

Chandra

(2015). Quantifying “problematic” DIF within an IRT framework: Application to a cancer stigma index. Quality of Life Research, 24(1), 95–103.

16.

Fan

(2001). Variable selection via nonconcave penalized likelihood and its oracle properties. Journal of the American Statistical Association, 96(456), 1348–1360.

17.

Feng

Song

(2017). Bayesian adaptive Lasso for ordinal regression with latent variables. Sociological Methods and Research, 46(4), 926–953.

18.

Hastie

Tibshirani

Friedman

(2009). The elements of statistical learning: Data mining, inference, and prediction (2nd ed.). Springer.

19.

Holland

P. W.

Thayer

D. T.

(1988). Differential item performance and the Mantel-Haenszel procedure. In Wainer

Braun

H. I.

(Eds.), Test validity (pp. 129–145). Lawrence Erlbaum Associates.

20.

Huang

P. H.

(2018). A penalized likelihood method for multi-group structural equation modelling. British Journal of Mathematical and Statistical Psychology, 71(3), 499–522.

21.

Ishwaran

Rao

J. S.

(2005). Spike and slab variable selection: Frequentist and Bayesian strategies. The Annals of Statistics, 33(2), 730–773.

22.

Jeon

Rijmen

Rabe-Hesketh

(2013). Modeling differential item functioning using a generalization of the multiple-group bifactor model. Journal of Educational and Behavioral Statistics, 38(1), 32–60.

23.

Kim

E. S.

Yoon

(2011). Testing measurement invariance: A comparison of multiple-group categorical CFA and IRT. Structural Equation Modeling: A Multidisciplinary Journal, 18(2), 212–228.

24.

Kim

Oshima

T. C.

(2013). Effect of multiple testing adjustment in differential item functioning detection. Educational and Psychological Measurement, 73(3), 458–470.

25.

Kim

S. K.

Cohen

A. S.

(1998). Detection of differential item functioning under the graded response model with the likelihood ratio test. Applied Psychological Measurement, 22(4), 345–355.

26.

Knight

W. J.

(2000). Asymptotics for Lasso-type estimators. The Annals of Statistics, 28(5), 1356–1378.

27.

Leng

Tran

M. N.

Nott

(2014). Bayesian adaptive Lasso. Annals of the Institute of Statistical Mathematics, 66(2), 221–244.

28.

Lord

F. M.

(1952). A theory of test scores (Psychometric Monograph No. 7). Psychometric Corporation.

29.

Magis

Facon

(2013). Item purification does not always improve DIF detection: A counterexample with Angoff’s delta plot. Educational and Psychological Measurement, 73(2), 293–311.

30.

Magis

Tuerlinckx

De Boeck

(2015). Detection of differential item functioning using the Lasso approach. Journal of Educational and Behavioral Statistics, 40(2), 111–135.

31.

Mantel

Haenszel

(1959). Statistical aspects of the analysis of data from retrospective studies of disease. Journal of the National Cancer Institute, 22(4), 719–748.

32.

Millsap

R. E.

Everson

H. T.

(1993). Methodology review: Statistical approaches for assessing measurement bias. Applied Psychological Measurement, 17(4), 297–334.

33.

Pan

E. H.

Dubé

(2017). An alternative to post hoc model modification in confirmatory factor analysis: The Bayesian Lasso. Psychological Methods, 22(4), 687–704.

34.

Park

Casella

(2008). The Bayesian Lasso. Journal of the American Statistical Association, 103(482), 681–686.

35.

Plummer

(2017). Jags version 4.3.0 user manual. https://sourceforge.net/projects/mcmc-jags/files/Manuals/4.x/

36.

R Core Team. (2022). R: A language and environment for statistical computing. R Foundation for statistical computing, Vienna, Austria. https://www.R-project.org

37.

Rosseel

(2012). lavaan: An R package for structural equation modeling. Journal of Statistical Software, 48(2), 1–36.

38.

Samejima

(1969). Estimation of latent ability using a response pattern of graded scores. Psychometrika, 34(1), 1–97.

39.

Samejima

(1973). Homogeneous case of the continuous response model. Psychometrika, 38(2), 203–219.

40.

Samejima

(1997). Graded response model. In van der Linden

W. J.

Hambleton

R. K.

(Eds.), Handbook of modern item response theory (pp. 85–100). Springer.

41.

Schauberger

Mair

(2020). A regularization approach for the detection of differential item functioning in generalized partial credit models. Behavior Research Methods, 52(1), 279–294.

42.

Serang

Jacobucci

Brimhall

K. C.

Grimm

K. J.

(2017). Exploratory mediation analysis via regularization. Structural Equation Modeling: A Multidisciplinary Journal, 24(5), 733–744.

43.

Shi

Song

Liao

Terry

Snyder

L. A.

(2017). Bayesian SEM for specification search problems in testing factorial invariance. Multivariate Behavioral Research, 52(4), 430–444.

44.

Spray

Miller

(1994). Identifying nonuniform DIF in polytomously scored test items (American College Testing Research Report Series 94 -1). American College Testing Program.

45.

Stan Development Team. (2023). RStan: The R interface to Stan [R package version 2.21.8]. http://mc-stan.org/

46.

Stark

Chernyshenko

O. S.

Drasgow

(2006). Detecting differential item functioning with confirmatory factor analysis and item response theory: Toward a unified strategy. Journal of Applied Psychology, 91(6), 1292–1306.

47.

Steinberg

Thissen

(2006). Using effect sizes for research reporting: Examples using item response theory to analyze differential item functioning. Psychological Methods, 11(4), 402–415.

48.

Svetina

Rutkowski

(2020). Multiple-group invariance with categorical outcomes using updated guidelines: An illustration using Mplus and the lavaan/semTools packages. Structural Equation Modeling: A Multidisciplinary Journal, 27(1), 111–130.

49.

Swaminathan

Rogers

H. J.

(1990). Detecting differential item functioning using logistic regression procedures. Journal of Educational Measurement, 27(4), 361–370.

50.

Teresi

J. A.

Ramirez

Lai

Silver

(2008). Occurrences and sources of differential item functioning (DIF) in patient-reported outcome measures: Description of DIF methods, and review of measures of depression, quality of life and general health. Psychology Science, 50(4), 538–612.

51.

Thissen

Steinberg

Wainer

(1993). Detection of differential item functioning using the parameters of item response models. In Holland

P. W.

Wainer

(Eds.), Differential item functioning (pp. 67–113). Lawrence Erlbaum Associates, Inc.

52.

Tibshirani

(1996). Regression shrinkage and selection via the Lasso. Journal of the Royal Statistical Society: Series B (Methodological), 58(1), 267–288.

53.

Tibshirani

Friedman

Hastie

Narasimhan

Simon

Qian

(2021). glmnet: Lasso and elastic-net regularized generalized linear models. https://www.rdocumentation.org/packages/glmnet/versions/4.1-3

54.

Tutz

Schauberger

(2015). A penalty approach to differential item functioning in Rasch models. Psychometrika, 80(1), 21–43.

55.

Wang

Zhu

R. Y.

G. J.

(2023). Using Lasso and adaptive Lasso to identify DIF in multidimensional 2PL models. Multivariate Behavioral Research, 58(2), 387–407.

56.

Wang

W. C.

(2004). Effects of anchor item methods on the detection of differential item functioning within the family of Rasch models. The Journal of Experimental Education, 72(3), 221–261.

57.

Wang

W. C.

Y. H.

(2004). Effects of average signed area between two item characteristic curves and test purification procedures on the DIF detection via the Mantel-Haenszel method. Applied Measurement in Education, 17(2), 113–144.

58.

Woods

C. M.

(2009). Empirical selection of anchors for tests of differential item functioning. Applied Psychological Measurement, 33(1), 42–57.

59.

Zhang

C. H.

(2010). Nearly unbiased variable selection under minimax concave penalty. The Annals of Statistics, 38(2), 894–942.

60.

Zou

(2006). The adaptive Lasso and its oracle properties. Journal of the American Statistical Association, 101(476), 1418–1429.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.03 MB