Estimation of Compensatory and Noncompensatory Multidimensional Item Response Models Using Markov Chain Monte Carlo

Abstract

Markov chain Monte Carlo (MCMC) estimation is investigated for multidimensional compensatory and noncompensatory item response models. Simulation analyses are used to evaluate parameter recovery for the multidimensional two-parameter logistic model (M2PL) and the multidimensional latent trait model (MLTM) under varying conditions of sample size (1,000, 3,000), number of items (25, 50), and correlation between abilities (.0, .3, and .6). Results suggest that an MCMC procedure using a Metropolis-Hastings algorithm can recover the parameters of both models but is less successful for the MLTM as the correlation between abilities increases. In general, estimation is more accurate for the M2PL than the MLTM. A Bayes factor criterion for comparing the relative .t of the models to a common data set is investigated using simulated data. Using real data, the M2PL is found to be the superior model for a test of English usage.

Get full access to this article

View all access options for this article.

References

Ackerman, T. A. (1989). Unidimensional IRT calibration of compensatory and noncompensatory multidimensional items. Applied Psychological Measurement, 13, 113-127.

Ackerman, T. A. (1994). Creating a test information pro.le for a two-dimensional latent space. Applied Psychological Measurement, 18, 257-275.

Ackerman, T. A. (1996). Graphical representation of multidimensional IRTanalyses. Applied Psychological Measurement, 20, 311-329.

Albert, J. H. (1992). Bayesian estimation of normal ogive item response curves using Gibbs sampling. Journal of Educational Statistics, 17, 251-269.

Baker, F. B. (1998). An investigation of the item parameter recovery characteristics of a Gibbs sampling approach. Applied Psychological Measurement, 22, 153-169.

Béguin, A. A., & Glas, C. A. W. (2001). MCMC estimation of multidimensional IRTmodels. Psychometrika, 66, 541-561.

Bolt, D. M., Cohen, A. S., & Wollack, J. A. (2001). A mixture item response model for multiplechoice data. Journal of Educational and Behavioral Statistics, 26, 381-409.

Embretson, S. (1983). Construct validity: Construct representation versus nomothetic span. Psychological Bulletin, 93, 179-197.

Embretson, S. E.,& Reise, S. P. (2000). Item response theory for psychologists. Mahwah, NJ: Lawrence Erlbaum.

10.

Fraser, C. (1988). NOHARM II: A Fortran program fortting unidimensional and multidimensional normal ogive models of latent trait theory. Armidale, New South Wales: University of New England, Centre for Behavioral Studies.

11.

Gelfand, A. E. (1996). Model determination using sampling-based methods. In W. R. Gilks, S. Richardson, & D. J. Spiegelhalter (Eds.), Markov chain Monte Carlo in practice(pp. 145-161). Washington, DC: Chapman & Hall.

12.

Gilks, W. R., Richardson, S., & Spiegelhalter, D. J. (1996). Introducing Markov chain Monte Carlo. In W. R. Gilks, S. Richardson, & D. J. Spiegelhalter (Eds.), Markov chain Monte Carlo in practice(pp. 1-19). Washington, DC: Chapman & Hall.

13.

Hoijtink, H., & Molenaar, I. W. (1997). A multidimensional item response model: Constrained latent class analysis using the Gibbs’ sampler and posterior predictive checks. Psychometrika, 62, 171-189.

14.

Kim, S.-H. (2001). An evaluation of a Markov chain Monte Carlo method for the Rasch model. Applied Psychological Measurement, 25, 163-176.

15.

Maris, E. (1995). Psychometric latent response models. Psychometrika, 60, 69-82.

16.

Maris, E. (1999). Estimating multiple classi.cation latent class models. Psychometrika, 64, 187-212.

17.

McDonald, R. P. (1997). Multidimensional normal ogive model. In W. J. VanderLinden & R. K. Hambleton (Eds.), Handbook of modern item response theory(pp. 257-269). New York: Springer-Verlag.

18.

O’Hagan, A. (1991). Discussion on posterior Bayes factors (by M. Aitkin). Journal of the Royal Statistical Society, Series B, 53, 136-136.

19.

O’Hagan, A. (1995). Fractional Bayes factors for model comparison. Journal of the Royal Statistical Society, Series B, 57, 99-138.

20.

Patz, R. J.,& Junker, B. W. (1999a). Applications and extensions of MCMC in IRT: Multiple item types, missing data, and rated responses. Journal of Educational and Behavioral Statistics, 24, 342-366.

21.

Patz, R. J.,& Junker, B. W. (1999b). Astraightforward approach to Markov chain Monte Carlo methods for item response models. Journal of Educational and Behavioral Statistics, 24, 146-178.

22.

Reckase, M. D. (1985). The dif.culty of test items that measure more than one ability. Applied Psychological Measurement, 9, 401-412.

23.

Reckase, M. D. (1997a). A linear logistic multidimensional model. In W. J. VanderLinden & R. K. Hambleton (Eds.), Handbook of modern item response theory(pp. 271-296). New York: Springer-Verlag.

24.

Reckase, M. D. (1997b). The past and future of multidimensional item response theory. Applied Psychological Measurement, 21, 25-36.

25.

Reckase, M. D., & McKinley, R. L. (1991). The discriminating power of items that measure more than one dimension. Applied Psychological Measurement, 15, 361-373.

26.

Sahu, S. K. (2002). Bayesian estimation and model choice in item response models. Journal of Statistical Computation and Simulation, 72, 217-232.

27.

Spiegelhalter, D., Thomas, A., Best, N., & Gilks, W. (1996). BUGS 0.5* Bayesian inference using Gibbs sampling manual (version ii) [Computer program]. Cambridge, UK: Medical Research Council Biostatistics Unit, Institute of Public Health, Cambridge University.

28.

Spiegelhalter, D., Thomas, A., & Best, N. (2000). WinBUGS version 1.3 [Computer program]. Cambridge, UK: MRCBiostatistics Unit, Institute of Public Health.

29.

Spray, J. A., Davey, T. C., Reckase, M. D., Ackerman, T. A., & Carlson, J. E. (1990). Comparison of two logistic multidimensional item response models(Research Rep. ONR 90-8). Iowa City, IA: American College Testing.

30.

Sympson, J. B. (1978). A model for testing with multidimensional items. In D. J. Weiss (Ed.), Proceedings of the 1977 Computerized Adaptive Testing Conference(pp. 82-98). Minneapolis: University of Minnesota, Department of Psychology, Psychometric Methods Program.

31.

Takane, Y., & DeLeeuw, J. (1987). On the relationship between item response theory and factor analysis of discretized variables. Psychometrika, 52, 393-408.

32.

VanderLinden, W. J., & Hambleton, R. K. (1997). Handbook of modern item response theory. New York: Springer-Verlag.

33.

Way, W. D., Ansley, T. N., & Forsyth, R. A. (1988). The comparative effects of compensatory and noncompensatory two-dimensional data on unidimensional IRTestimates. Applied Psychological Measurement, 12, 239-252.

34.

Whitely, S. E. (1980). Multicomponent latent trait models for ability tests. Psychometrika, 45, 479-494.

35.

Wilson, D. T.,Wood, R.,& Gibbons, R. (1998). TESTFACT: Test scoring, item statistics, and item factor analysis. Mooresville, IN: Scienti.c Software.

36.

Wollack, J. A., Bolt, D. M., Cohen, A. S., & Lee, Y.-S. (2001). Recovery of item parameters in the nominal response model: A comparison of marginal maximum likelihood and Markov chain Monte Carlo estimation. Applied Psychological Measurement, 26, 337-350.