Multidimensional item response theory (IRT) models have been proposed for better understanding the dimensional structure of data or to define diagnostic profiles of student learning. A compensatory multidimensional two-parameter partial credit model (M-2PPC) for constructed-response items is presented that is a generalization of those proposed to date along with a compensatory multidimensional three-parameter logistic model for multiple-choice data (M-3PL). Estimation of these models using Markov chain Monte Carlo methods is discussed. To further evaluate these models and characterize item and test functioning, multidimensional representations of statistics such as information, difficulty, and discrimination for the M-3PL and M-2PPC are presented. Many assessment programs have a mixture of item types in which multiple choice and constructed response are administered together. An example is presented in which the dimensional structure of a test containing mixed item types is examined. Goodness-of-fit testing under various model formulations and derived statistics are discussed.
Ackerman, T. (1989). Unidimensional IRT calibration of compensatory and noncompensatory items. Applied Psychological Measurement, 13, 113-127.
2.
Ackerman, T. (1994). Creating a test information profile for a two-dimensional latent space. Applied Psychological Measurement, 18, 257-275.
3.
Ackerman, T. (1996). Graphical representation of multidimensional item response theory analyses. Applied Psychological Measurement, 20, 311-329.
4.
Ackerman, T., Gierl, M. J.,&Walker, C. M. (2003). Using multidimensional item response theory to evaluate educational psychological tests. Educational Measurement: Issues and Practice, 22, 37-51.
5.
Adams, R. J., Wilson, M., & Wang, W. C. (1997). The multidimensional random coefficients multinomial logit model. Applied Psychological Measurement, 21, 1-23.
6.
Akaike, H. (1987). Factor analysis and AIC. Psychometrika, 62, 317-332.
7.
De Ayala, R. J. (1989). A comparison of the nominal response model and the three-parameter logistic model in computerized adaptive testing. Educational and Psychological Measurement, 49, 789-805.
8.
De Ayala, R. J. (1992). The nominal response model in computerized adaptive testing. Educational and Psychological Measurement, 16, 327-343.
9.
Ercikan, K., Schwarz, R. D., Julian, M. W., Burket, G., & Link, V. (1998). Calibration and scoring of tests with multiple-choice and constructed-response item types. Journal of Educational Measurement, 3, 137-154.
10.
Fraser, C., & McDonald, R. P. (1988). NOHARM: Least squares item factor analysis. Multivariate Behavioral Research, 23, 267-269.
11.
Green, B. F. (1990). Notes on the item information function in the multidimensional compensatory IRT model (Report No. 88-10). Baltimore: Johns Hopkins University, Psychometric Laboratory.
12.
Kelderman, H. (1996). Multidimensional Rasch models for partial-credit scoring. Applied Psychological Measurement, 20, 155-168.
13.
Lane, S. (2005, April). Status and future directions for performance assessments in education. Paper presented at the annual meeting of the American Educational Research Association, Montreal.
14.
Muraki, E. (1999). POLYFACT Version 2 [Computer program]. Princeton, NJ: Educational Testing Service.
15.
Patz, R., & Junker, B. W. (1999a). Applications and extensions of MCMC in IRT: Multiple item types, missing data, and rated responses. Journal of Educational and Behavioral Statistics, 24, 342-346.
16.
Patz, R., & Junker, B. W. (1999b). A straightforward approach to Markov chain Monte Carlo methods for item response models. Journal of Educational and Behavioral Statistics, 24, 146-178.
17.
Patz, R., & Yao, L. (2003, April). Hierarchical and multidimensional models for vertical scaling. Paper presented at the annual meeting of the National Council on Measurement in Education, Chicago.
18.
Reckase, M. D. (1985). The difficulty of test items that measure more than one ability. Applied Psychological Measurement, 9, 401-412.
19.
Reckase, M. D., & McKinley, R. L. (1991). The discriminating power of items that measure more than one dimension. Applied Psychological Measurement, 15, 361-373.
20.
Traub, R. E. (1993). On the equivalence of traits assessed by multiple-choice and constructed-response tests. In R. E. Bennett & W. C. Ward (Eds.), Construction versus choice in cognitive measurement (pp. 29-44). Hillsdale, NJ: Lawrence Erlbaum.
21.
Walker, C. M., & Beretvas, S. N. (2003). Comparing multidimensional and unidimensional proficiency classifications: Multidimensional IRT as a diagnostic aid. Journal of Educational Measurement, 40, 255-275.
22.
Waller, N. (2002). MicroFACT: A microcomputer factor analysis program for ordered polytomous data and mainframe size problems [Computer software and manual]. St. Paul, MN: Assessment Systems Corporation.
23.
Wilson, D., Wood, R., & Gibbons, R. D. (1991). TESTFACT: Test scoring, item statistics, and item factor analysis [Computer software]. Mooresville, IN: Scientific Software.
24.
Yao, L. (2003). BMIRT: Bayesian multivariate item response theory [Computer software]. Monterey, CA: CTB/McGraw-Hill.
25.
Yao, L. (2004a, August). Bayesian multivariate item response theory and BMIRT software. Paper presented at the Joint Statistical Meetings, Toronto, Canada.
26.
Yao, L. (2004b). LinkMIRT: Linking of multivariate item response model [Computer software]. Monterey, CA: CTB/McGraw-Hill.
27.
Yao, L., & Boughton K. (2005a). A multidimensional item response theory approach for improving sub-score proficiency estimation in cognitive diagnostic assessments. Manuscript under review.
28.
Yao, L., & Boughton, K. A. (2005b). Multidimensional parameter recovery: Markov chain Monte Carlo versus NOHARM. Manuscript under review.
