The present study presents the formulation of graded response models in the multilevel framework (as nonlinear mixed models) and demonstrates their use in estimating item parameters and investigating the group-level effects for specific covariates using Bayesian estimation. The graded response multilevel model (GRMM) combines the formulation of graded response models with the discrimination parameter fixed at one for all items by Tuerlinckx and Wang and of two parameter models by Rijmen and Briggs to offer graded response models with item-specific discrimination parameters. Apart from the contribution to the body of knowledge by formulating GRMMs, the significance of the present study includes providing a meeting point between psychometrics and statistics, overcoming the Neyman—Scott problem by using Bayesian estimation, estimation of abilities of persons with extreme scores, and demonstration of general purpose software for estimating item response theory parameters. Data from the emotional functioning scale on 11,158 healthy and chronically ill children and adolescents were used from the PedsQL 4.0 Generic Core Scales database to illustrate the model. Estimates for the item parameters from WINBUGS using Bayesian priors and Multilog were compared for the GRMM and the ordinary graded response models, respectively.
Adams, R.J., Wilson, M., & Wu, M. ( 1997). Multilevel item response models: An approach to errors in variables regression. Journal of Educational and Behavioral Statistics, 22, 47-76.
2.
Baker, F.B. ( 1985). The basics of item response theory. Portsmouth, NH: Heinemann.
Beretvas, S.N., Meyers, J.L., & Rodriguez, R.A. (2005). The cross-classified multilevel measurement model: An explanation and demonstration. Journal of Applied Measurement, 6, 322-341.
5.
Beretvas, S.N., & Williams, N.J. (2004). The use of hierarchical generalized linear model for item dimensionality assessment. Journal of Educational Measurement, 41, 379-395.
6.
Bock, R.D., & Aitkin, M. ( 1981). Marginal maximum likelihood estimation of item parameters: Application of an EM algorithm. Psychometrika, 46, 443-459.
7.
Breslow, N.E., & Clayton, D.G. ( 1993). Approximate inference in generalized linear mixed models . Journal of the American Statistical Association, 88, 9-25.
8.
Burton, R. ( 2004). Can item response theory help us improve our tests? Medical Education, 38, 338-339.
9.
Burton, R. ( 2005). Multiple-choice and true/false tests: Myths and misapprehensions . Assessment & Evaluation in Higher Education, 30, 65-72.
10.
De Boeck, P., & Wilson, M. (Eds.). (2004). Explanatory item response models: A generalized linear and nonlinear approach. New York: Springer.
11.
Fahrmeir, L., & Tutz, G. ( 2001). Multivariate statistical modelling based on generalized linear models. New York: Springer .
12.
Fox, J.-P. ( 2004). Applications of multilevel IRT modeling. School Effectiveness and School Improvement, 15, 261-280.
13.
Fox, J.-P. ( 2005). Multilevel IRT using dichotomous and polytomous response data. British Journal of Mathematical and Statistical Psychology , 58, 145-172.
14.
Fox, J.-P., & Glas, C.A.W. ( 2001). Bayesian estimation of a multilevel IRT model using Gibbs sampling. Psychometrika, 66, 269-286.
15.
Fox, J.-P., & Glas, C.A.W. ( 2003). Bayesian modeling of measurement error in predictor variables using item response theory. Psychometrika, 68, 169-191.
16.
Gelman, A., Carlin, J.B., Stern, H.S., & Rubin, D.B. ( 1995). Bayesian data analysis. London: Chapman & Hall.
17.
Hill, C., Edwards, M., Thissen, D., Langer, M., Wirth, R., Burwinkle, T., et al. (2007). Practical issues in the application of item response theory: A demonstration using items from the Pediatric Quality of Life Inventory (PedsQL) 4.0 Generic Core Scales. Medical Care, 45, S39S47.
18.
Kamata, A. ( 1998). Some generalizations of the Rasch model: An application of the hierarchical generalized linear model. Unpublished doctoral dissertation , Michigan State University, East Lansing.
19.
Kamata, A. ( 2001). Item analysis by the hierarchical generalized linear model. Journal of Educational Measurement, 38, 79-93.
20.
Kim, S., & Cohen, A. ( 1992). IRTDIF: A computer program for IRT differential functioning analysis. Applied Psychological Measurement, 16, 158.
21.
Langer, M.M., Hill, C.D., Thissen, D., Burwinkle, T.M., Varni, J.W., & DeWalt, D.A. (2008). Item response theory detects differential item functioning between healthy and ill children in pediatric QoL measures. Journal of Clinical Epidemiology, 61, 268-276.
22.
Lord, F.M. ( 1968). Some test theory for tailored testing. Princeton, NJ: Educational Testing Service.
23.
Lord, F.M., & Novick, M.R. ( 1968). Statistical theories of mental test scores. Reading, MA: Addison-Wesley.
24.
Lunn, D.J., Thomas, A., Best, N., & Spiegelhalter, D. (2000) WinBUGS-a Bayesian modelling framework: Concepts, structure, and extensibility. Statistics and Computing, 10, 325-337.
25.
McKinley, R., & Mills, C. ( 1989). Item response theory: Advances in achievement and attitude measurement. In B. Thompson (Ed.), Advances in social science methodology (Vol. 1, pp. 71-135). Greenwich, CT: JAI Press.
26.
McKinley, R.L., & Reckase, M.D. ( 1983). An application of a multidimensional extension of the two-parameter logistic latent trait model (Research Rep. ONR83-3). Iowa City, IA: The American College Testing Program .
27.
Mislevy, R.J. ( 1983). Item response models for grouped data. Journal of Educational Statistics, 8, 271-288.
28.
Neyman, J., & Scott, E.L. ( 1948). Consistent estimates based on partially consistent observations . Econometrika, 16, 1-5.
29.
Ostini, R., & Nering, M.L. ( 2006). Polytomous item response theory models. Thousand Oaks, CA: Sage.
30.
Rasch, G. ( 1960). Probabilistic models for some intelligence and attainment tests. Copenhagen, Denmark: Danish Institute for Educational Research. (Expanded Edition, 1980. Chicago: University of Chicago Press)
31.
Reise, S.P. ( 2000). Using multilevel logistic regression to evaluate person-fit in IRT models. Multivariate Behavioral Research, 35, 543-568.
32.
Reise, S.P., & Yu, J. ( 1990). Parameter recovery in the graded response model using MULTILOG. Journal of Educational Measurement, 27, 133-144.
33.
Rijmen, F., & Briggs, D. ( 2004). Multidimensional person variance and latent item predictors . In P. De Boeck & M. Wilson (Eds.), Explanatory item response models: A generalized linear and nonlinear approach (pp. 247-265). New York: Springer.
34.
Rijmen, F., Tuerlinckx, F., De Boeck, P., & Kuppens, P. ( 2003). A nonlinear mixed model framework for item response theory . Psychological Methods, 8, 185-205.
35.
Samejima, F. ( 1969). Estimation of latent ability using a response pattern of graded scores. Psychometric Monograph, No. 17, 34, Part 2.
36.
Samejima, F. ( 1979). A new family for multiple choice items ( Research Rep. No. 79-4). Knoxville: University of Tennessee.
37.
Samejima, F. ( 1997). Graded response model. In W. J. van der Linden & R. Hambleton (Eds.), Handbook of modern item response theory (pp. 85-100). New York: Springer.
38.
Searle, S.R., & McCulloch, C.E. (2001). Generalized linear and mixed models. New York: Wiley-Interscience .
39.
Thissen, D. ( 1991). MULTILOG user’s guide: Multiple, categorical item analysis and test scoring using item response theory (Version 6.0). Chicago: Scientific Software.
40.
Tuerlinckx, F., & Wang, W.C. ( 2004). Models for polytomous data. In P. De Boeck & M. Wilson (Eds.), Explanatory item response models: A generalized linear and nonlinear approach (pp. 75-109). New York: Springer.
41.
Turhan, A. ( 2006). Multilevel 2PL item response model vertical equating with the presence of differential item functioning. Unpublished doctoral dissertation , Florida State University, Tallahassee .
42.
Van den Noortgate, W., & Paek, I. ( 2004). Person regression models In P. De Boeck & M. Wilson (Eds.), Explanatory item response models: A generalized linear and nonlinear approach (pp. 167-187). New York: Springer.
43.
Varni, J.W., Burwinkle, T.M., Seid, M., & Skarr, D. ( 2003). The PedsQL™ 4.0 as a pediatric population health measure: Feasibility, reliability, and validity. Ambulatory Pediatrics, 3, 329-341.
44.
Varni, J.W., Limbers, C.A., & Burwinkle, T.M. (2007). How young can children reliably and validly self-report their health-related quality of life? An analysis of 8,591 children across age subgroups with the PedsQL™ 4.0 Generic Core Scales. Health and Quality of Life Outcomes, 5, 1-13.
45.
Varni, J.W., Seid, M., & Kurtin, P.S. ( 2001). PedsQL™ 4.0: Reliability and validity of the Pediatric Quality of Life Inventory Version 4.0 Generic Core Scales in healthy and patient populationsMedical Care, 39, 800-812.
46.
Wallander, J.L., & Varni, J.W. ( 1998). Effects of pediatric chronic physical disorders on child and family adjustment. Journal of Child Psychology and Psychiatry , 39, 29-46.
47.
Williams, N.J. ( 2003). Item and person parameter estimation using hierarchical generalized linear models and polytomous item response theory models. Unpublished doctoral dissertation, University of Texas.
48.
Wilson, M., & De Boeck, P. ( 2004). Descriptive and explanatory item response models. In P. De Boeck & M. Wilson (Eds.), Explanatory item response models: A generalized linear and nonlinear approach (pp. 43-74). New York: Springer.
