Fitting a Mixture Item Response Theory Model to Personality Questionnaire Data: Characterizing Latent Classes and Investigating Possibilities for Improving Prediction
Restricted accessResearch articleFirst published online November, 2008
Fitting a Mixture Item Response Theory Model to Personality Questionnaire Data: Characterizing Latent Classes and Investigating Possibilities for Improving Prediction
Mixture item response theory (IRT) models aid the interpretation of response behavior on personality tests and may provide possibilities for improving prediction. Heterogeneity in the population is modeled by identifying homogeneous subgroups that conform to different measurement models. In this study, mixture IRT models were applied to the Extroversion and Neuroticism scales of the Amsterdam Biographical Questionnaire, and a three-class mixture version of the nominal response model was identified as the best fitting model. The latent classes differed with respect to social desirability and ethnic background. Within latent classes, response tendencies demonstrated a differential use of the ``?'' category. An important issue is whether applying mixture IRT models results in a better prediction of relevant external criteria compared to a one-class model. For the Neuroticism scale the prediction improved, but not for the Extraversion scale. The results demonstrate the possible advantage of applying mixture IRT models to personality questionnaires.
Barrick, M., & Mount, M. (1996). Effects of impression management and self-deception on the predictive validity of personality constructs. Journal of Applied Psychology, 81, 261-272.
2.
Bass, B. (1957). Faking by sales applicants of a forced choice personality inventory. Journal of Applied Psychology, 41, 403-404.
3.
Bem, D., & Allen, A. (1974). On predicting some of the people some of the time: The search for cross-situational consistencies in behavior. Psychological Review, 81, 506-520.
4.
Bock, R.D. (1972). Estimating item parameters and latent ability when responses are scored in two or more nominal categories. Psychometrika, 37, 29-51.
5.
Bolt, D.M., Cohen, A.S., & Wollack, J.A. (2002). Item parameter estimation under conditions of test speededness: Application of a mixture Rasch model with ordinal constraints . Journal of Educational Measurement, 39, 331-348.
6.
Cohen, A.S., & Bolt, D.M. (2005). A mixture model analysis of differential item functioning. Journal of Educational Measurement, 42, 133-148.
7.
Cohen, A.S., Gregg, N., & Deng, M. (2005). The role of extended time and item content on a high-stakes mathematics test. Learning Disabilities Research and Practice, 20, 225-233.
8.
Cox, N.R. (1974). Estimation of the correlation between a continuous and a discrete variable. Biometrics, 30, 171-178.
Dempster, A.P., Laird, N.M., & Rubin, D.B. (1977). Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society, Series B, 39, 1-38.
11.
DuBois, B., & Burns, J.A. (1975). An analysis of the meaning of the question mark response category in attitude scales. Educational and Psychological Measurement, 35, 869-884.
12.
Dunnett, S., Koun, S., & Barber, P.J. (1981). Social desirability in the Eysenck Personality Inventory. British Journal of Psychology, 72, 19-26.
13.
Dunnette, M., MacCartney, K., Carlson, H., & Kirchner, W. (1962). A study of faking behavior on a forced-choice self-description checklist. Personnel Psychology, 15, 13-24.
14.
Eid, M., & Rauber, M. (2000). Detecting measurement invariance in organizational surveys. European Journal of Psychological Assessment, 16, 20-30.
15.
Eysenck, H.J. (1959). Manual of the Maudsley Personality Inventory. London: University of London Press.
16.
Eysenck, H.J., & Eysenck, S.B.G. (1975). Manual of the Eysenck Personality Questionnaire. London: Hodder & Stoughton.
17.
Ghiselli, E. (1956). Differentiation of individuals in terms of their predictability. Journal of Applied Psychology, 40, 374-377.
18.
Ghiselli, E. (1963). Moderating effects and differential reliability and validity. Journal of Applied Psychology, 47, 81-86.
19.
Goodman, L. (1975). A new model for scaling response patterns: An application of the quasi-independence concept. Journal of the American Statistical Association, 70, 755-768.
20.
Heinen, T. (1996). Latent class and discrete latent trait models: Similarities and differences. Thousand Oaks, CA: Sage.
21.
Hernández, A., Drasgow, F., & González-Romá, V. (2004). Investigating the functioning of a middle category by means of a mixed-measurement model. Journal of Applied Psychology, 89, 687-699.
22.
Kelderman, H. (2006). Loglinear multivariate and mixture Rasch models . In M. Von Davier & C. H. Carstensen (Eds.), Multivariate and mixture distribution Rasch models: Extensions and applications (pp. 79-99). New York: Springer.
23.
Kelderman, H., & Macready, G.B. (1990). The use of loglinear models for assessing differential item functioning across manifest and latent examinee groups. Journal of Educational Measurement, 27, 307-327.
24.
Kolen, M., & Brennan, R. (1995). Test equating: Methods and practices. New York: Springer.
25.
Kolk, N.J., Born, M.P., & van der Flier, H. (2004). Three method factors explaining the low correlations between assessment center dimension ratings and scores on personality inventories. European Journal of Personality, 18, 127-141.
26.
Maij-de Meij, A.M., Kelderman, H., & van der Flier, H. (2005). Latent-trait latent-class analysis of self-disclosure in the work environment. Multivariate Behavioral Research, 40, 435-460.
27.
Masters, G.N. (1982). A Rasch model for partial credit scoring. Psychometrika, 47, 149-174.
28.
McLachlan, G.J., & Peel, D. (2000). Finite mixture models. New York: John Wiley.
29.
Mislevy, R.J., & Verhelst, N. (1990). Modeling item responses when different subjects employ different solution strategies. Psychometrika, 55, 195-215.
30.
Paulhus, D. (1984). Two-component models of socially desirable responding . Journal of Personality and Social Psychology, 46, 598-609.
31.
Paulhus, D. (1986). Self-deception and impression management in test responses. In A. Angleiter & J. Wiggings (Eds.), Personality assessment via questionnaires (pp. 143-165). Berlin, Germany: Springer-Verlag.
32.
Rost, J. (1990). Rasch models in latent classes: An integration of two approaches to item analysis. Applied Psychological Measurement, 14, 271-282.
33.
Rost, J. (1991). A logistic mixture distribution model for polychotomous item responses. British Journal of Mathematical and Statistical Psychology, 44, 75-92.
34.
Rost, J., Carstensen, C., & Von Davier, M. (1997). Applying the mixed Rasch model to personality questionnaires. In J. Rost & R. Langeheine (Eds.), Applications of latent trait and latent class models in the social sciences (pp. 324-332). Munster, Germany: Waxmann.
35.
Schmitt, A.P., Holland, P., & Dorans, N.J. (1993). Evaluating hypotheses about differential item functioning. In P. W. Holland & H. Wainer (Eds.), Differential item functioning (pp. 281-315). Hillsdale, NJ: Lawrence Erlbaum.
36.
Schwarz, G. (1978). Estimating the dimension of a model. Annals of Statistics, 6, 461-464.
37.
Smit, A., Kelderman, H., & van der Flier, H. (1999). Collateral information and mixed Rasch models. Methods of Psychological Research Online, 4, 19-32.
38.
Smit, A., Kelderman, H., & van der Flier, H. (2000). The mixed Birnbaum model: Estimation using collateral information. Methods of Psychological Research Online, 5, 31-43.
39.
Smit, A., Kelderman, H., & van der Flier, H. (2003). Latent trait latent class analysis of an Eysenck Personality Questionnaire. Methods of Psychological Research Online, 8, 23-50.
40.
Te Nijenhuis, J., & van der Flier, H. (1999). Bias research in the Netherlands: Review and implications. European Journal of Psychological Assessment, 15, 162-175.
41.
Te Nijenhuis, J., van der Flier, H., & Van Leeuwen, L. (1997). Comparability of personality test scores for immigrants and majority group members: Some Dutch findings . Personality and Individual Differences, 23, 849-859.
42.
Thissen, D., Chen, W., & Bock, R. (2003). Multilog 7.0 [Computer software]. Chicago: Scientific Software.
43.
Thissen, D., Steinberg, L., & Wainer, H. (1993). Detection of differential item functioning using the parameters of item response models. In P. W. Holland & H. Wainer (Eds.), Differential item functioning (pp. 67-114). Hillsdale, NJ: Lawrence Erlbaum.
44.
Van de Vijver, F., & Phalet, K. (2004). Assessment in multicultural groups: The role of acculturation. Applied Psychology: An International Review , 52, 215-236.
45.
Vermunt, J.K. (1997). LEM: A general program for the analysis of categorical data [Computer software]. Tilburg, Netherlands: Tilburg University. Available from http://www.uvt.nl/faculteiten/fsw/organisatie/departementen/mto/software2.html
46.
Von Davier, M., & Yamamoto, K. (2004). Partially observed mixtures of IRT models: An extension of the generalized partial-credit model. Applied Psychological Measurement, 28, 389-406.
47.
Wilde, G. (1970). Neurotische labiliteit gemeten volgens de vragenlijstmethode [Neurotic lability measured by the questionnaire method]. Amsterdam: Van Rossen.
48.
Zickar, M.J., Gibby, R.E., & Robie, C. (2004). Uncovering faking samples in applicant, incumbent, and experimental data sets: An application of mixed-model item response theory . Organizational Research Methods, 7, 168-190.
