An EM Approach to Parameter Estimation for the Zinnes and Griggs Paired Comparison IRT Model

Abstract

Borman et al. recently proposed a computer adaptive performance appraisal system called CARS II that utilizes paired comparison judgments of behavioral stimuli. To implement this approach,the paired comparison ideal point model developed by Zinnes and Griggs was selected. In this article,the authors describe item response and information functions for the Zinnes and Griggs model and present procedures for estimating stimulus and person parameters. Monte carlo simulations were conducted to assess the accuracy of the parameter estimation procedures. The results indicated that at least 400 ratees (i.e.,ratings) are required to obtain reasonably accurate estimates of the stimulus parameters and their standard errors. In addition,latent trait estimation improves as test length increases. The implications of these results for test construction are also discussed.

Get full access to this article

View all access options for this article.

References

Abramowitz,M. ,& Stegun,I. (1968). Handbook of mathematical functions. New York: Dover.

Andrich,D. (1995). Hyperbolic cosine latent trait models for unfolding direct responses and pairwise preferences. Applied Psychological Measurement, 19, 269-290.

Andrich,D. ,& Luo,G. (1993). A hyperbolic cosine latent trait model for unfolding dichotomous single-stimulus responses. Applied Psychological Measurement, 17, 253-276.

Birnbaum,A. (1968). Some latent trait models and their use in inferring an examinee’s ability. In F. M. Lord & M. R. Novick (Eds.), Statistical theories of mental test scores (pp. 397-472). Reading,MA: Addison-Wesley.

Bock,R. D. ,& Aitkin,M. (1981). Marginal maximum likelihood estimation of item parameters: Application of an EM algorithm. Psychometrika, 46, 443-459.

Borman,W. C. , Buck,D. , Hanson,M. A. , Motowidlo, S. J. , Stark,S. ,& Drasgow,F. (2001). An examination of the comparative reliability,validity,and accuracy of performance ratings made using computerized adaptive rating scales. Journal of Applied Psychology, 86, 965-973.

Borman,W. C. , Hanson,M. A. , Motowidlo,S. J. , Drasgow,F. , Foster,L. ,& Kubisiak,U. C. (1998, April). Computerized adaptive rating scales that measure contextual performance. Paper presented at the 13th Annual Conference of the Society for Industrial and Organizational Psychology,Dallas, TX.

Bossuyt,P. (1990). A comparison of probabilistic unfolding theories for paired comparisons data. Berlin: Springer-Verlag.

Coombs,C. H. (1950). Psychological scaling without a unit of measurement. Psychological Review, 57, 145-158.

10.

Davison,M. (1977). On a metric,unidimensional unfolding model for attitudinal and developmental data. Psychometrika, 42, 523-548.

11.

Harwell,M. , Stone,C. A. , Hsu,T. C. ,& Kirisc I. (1996). Monte carlo studies in item response theory. Applied Psychological Measurement, 20, 101-125.

12.

Murphy,K. R. ,& Cleveland,J. N. (1991). Performance appraisal: An organizational perspective. Boston: Allyn & Bacon.

13.

Press,W. H. , Flannery,B. P. , Teukolsky,S. A. ,& Vetterling,W. T. (1990). Numerical recipes: The art of scientific computing. New York: Cambridge University Press.

14.

Roberts,J. S. , Donoghue,J. R. ,& Laughlin,J. E. (1999,April). Estimating parameters in the generalized graded unfolding model: Sensitivity to the prior distribution assumption and the number of quadrature points used. Paper presented at the annual meeting of the National Council on Measurement in Education,Montreal,Canada.

15.

Seong,T. (1990). Sensitivity of marginal maximum likelihood estimation of item and ability parameters to the characteristics of the prior ability distributions. Applied Psychological Measurement, 14, 299-311.

16.

Thurstone,L. L. (1927). Psychophysical analysis. American Journal of Psychology, 38, 368-389.

17.

van Schuur,W. H. (1989). Unfolding German political parties: A description and application of multiple unidimensional unfolding. In G. De So Ferger ,& K. C. Klauer (Eds.), New developments in psychological choice modeling (pp. 259-277). Amsterdam: North Holland.

18.

Zinnes,J. L. ,& Griggs,R. A. (1974). Probabilistic, multidimensional unfolding analysis. Psychometrika, 39, 327-350.