This paper reviews developments in nonparametric item response theory (NIRT). Historically, Mokken's (1971) item response theory (IRT) model and procedure of scale analysis have been the starting point for NIRT models; the developments that Mokken's work have inspired and those that originated independently are discussed here. Attention is focused on results pertaining to dichotomously scored items. The usefulness of new theoretical results for practical test construction is explained. NIRT is especially useful when the purpose of test application is to order persons, items, or both. The assumptions inherent in several NIRT models, model-data fit methods, item selection procedures, and reliability methods are discussed. NIRT models often allow more items into a test than stronger models, yielding higher reliability and better coverage of the latent trait. Moreover, the theoretical results from NIRT are often relevant to parametric IRT. Examples include conditional association and manifest monotonicity, as well as recent results for polytomous NIRT models that concem the ordering of persons, which also hold for several parametric IRT models.
Get full access to this article
View all access options for this article.
References
1.
Andrich, D. (1978). A rating scale formulation for ordered response categories. Psychometrika, 43, 561-573.
2.
Cliff, N. (1977). A theory of consistency of ordering generalizable to tailored testing. Psychometrika, 42, 375-401.
3.
Cliff, N. (1979). Test theory without true scores?Psychometrika, 44, 373-393.
4.
Cliff, N. , & Donoghue, J. R. (1992). Ordinal test fidelity estimated by an item sampling model. Psychometrika, 57, 217-236.
5.
Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16, 297-334.
6.
Croon, M. A. (1991). Investigating Mokken scalability of dichotomous items by means of ordinal latent class analysis. British Journal of Mathematical and Statistical Psychology, 44, 315-331.
7.
De Gruijter, D. N. M. (1994). Comparison of the nonparametric Mokken model and parametric IRT models using latent class analysis. Applied Psychological Measurement, 18, 27-34.
8.
De Jong, A. (1984). Over psychiatrische invaliditeit (On psychiatric disability). Unpublished doctoral dissertation, Rijksuniversiteit, Groningen.
9.
De Jong, A. , & Molenaar, I. W. (1987). An application of Mokken's model for stochastic, cumulative scaling in psychiatric research. Journal of Psychiatric Research, 21, 137-149.
10.
Douglas, J. (1997). Joint consistency of nonparametric item characteristic curves and ability estimation. Psychometrika, 62, 7-28.
11.
Ellis, J. L. (1993). Subpopulation invariance of patterns in covariance matrices. British Journal of Mathematical and Statistical Psychology, 46, 231-254.
12.
Ellis, J. L. (1994). Foundations of monotone latent variable models. Nijmegen, The Netherlands: NICI, Catholic University of Nijmegen.
13.
Ellis, J. L. , & Junker, B. W. (in press). Tailmeasurability in monotone latent variable models. Psychometrika.
14.
Ellis, J. L. , & Van den Wollenberg, A. L. (1993). Local homogeneity in latent trait models. A characterization of the homogeneous monotone IRT model. Psychometrika, 58, 417-429.
15.
Embretson, S. E. (1991). A multidimensional latent trait model for measuring learning and change. Psychometrika, 56, 495-515.
16.
Esary, J. D. , Proschan, F., & Walkup, D. W. (1967). Association of random variables, with applications. The Annals of Mathematical Statistics, 38, 1466-1474.
17.
Fischer, G. H. (1974). Einfiihrung in die Theorie psychologischer Tests [Introduction to psychological test theory]. Bern: Huber.
18.
Fischer, G. H. (1995). Derivations of the Rasch model. In G. H. Fischer & I. W. Molenaar (Eds.), Rasch models: Foundations, recent developments, and applications (pp. 15-38). New York: Springer.
19.
Fischer, G. H. , & Molenaar, I. W. (1995). Rasch models: Foundations, recent developments, and applications. New York: Springer.
20.
Gillespie, M. W. , TenVergert, E. M., & Kingma, J. (1988). Secular trends in abortion attitudes: 1975-1980-1985. The Journal of Psychology, 122, 323-341.
21.
Goa, F., & Stout, W. F. (1996). Using resampling to eliminate DIMTEST estimation bias. Manuscript submitted for publication.
22.
Grayson, D. A. (1988). Two-group classification in latent trait theory: Scores with monotone likelihood ratio. Psychometrika, 53, 383-392.
23.
Guttman, L. (1944). A basis for scaling qualitative data. American Sociological Review, 9, 139-150.
24.
Guttman, L. (1945). A basis for analyzing test-retest reliability. Psychometrika, 10, 255-282.
25.
Guttman, L. (1950). The basis for scalogram analysis. In S. A. Stouffer, L. Guttman, E. A. Suchman, P. F. Lazarsfeld, S. A. Star, & J. A. Claussen (Eds.), Measurement and prediction (pp. 60-90). Princeton NJ: Princeton University Press.
26.
Hambleton, R. K. , & Swaminathan, H. (1985). Item response theory. Principles and applications. Boston: Kluwer Nijhoff.
27.
Hattie, J. , Krakowski, K., Rogers, H. J., & Swaminathan, H. (1996). An assessment of Stout's index of essential unidimensionality. Applied Psychological Measurement, 20, 1-14.
28.
Hemker, B. T. (1996). Unidimensional IRT models for polytomous items, with results for Mokken scale analysis. Unpublished doctoral dissertation, Utrecht University, The Netherlands.
29.
Hemker, B. T. , Sijtsma, K., & Molenaar, I. W. (1995). Selection of unidimensional scales from a multidimensional item bank in the polytomous Mokken IRT model. Applied Psychological Measurement, 19, 337-352.
30.
Hemker, B. T. , Sijtsma, K., Molenaar, I. W., & Junker, B. W. (1996). Polytomous IRT models and monotone likelihood ratio of the total score. Psychometrika, 61, 679-693.
31.
Hemker, B. T. , Sijtsma, K., Molenaar, I. W., & Junker, B. W. (1997). Stochastic ordering using the latent trait and the sum score in polytomous IRT models. Psychometrika, 62, 331-347.
32.
Henning, H. J. (1976). Die Technik der Mokken-Skalenanalyse [The technique of Mokken scale analysis]. Psychologische Beitrdge, 18, 410-430.
33.
Holland, P. W. (1981). When are item response models consistent with observed data?Psychometrika, 46, 79-92.
34.
Holland, P. W. (1990). On the sampling theory foundations of item response theory models. Psychometrika, 55, 577-601.
35.
Holland, P. W. , & Rosenbaum, P. R. (1986). Conditional association and unidimensionality in monotone latent variable models. The Annals of Statistics, 14, 1523-1543.
36.
Hulin, C. L. , Drasgow, F., & Parsons, C. K. (1983). Item response theory: Application to psychological measurement. Homewood IL: Dow Jones-Irwin.
37.
Huynh, H. (1994). A new proof for monotone likelihood ratio for the sum of independent Bernoulli random variables. Psychometrika, 59, 77-79.
38.
Jansen, P. G. W. (1982a). De onbruikbaarheid van Mokkenschaalanalyse [On the uselessness of Mokken scale analysis]. Tijdschrift voor Onderwijsresearch, 7, 11-24.
39.
Jansen, P.W.G. (1982b). Homogenitatsmessung mit Hilfe des Koeffizienten H von Loevinger: Eine kritische Diskussion [Measuring homogeneity by means of Loevinger's coefficient H: A critical discussion]. Psychologische Beitrage, 24, 96-105.
40.
Jansen, P. G. W. , Roskam, E. E. Ch. I., & Van den Wollenberg, A. L. (1982). De Mokkenschaal gewogen [Weighing the Mokken scaling procedure]. Tijdschrift voor Onderwijsresearch, 7, 31-42.
41.
Jansen, P. G. W. , Roskam, E. E. Ch. I., & Van den Wollenberg, A. L. (1984). Discussion on the usefulness of the Mokken procedure for nonparametric scaling. Psychologische Beitrage, 26, 722-735.
42.
Junker, B. W. (1991). Essential independence and likelihood-based ability estimation for polytomous items. Psychometrika, 56, 255-278.
43.
Junker, B. W. (1993). Conditional association, essential independence and monotone unidimensional item response models. The Annals of Statistics, 21, 1359-1378.
44.
Junker, B. W. (in press). Some remarks on Scheiblechner's treatment of ISOP models. Psychometrika.
45.
Junker, B. W. , & Ellis, J. L. (1997). A characterization of monotone unidimensional latent variable models. The Annals of Statistics, 25, 1327-1343.
46.
Junker, B. W., & Sijtsma, K. (1997). Latent and manifest monotonicity in item response models. Manuscript submitted for publication.
47.
Kelderman, H. , & Rijkes, C. P. M. (1994). Loglinear multidimensional IRT models for polytomously scored items. Psychometrika, 59, 149-176.
48.
Kim, H. R. , Zhang, J., & Stout, W. (1996).A new index of dimensionality-DETECT (Internal Report). Urbana-Champaign: University of Illinois, Dept. of Statistics.
49.
Kingma, J. , & TenVergert, E.M. (1985). A nonparametric scale analysis of the development of conservation. Applied Psychological Measurement, 9, 375-387.
50.
Laros, J. A. , & Tellegen, P. J. (1991). Construction and validation of the SON-R 5 1/2-17, the Snijders-Oomen non-verbal intelligence test. Groningen, The Netherlands: Wolters-Noordhoff.
51.
Lehmann, E. L. (1959). Testing statistical hypotheses. New York: Wiley.
52.
Lehmann, E. L. (1966). Some concepts of dependence. The Annals of Mathematical Statistics, 37, 1137-1153.
53.
Lehmann, E. L. (1986). Testing statistical hypotheses (2nd ed.). New York: Chapman & Hall.
54.
Levine, M. V. (1970). Transformations that render curves parallel. Journal of Mathematical Psychology, 7, 410-443.
55.
Lewis, C. (1982). Developments in nonparametric ability estimation (Heymans Bulletin HB-82-609-EX). Groningen, The Netherlands: Rijksuniversiteit Groningen.
56.
Lewis, C. (1983). Bayesian inference for latent abilities. In S. B. Anderson & J. S. Helmick (Eds.), On educational testing (pp. 224-251). San Francisco: Jossey-Bass.
57.
Loevinger, J. (1948). The technique of homogeneous tests compared with some aspects of "scale analysis" and factor analysis. Psychological Bulletin, 45, 507-530.
58.
Lord, F. M. (1980). Applications of item response theory to practical testing problems. Hillsdale NJ: Erlbaum.
59.
Lord, F. M. , & Novick. M. R. (1968). Statistical theories of mental test scores. Reading MA: Addison-Wesley.
60.
Masters, G. N. (1982). A Rasch model for partial credit scoring. Psychometrika, 47, 149-174.
61.
Meijer, R. R. (1994). Nonparametric personfit analysis. Unpublished doctoral dissertation, Vrije Universiteit, Amsterdam.
62.
Meijer, R. R. , Molenaar, I. W., & Sijtsma, K. (1994). Influence of test and person characteristics on nonparametric appropriateness measurement. Applied Psychological Measurement, 18, 111-120.
63.
Meijer, R. R. , Sijtsma, K., & Molenaar, I. W. (1995). Reliability estimation for single dichotomous items based on Mokken's IRT model. Applied Psychological Measurement, 19, 323-335.
64.
Meijer, R. R. , Sijtsma, K., & Smid, N. G. (1990). The oretical and empirical comparison of the Mokken and the Rasch approach to IRT. Applied Psychological Measurement, 14, 283-298.
65.
Meredith, W. (1965). Some results based on a general stochastic model for mental tests. Psychometrika, 30,419-440.
66.
Mokken, R. J. (1971). A theory and procedure of scale analysis. Berlin: De Gruyter.
67.
Mokken, R. J. (1997). Nonparametric models for dichotomous responses. In W. J. van der Linden & R. K. Hambleton (Eds.), Handbook of modern item response theory (pp. 351-367). New York: Springer.
68.
Mokken, R. J. , & Lewis, C. (1982). A nonparametric approach to the analysis of dichotomous item responses. Applied Psychological Measurement, 6, 417-430.
69.
Mokken, R. J. , Lewis, C., & Sijtsma, K. (1986). Rejoinder to 'The Mokken scale: A critical discussion."Applied Psychological Measurement, 10, 279-285.
70.
Molenaar, I. W. (1982a). De beperkte bruikbaarheid van Jansens kritiek [On the limited usefulness of Jansen's criticisms]. Tijdschrift voor Onderwijsresearch, 7, 25-30.
71.
Molenaar, I. W. (1982b). Een tweede weging van de Mokkenschaal [A second weighing of the Mokken scaling procedure]. Tijdschrift voor Onderwijsresearch, 7, 172-181.
72.
Molenaar, I. W. (1982c). Mokken scaling revisited. Kwantitatieve Methoden, 3(8), 145-164.
73.
Molenaar, I. W. (1986). Een vingeroefening in item response theorie voor drie geordende antwoordcategorieen [An exercise in item response theory for three ordered answer categories]. In G. F. Pikkemaat & J. J. A. Moors (Eds.), Liberamicorum Jaap Muilwijk (pp. 39-57). Groningen, The Netherlands: Econometrisch Instituut.
74.
Molenaar, I. W. (1991). A weighted Loevinger Hcoefficient extending Mokken scaling to multicategory items. Kwantitatieve Methoden, 12(37), 97-117.
75.
Molenaar, I. W. (1997). Nonparametric models for polytomous responses. In W. J. van der Linden & R. K. Hambleton (Eds.), Handbook of modern item response theory (pp. 369-380). New York: Springer.
76.
Molenaar, I. W. , & Sijtsma, K. (1984). Internal consistency and reliability in Mokken's nonparametric item response model. Tijdschrift voor Onderwijsresearch, 9, 257-268.
77.
Molenaar, I. W. , & Sijtsma, K. (1988). Mokken's approach to reliability estimation extended to multicategory items. Kwantitatieve Methoden, 9(28), 115-126.
78.
Molenaar, I. W. , Debets, P., Sijtsma, K., & Hemker, B. T. (1994). User's manualfor MSP. Groningen, The Netherlands: iecProGAMMA.
79.
Moorer, P. & Suurmeijer, T. P. B. M. (1994). A study of the unidimensionality and cumulativeness of the MOS short-form general health survey. Psychological Reports, 74, 467-470.
80.
Muraki, E. (1992). A generalized partial credit model: Application of an EM algorithm. Applied Psychological Measurement, 16, 159-176.
81.
Nandakumar, R. (1993). Assessing essential unidimensionality of real data. Applied Psychological Measurement, 17, 29-38.
82.
Nandakumar, R. (1994). Assessing dimensionality of a set of item responses: Comparison of different approaches. Journal of Educational Measurement, 31, 17-35.
83.
Nandakumar, R. , & Stout, W. F. (1993). Refinements of Stout's procedure for assessing latent trait unidimensionality. Journal of Educational Statistics, 18, 41-68.
84.
Niemoller, B. (1980). Mokken scale. STAP User's Manual, Volume 4. Amsterdam: University of Amsterdam Press.
85.
Niemoller, B., & Van Schuur, W. H. (1980). Mokken test. STAP User's Manual, Volume 4. Amsterdam: University of Amsterdam Press.
86.
Niemoller, K. , & Van Schuur, H. (1983). Stochastic models for unidimensional scaling: Mokken and Rasch. In D. McKay, N. Schofield, & P. Whitely (Eds.), Data analysis and the social sciences (pp. 120-170). London: Frances Pinter.
87.
Nunnally, J. C. (1978). Psychometric theory. New York: McGraw-Hill.
88.
Post, W. J. (1992). Nonparametric unfolding models: A latent structure approach. Leiden University, the Netherlands: DSWO Press.
89.
Post, W. J. , & Snijders, T. A. B. (1993). Nonparametric unfolding models for dichotomous data. Methodika, 7, 130-156.
90.
Ramsay, J. O. (1991). Kernel smoothing approaches to nonparametric item characteristic curve estimation. Psychometrika, 56, 611-630.
91.
Ramsay, J. 0. (1993). TESTGRAF: A computer program for nonparametric analysis of testing data. Unpublished manuscript, McGill University.
92.
Ramsay, J. O. (1995). A similarity-based smoothing approach to nondimensional item analysis. Psychometrika, 60, 323-339.
93.
Rasch, G. (1960). Probabilistic models for some intelligence and attainment tests. Copenhagen: Nielsen & Lydiche.
94.
Ringdal, G. I. , & Ringdal, K. (1993). Testing the EORTC Quality of Life Questionnaire on cancer patients with heterogeneous diagnosis. Quality of Life Research, 2, 129-140.
95.
Rosenbaum, P. R. (1984). Testing the conditional independence and monotonicity assumptions of item response theory. Psychometrika, 49, 425-435.
96.
Rosenbaum, P. R. (1985). Comparing distributions of item responses for two groups. British Journal of Mathematical and Statistical Psychology, 38, 206-215.
97.
Rosenbaum, P. R. (1987a). Comparing item characteristic curves. Psychometrika, 52, 217-233.
98.
Rosenbaum, P. R. (1987b). Probability inequalities for latent scales. British Journal of Mathematical and Statistical Psychology, 40, 157-168.
99.
Roskam, E. E. , & Jansen, P. G. W. (1984). A new derivation of the Rasch model. In E. Degreef & J. van Buggenhaut (Eds.), Trends in mathematical psychology (pp. 293-307). Amsterdam: Elsevier Science Publishers.
100.
Roskam, E. E. , Van den Wollenberg, A. L., & Jansen, P. G. W. (1986). The Mokken scale: A critical discussion. Applied Psychological Measurement, 10, 265-277.
101.
Rost, J. (1996). Test Theorie. Test Konstruktion [Test theory. Test construction]. Bern: Huber.
102.
Roussos, L. , & Stout, W. (1995, April). Simulation study of the effectiveness of using new proximity measures with hierarchical cluster analysis to detectdimensionality structure in simulated data. Paper presented at the Annual meeting of the National Council on Measurement in Education, San Francisco.
103.
Roussos, L. A. , Stout, W. F., & Marden, J. I. (1994). Analysis of the multidimensional simple structure of standardized tests using DIMTEST with hierarchical cluster analysis. Paper presented at the Annual meeting of the National Council on Measurement in Education, San Francisco.
104.
Samejima, F. (1969). Estimation of latent trait ability using a response pattern of graded scores. Psychometrika Monograph Supplement, No. 17.
Schulman, R. S. , & Haden, R. L. (1975). A test theory model for ordinal measurements. Psychometrika, 40, 455-472.
107.
Shealy, R. T. , & Stout, W. F. (1993). An item response theory model for test bias and differential item functioning. In P. W. Holland & H. Wainer (Eds.), Differential itemfunctioning (pp. 197-239). Hillsdale NJ: Erlbaum.
108.
Sijtsma, K. (1984). Useful nonparametric scaling: A reply to Jansen. Psychologische Beitrage, 26, 423-437.
109.
Sijtsma, K. (1986). Another note on the usefulness of Mokken scaling. Psychologische Beitrage, 28, 425-432.
110.
Sijtsma, K. (1988). Contributions to Mokken's nonparametric item response theory. Amsterdam: Free University Press.
111.
Sijtsma, K. , & Hemker, B. T. (in press). Nonparametric polytomous IRT models for invariant item ordering, with results for parametric models. Psychometrika.
112.
Sijtsma, K. , & Junker, B. W. (1996). A survey of theory and methods of invariant item ordering. British Journal of Mathematical and Statistical Psychology, 49, 79-105.
113.
Sijtsma, K. , & Junker, B. W. (1997). Invariant item ordering of transitive reasoning tasks. In J. Rost & R. Langeheine (Eds.), Applications of latent trait and latent class models in the social sciences (pp. 100-110). Munster, Germany: Waxmann Verlag.
114.
Sijtsma, K. , & Meijer, R. R. (1992). A method for investigating the intersection of item response functions in Mokken's nonparametric IRT model. Applied Psychological Measurement, 16, 149-157.
115.
Sijtsma, K. , & Molenaar, I. W. (1987). Reliability of test scores in nonparametric item response theory. Psychometrika, 52, 79-97.
116.
Sijtsma, K. , & Prins, P. M. (1986). Itemselectie in het Mokken model [Item selection in the Mokken model]. Tijdschrift voor Onderwijsresearch, 11, 121-129.
117.
Stokman, F. N. (1977). Roll calls and sponsorship: A methodological analysis of third world group formation in the United Nations. Leyden: Sijthoff.
118.
Stokman, F. N. , & Van Schuur, W. H. (1980). Basic scaling. Quality & Quantity, 14, 5-30.
119.
Stout, W. F. (1987). A nonparametric approach for assessing latent trait unidimensionality. Psychometrika, 52, 589-617.
120.
Stout, W. F. (1990). A new item response theory modeling approach with applications to unidimensionality assessment and ability estimation. Psychometrika, 55, 293-325.
121.
Stout, W. F. , Habing, B., Douglas, J., Kim, H., Roussos, L., & Zhang, J. (1996). Conditional covariance based nonparametric multidimensionality assessment. Applied Psychological Measurement, 20, 331-354.
122.
Suppes, P. , & Zanotti, M. (1981). When are probabilistic explanations possible?Synthese, 48, 191-199.
123.
Van der Flier, H. (1980). Vergelijkbaarheid van individuele testprestaties [Comparability of individual test performance]. Lisse: Swets & Zeitlinger.
124.
Van der Flier, H. (1982). Deviant response patterns and comparability of test scores. Journal of CrossCultural Psychology, 13, 267-298.
125.
Van der Linden, W. J. , & Boekkooi-Timminga, E. (1989). A maximin model for test design with practical constraints. Psychometrika, 54, 237-247.
126.
Van der Veen, G. (1992). Principes in praktijk. CNV-leden over collectieve acties [Principles into practice. Labor union members on means of political pressure]. Kampen, The Netherlands: Kok.
127.
Van Schuur, W. H. (1984). Structure in political beliefs: A new model for stochastic unfolding with application to European party activists. Amsterdam: CT Press.
128.
Verhelst, N. D. , & Glas, C. A. W. (1995). The one parameter logistic model. In G. H. Fischer & I. W. Molenaar (Eds.), Rasch models: Foundations, recent developments, and applications (pp. 215-237). New York: Springer-Verlag.
129.
Verweij, A. C. (1994). Scaling transitive inference in 7-12 year old children. Unpublished doctoral dissertation, Vrije Universiteit, Amsterdam.
130.
Verweij, A. C. , Sijtsma, K., & Koops, W. (1996). A Mokken scale for transitive reasoning suited for longitudinal research. International Journal of Behavioral Development, 19, 219-238.
131.
Wainer, H. (1990). Computerized adaptive testing: A primer. Hillsdale NJ: Erlbaum.
132.
Weijmar Schultz, W. M. C., & Van de Wiel, H. B. M. (1991). Sexual functioning after gyneacological cancer treatment. Unpublished doctoral dissertation, Rijksuniversiteit, Groningen.
133.
Weiss, D. J. (1985). Adaptive testing by computer. Journal of Consulting and Clinical Psychology, 53, 774-789.
134.
Zhang, J. , & Stout, W. (1995, April). Theoretical results concerning DETECT Paper presented at the Annual meeting of the National Council on Measurement in Education, San Francisco.
