A constrained computerized adaptive testing (CAT) algorithm is presented that can be used to equate CAT number-correct (NC) scores to a reference test. As a result, the CAT NC scores also are equated across administrations. The constraints are derived from van der Linden & Luecht’s (1998) set of conditions on item response functions that guarantees identical observed NC score distributions on two test forms. An item bank from the Law School Admission Test was used to compare the results of the algorithm with those for equipercentile observed-score equating, as well as the prediction of NC scores on a reference test using its test response function. The effects of the constraints on the statistical properties of the θ estimator in CAT were examined.
Get full access to this article
View all access options for this article.
References
1.
Chang, H. , &Ying, Z.(inpress). Nonlinearsequential designs for logistic item response theory models with applications to computerized adaptive tests. Annals of Statistics.
2.
Chen, W.-H. , & Thissen, D. (1999). Estimation of item parameters for the three-parameter logistic model using the marginal likelihood of summed scores. British Journal of Mathematical and Statistical Psychology, 52, 19–37.
3.
ILOG (1998). CPLEX 6.0 user’s manual. Incline Village NV: Author.
4.
Kolen, M. J. , & Brennan, R. L. (1995). Test equating: Methods and practices. New York: Springer-Verlag.
5.
Lawrence, I. , & Feigenbaum, M. (1997). Linking scores for computer-adaptive and paper-and-pencil administrations of the SAT (Research Report No. 97-12). Princeton NJ: Educational Testing Service.
6.
Lehmann, E. L. (1999). Elements of large-sample theory. New York: Springer-Verlag.
7.
Lord, F. M. (1980). Applications of item response theory to practical testing problems. Hillsdale NJ: Erlbaum.
8.
Lord, F. M. , & Novick, M. R. (1968). Statistical theories of mental test scores. Reading MA: Addison-Wesley.
9.
Lord, F. M. , & Wingersky, M. S. (1984). Comparison of IRT true-score and equipercentile observed-score “equatings”. Applied Psychological Measurement, 8, 452–461.
10.
Segall, D. O. (1997). Equating the CAT-ASVAB. In W. A. Sands, B. K. Waters, & J. R. McBride (Eds.), Computerized adaptive testing: From inquiry to operation (pp. 181–198). Washington DC: American Psychological Association.
11.
Stocking, M. L. (1996). An alternative method for scoring adaptive tests. Journal of Educational and Behavioral Statistics, 21, 365–389.
12.
Timminga, E. , van der Linden, W. J., & Schweizer, D. A. (1996). ConTEST 2.0: A decision support system for item banking and optimal test assembly [Computer program and manual]. Groningen, The Netherlands: ProGAMMA.
13.
van der Linden, W. J. (1998a). Optimal assembly of psychological and educational tests. Applied Psychological Measurement, 22, 195–211.
14.
van der Linden, W. J. (1998b). Bayesian item selection criteria for adaptive testing. Psychometrika, 63, 201–216.
15.
van der Linden, W. J. (1999). A procedure for empirical initialization of the trait estimator in adaptive testing. Applied Psychological Measurement, 23, 21–29.
16.
van der Linden, W. J. (2000a). Constrained adaptive testing with shadow tests. In W. J. van der Linden & C. A. W. Glas (Eds.), Computerized adaptive testing: Theory and practice (pp. 27–52). Boston: Kluwer-Nijhoff.
17.
van der Linden, W. J. (2000b). A test-theoretic approach to observed-score equating. Psychometrika, 65, 437–456.
18.
van der Linden, W. J. , & Luecht, R. M. (1998). Observed-score equating as a test assembly problem. Psychometrika, 62, 401–418.
19.
van der Linden, W. J. , & Reese, L. M. (1998). A model for optimal constrained adaptive testing. Applied Psychological Measurement, 22, 259–270.
20.
van der Linden, W. J. , Scrams, D. J., & Schnipke, D. L. (1999). Using response-time constraints to control for speededness in computerized adaptive testing. Applied Psychological Measurement, 23, 195–210.
21.
Veldkamp, B. P. (2001). Principles and methods of constrained test assembly (Chapter 4). Doctoral dissertation, University of Twente, The Netherlands.
22.
Yen, W. M. (1984). Obtaining maximum-likelihood estimates from number-correct scores for the three-parameter logistic model. Journal of Educational Measurement, 21, 93–111.
