Interest in the application of large-scale computerized adaptive testing has focused attention on issues that arise when theoretical advances are made operational. One such issue is that of the order in which exaniinees address questions within a test or separately timed test section. In linear testing, this order is entirely under the control of the examinee, who can look ahead at questions and return and revise answers to questions. Using simulation, this study investigated three models that permit restricted examinee control over revising previous answers in the context of adaptive testing. Even under a worstcase model of examinee revision behavior, two of the models of permitting item revisions worked well in preserving test fairness and accuracy. One model studied may also preserve some cognitive processing styles developed by examinees for a linear testing environment.
Get full access to this article
View all access options for this article.
References
1.
Eignor, D. R. , Folk, V. G., Li, M.-Y, & Stocking, M. L. (1994, April). Pinpointing PRAXIS I CAT characteristics through simulation procedures. Paper presented at the Annual meeting of the National Council on Measurement in Education, New Orleans.
2.
Eignor, D. R., Way, W. D., Stocking, M. L., & Steffen, M. (1993). Case studies in computer adaptive test design through simulations (Research Rep. 93-56). Princeton NJ: Educational Testing Service.
3.
Green, B. F. , Bock, R. D., Humphreys, L. G., Linn, R. L., & Reckase, M. D. (1984). Technical guidelines for assessing computerized adaptive tests. Journal of Educational Measurement, 21, 347-360.
4.
Jacobson, R. L. (1993, September 13). New computer technique seen producing a revolution in testing. The Chronicle of Higher Education, p. A22-A22.
5.
Lord, F. M. (1977). A broad-range tailored test of verbal ability. Applied Psychological Measurement, 1, 95-100.
6.
Lord, F. M. (1980). Applications of item response theory to practical testing problems. Hillsdale NJ: Erlbaum.
7.
Lunz, M. E. , Bergstrom, B. A., & Wright, B. D. (1992). The effect of review on student ability and test efficiency for computerized adaptive tests. Applied Psychological Measurement, 16, 33-40.
8.
Mills, C. N. , & Stocking, M. L. (1996). Practical issues in large-scale computerized adaptive testing. Applied Measurement in Education, 9, 287-304.
9.
Mislevy, R. J. (1984). Estimating latent distributions. Psychometrika, 49, 359-381.
10.
Mislevy, R. J. (1986). Bayes modal estimation in item response models. Psychometrika, 51, 177-195.
11.
Mislevy, R. J. , & Bock, R. D. (1983). BILOG: Item and test scoring with binary LOGISTmodels [Computer program]. Mooresville IN: Scientific Software.
12.
Schaeffer, G., Steffen, M., & Golub-Smith, M. (1993). Introduction of a computer adaptive GRE general test (Research Rep. 93-57). Princeton NJ: Educational Testing Service.
13.
Stocking, M. L. (1996). Revising answers to items in computerized adaptive tests: A comparison of three models (Research Rep. 96-12). Princeton NJ: Educational Testing Service.
14.
Stocking, M. L. , & Swanson, L. (1993). A method for severely constrained item selection in adaptive testing. Applied Psychological Measurement, 17, 277-292.
15.
Stone, G. E. , & Lunz, M. E. (1994). The effect of review on the psychometric characteristics of computerized adaptive tests. Applied Measurement in Education, 7, 211-222.
16.
Swanson, L. , & Stocking, M. L. (1993). A model and heuristic for solving very large item selection problems. Applied Psychological Measurement, 17, 151-166.
17.
Sympson, J. B. , & Hetter, R. D. (1985). Controlling item-exposure rates in computerized adaptive testing. Proceedings of the 27th annual meeting of the Military Testing Association (pp. 973-977). San Diego CA: Navy Personnel Research and Development Center.
18.
Wainer, H. (1992). Some practical considerations when converting a linearly administered test to an adaptive format (Research Rep. 92-13). Princeton NJ: Educational Testing Service.
19.
Wainer, H. , Dorans, N. J., Flaugher, R., Green, B. F., Mislevy, R. J., Steinberg, L., & Thissen, D. (1990). Computerized adaptive testing: A primerHillsdale NJ: Erlbaum.
20.
Ward, W. C. (1988). The College Board computerized placement tests: An application of computerized adaptive testing. Machine-Mediated Learning, 2, 217-282.
21.
Weiss, D. J. (1982). Improving measurement quality and efficiency with adaptive testing. Applied Psychological Measurement, 6, 473-492.
22.
Wingersky, M. S. (1983). LOGIST: A program for computing maximum likelihood procedures for logistic test models. In R. K. Hambleton (Ed.), Applications of item response theory (pp. 45-56). Vancouver: Educational Research Institute of British Columbia.
