In sequential mastery testing (SMT), assessment via computer is used to classify examinees into one of two mutually exclusive categories. Unlike paper-and-pencil tests, SMT has the capability to use variable-length stopping rules. One approach to shortening variable-length tests is stochastic curtailment, which halts examination if the probability of changing classification decisions is low. The estimation of such a probability is therefore a critical component of a stochastically curtailed test. This article examines several variations on stochastic curtailment where the key probability is estimated more aggressively than the standard formulation, resulting in additional savings in average test length (ATL). In two simulation sets, the variations successfully reduced the ATL, and in many cases the average loss, compared with the standard formulation.
Bartroff, J., Finkelman, M., & Lai, T.L. ( 2008). Modern sequential analysis and its applications to computerized adaptive testing. Psychometrika, 73, 473-486.
2.
Birnbaum, A. ( 1968). Some latent trait models and their use in inferring an examinee’s ability. In F. M. Lord & M. R. Novick (Eds.), Statistical theories of mental test scores. Reading, MA : Addison-Wesley.
3.
Chang, H.-H., Qian, J., & Ying, Z. ( 2001). a-Stratified multistage CAT with b-blocking. Applied Psychological Measurement, 25, 333-341.
4.
Chang, H.-H., & Stout, W. ( 1993). The asymptotic posterior normality of the latent trait in an IRT model. Psychometrika, 58, 37-52.
5.
Chang, H.-H., & Ying, Z. ( 1996). A global information approach to computerized adaptive testing. Applied Psychological Measurement, 20, 213-229.
6.
Eggen, T.J.H.M. ( 1999). Item selection in adaptive testing with the sequential probability ratio test. Applied Psychological Measurement , 23, 249-261.
7.
Eggen, T.J.H.M., & Straetmans, G.J.J.M. (2000). Computerized adaptive testing for classifying examinees into three categories. Educational and Psychological Measurement, 60, 713-734.
8.
Eggen, T., & Wouda, J. ( 2008, July). Stochastic curtailment in computerized classification testing in three categories. Paper presented at the III European Congress of Methodology, Oviedo, Spain.
9.
Eisenberg, B., & Ghosh, B.K. ( 1980). Curtailed and uniformly most powerful sequential tests . The Annals of Statistics, 8, 1123-1131.
10.
Eisenberg, B., & Simons, G. ( 1978). On weak admissibility of tests. The Annals of Statistics, 6, 319-332. Finkelman, M. (2008). On using stochastic curtailment to shorten the SPRT in sequential mastery testing. Journal of Educational and Behavioral Statistics , 33, 442-463.
11.
Hambleton, R.K., Swaminathan, H., & Rogers, H.J. ( 1991). Fundamentals of item response theory. Newbury Park, CA: Sage.
12.
Jennison, C., & Turnbull, B.W. (2000). Group sequential methods with applications to clinical trials. Boca Raton, FL: Chapman & Hall/CRC.
13.
Kingsbury, G.G., & Zara, A.R. ( 1989). Procedures for selecting items for computerized adaptive tests. Applied Measurement in Education, 2, 359-375.
14.
Lai, T.L., & Shih, M.C. ( 2004). Power, sample size and adaptation considerations in the design of group sequential clinical trials. Biometrika, 91, 507-528.
15.
Lan, K.K.G., Simon, R., & Halperin, M. ( 1982). Stochastically curtailed tests in long-term clinical trials. Communications in Statistics-Sequential Analysis, 1, 207-219.
16.
Lord, F.M. ( 1980). Applications of item response theory to practical testing problems. Hillsdale, NJ: Lawrence Erlbaum.
17.
Petrov, V.V. ( 1995). Limit theorems of probability theory. Oxford, UK: Oxford University Press.
18.
Reckase, M.D. ( 1983). A procedure for decision making using tailored testing . In D. J. Weiss (Ed.), New horizons in testing: Latent trait theory and computerized adaptive testing (pp. 237-254). New York: Academic Press.
19.
Spray, J.A., & Reckase, M.D. (April, 1994). The selection of test items for decision making with a computer adaptive test. Paper presented at the Annual Meeting of the National Council on Measurement in Education, New Orleans, LA.
20.
Spray, J.A., & Reckase, M.D. ( 1996). Comparison of SPRT and sequential Bayes procedures for classifying examinees into two categories using a computerized test. Journal of Educational and Behavioral Statistics, 21, 405-414.
21.
Stocking, M.L., & Lewis, C. ( 1998). Controlling item exposure conditional on ability in computerized adaptive testing. Journal of Educational and Behavioral Statistics , 23, 57-75.
22.
Sympson, J.B., & Hetter, R.D. ( 1985). Controlling item exposure rates in computerized adaptive testing. In Proceedings of the 27th annual meeting of the Military Testing Association (pp. 937-977). San Diego, CA: Navy Personnel Research and Development Center.
23.
van der Linden, W.J. ( 1998). Bayesian item selection criteria for adaptive testing . Psychometrika, 63, 201-216.
24.
van der Linden, W.J. ( 2000). Constrained adaptive testing with shadow tests. In W. J. van der Linden & C. A. W. Glas (Eds.), Computerized adaptive testing: Theory and practice (pp. 27-52). Boston: Kluwer.
25.
Vos, H.J. ( 2000). A Bayesian procedure in the context of sequential mastery testing. Psicológica, 21, 191-211.
26.
Wald, A. ( 1947). Sequential analysis. New York: John Wiley.
27.
Wiberg, M. ( 2003). An optimal design approach to criterion-referenced computerized testing. Journal of Educational and Behavioral Statistics , 28, 97-110.
