A Knowledge-Based Approach for Item Exposure Control in Computerized Adaptive Testing

Abstract

The purpose of this study is to investigate a functional relation between item exposure parameters (IEPs) and item parameters (IPs) over parallel pools. This functional relation is approximated by a well-known tool in machine learning. Let P and Q be parallel item pools and suppose IEPs for P have been obtained via a Sympson and Hetter–type simulation. Based on these simulated parameters, a functional relation k = f_P (a, b, c) relating IPs to IEPs of P is obtained by an artificial neural network and used to estimate IEPs of Q without tedious simulation. Extensive experiments using real and synthetic pools showed that this approach worked pretty well for many variants of the Sympson and Hetter procedure. It worked excellently for the conditional Stocking and Lewis multinomial selection procedure and the Chen and Lei item exposure and test overlap control procedure. This study provides the first step in an alternative means to estimate IEPs without iterative simulation.

Keywords

computerized adaptive testing item exposure control item exposure parameters parallel item pools machine learning artificial neural networks

Get full access to this article

View all access options for this article.

References

Ariel

Veldkamp

van der Linden

(2004). Constructing rotating item pools for constrained adaptive testing. Journal of Educational Measurement, 41, 345–359.

Baringhaus

Franz

(2004). On a new multivariate two-sample test. Journal of Multivariate Analysis, 88, 190–206.

Chang

Qian

Ying

(2001). a-Stratified multistage computerized adaptive testing with b blocking. Applied Psychological Measurement, 25, 333–341.

Chang

Ying

(1999). a stratified multistage computerized adaptive testing. Applied Psychological Measurement, 23, 211–222.

Chang

Ansley

(2003). A comparative study of item exposure control methods in computerized adaptive testing. Journal of Educational Measurement, 40, 71–103.

Chen

S. Y.

Lei

P. W.

(2005). Controlling item exposure and test overlap in computerized adaptive testing. Applied Psychological Measurement, 29, 204–217.

Davey

Parshall

C. G.

(1995, April). New algorithms for item selection and exposure control with computerized adaptive testing. Paper presented at the annual meeting of the American Educational Research Association, San Francisco.

Gallant

White

(1992). On learning derivatives of an unknown mapping with mutilayer feedforward networks. Neural Networks, 5, 129–138.

Gupta

M. M.

Jin

Homma

(2003). Static and dynamic neural networks, from fundamentals to advanced theory. Hoboken, NJ: John Wiley & Sons.

10.

Hetter

R. D.

Sympson

J. B.

(1997). Item exposure control in CAT-ASVAB. In Sands

W. A.

Waters

B. K.

McBride

J. R.

(Eds.). Computerized adaptive testing: From inquiry to operation (pp. 141–144). Washington, DC: American Psychological Association.

11.

Kingsbury

G. G.

Zara

A. R.

(1989). Procedures for selecting items for computerized adaptive tests. Applied Measurement in Education, 2, 359–375.

12.

Leung

Chang

Hau

(2002). Item selection in computerized adaptive testing: Improving the. a-stratified design with the Sympson-Hetter algorithm. Applied Psychological Measurement, 26, 376–392.

13.

Leung

Chang

Hau

(2003). Computerized adaptive testing: A comparison of three content balancing methods. The Journal of Technology, Learning and Assessment, 2, 1–15.

14.

McBride

Martin

(1983). Reliability and validity of adaptive ability tests in a military setting. In Weiss

D. J.

(Ed.) New horizons in testing (pp. 223–236). New York: Academic Press.

15.

Minsky

M. L.

Papert

S. A.

(1969). Perceptrons. Cambridge, MA: MIT Press.

16.

Munakata

(1998). Fundamentals of the new artificial intelligence beyond traditional paradigms. New York: Springer-Verlag.

17.

Segall

D. O.

Moreno

K. E.

(1999). Development of the CAT-ASVAB. In Drasgow

Olson-Buchanan

J. B.

(Eds.). Innovations in computerized assessment (pp. 35–65). Hillsdale, NJ: Lawrence Erlbaum Associates.

18.

Stocking

(1994). Three practical issues for modern adaptive testing item pools (Research Report 94–5). Princeton, NJ: Educational Testing Service.

19.

Stocking

Lewis

(1998). Controlling item exposure conditional on ability in computerized adaptive testing. Journal of Educational and Behavior Statistics, 23, 57–75.

20.

Stocking

Lewis

(2000). Methods of controlling the exposure of items in CAT. In van der Linden

W. J.

Glas

A. W.

(Eds.). Computerized adaptive testing: Theory and practice. Norwell, MA: Kluwer Academic Publishers.

21.

Sympson

Hetter

(1985, October). Controlling item-exposure rates in computerized adaptive testing. In Proceedings of the 27th Annual Meeting of the Military Testing Association (pp. 973–977). San Diego, CA: Navy Personnel Research and Development Center.

22.

Taylor

Thompson

(1986). Data based random number generation for a multivariate distribution via stochastic simulation. Computational Statistics & Data Analysis, 4, 93–101.

23.

van der Linden

(2003). Some alternatives to Sympson-Hetter item-exposure control in computerized adaptive testing. Journal of Educational and Behavioral Statistics, 28, 249–265.

24.

Way

(1998). Protecting the integrity of computerized testing item pools. Educational Measurement: Issues and Practice, 17, 17–27.