Immediate Feedback and Opportunity to Revise Answers

Abstract

Recently, Attali and Powers investigated the usefulness of providing immediate feedback on the correctness of answers to constructed response questions and the opportunity to revise incorrect answers. This article introduces an item response theory (IRT) model for scoring revised responses to questions when several attempts are allowed. The model is based on Samejima’s graded response model, where the graded responses are defined as the number of attempts needed until a correct answer is given, and the response with the least credit is awarded for an incorrect answer after the maximum number of attempts has been reached. Advantages of this conceptualization are discussed and empirical results are presented.

Keywords

constructed-response items feedback revision IRT graded response model

Get full access to this article

View all access options for this article.

References

Attali

Powers

(2008). Effect of immediate feedback and revision on psychometric properties of open-ended GRE Subject Test items (ETS Research Report No. RR-08-21, GRE Research Report No. GREB-04-05). Princeton, NJ: Educational Testing Service.

Attali

Powers

(2010). Immediate feedback and opportunity to revise answers to open-ended questions. Educational and Psychological Measurement, 70, 22-35.

Campione

J. C.

Brown

A. L.

(1987). Linking dynamic testing with school achievement. In Lidz

C. S.

(Ed.), Dynamic testing (pp. 82-115). New York, NY: Guilford.

Epstein

M. L.

Epstein

B. B.

Brosvic

G. M.

(2001). Immediate feedback during academic testing. Psychological Reports, 88, 889-894.

Gilman

Ferry

(1972). Increasing test reliability through self-scoring procedures. Journal of Educational Measurement, 9, 205-207.

Grigorenko

E. L.

Sternberg

R. J.

(1998). Dynamic testing. Psychological Bulletin, 124, 75-111.

Kulhavy

R. W.

Stock

W. A.

(1989). Feedback in written instruction: The place of response certitude. Educational Psychology Review, 1, 279-308.

Masters

G. N.

(1982). A Rasch model for partial credit scoring. Psychometrika, 47, 149-174.

Masters

G. N.

Wright

B. D.

(1996). The partial credit model. In van der Linden

W. J.

Hambleton

R. K.

(Eds.), Handbook of modern item response theory (pp. 101-122). New York, NY: Springer.

10.

Muraki

(1992). A generalized partial credit model: Application of an EM algorithm. Applied Psychological Measurement, 16, 159-176.

11.

Pressey

S. L.

(1926). A simple apparatus which gives tests and scores—and teaches. School and Society, 23(586), 373-376.

12.

Samejima

(1969). Estimation of ability using a response pattern of graded scores. Psychometrika Monograph, No. 17.

13.

Samejima

(1997). Graded response model. In van der Linden

W. J.

Hambleton

(Eds.), Handbook of modern item response theory (pp. 85-100). New York, NY: Springer.

14.

Schmidt

R. A.

Young

D. E.

Swinnen

Shapiro

D. C.

(1989). Summary knowledge results for skill acquisition: Support for the guidance hypothesis. Journal of Experimental Psychology: Learning, Memory and Cognition, 15, 352-359.

15.

Schooler

L. J.

Anderson

J. R.

(1990). The disruptive potential of immediate feedback. In Proceedings of the Cognitive Science Society, Hillsdale, NJ: Erlbaum.

16.

Thissen

Chen

W.-H.

Bock

R. D.

(2003). MULTILOG (Version 7.03) [Computer software]. Lincolnwood, IL: Scientific Software International.

17.

Thissen

Steinberg

(1986). A taxonomy of item response models. Psychometrika, 51, 567-577.