Comparison of Difficulties and Reliabilities of Quantitative Word Problems in Completion and Multiple-Choice Item Formats

Abstract

Quantitative word problems were written as parallel completion and multiple-choice items, and were ad ministered to 232 undergraduate students to compare the reliabilities and item difficulties associated with these formats. The multiple-choice options were writ ten using specific numerical responses for each of five alternatives, revised by replacing the fifth option with "none of the above," and also by replacing each of the five responses with ranges of numerical values. Differences in distributions of scores imply a need to reestablish standards if changes are made in the pro portions of completion and multiple-choice items in cluded in a test. Findings did not support camouflag ing the correct response by using "none of the above" or ranges of numerical values as multiple-choice alter natives. The increased time required to develop and administer a multiple-choice test with reliability equal to that of a completion test suggests use of the latter even in classes with relatively large enrollments.

Get full access to this article

View all access options for this article.

References

Ebel, R.L. (1979). Essentials of educational measurement (3rd ed.). Englewood Cliffs NJ: Prentice-Hall.

Frederiksen, N. , & Satter, G.A. (1953). The construction and validation of an arithmetic computation test. Educational and Psychological Measurement , 13, 209-227.

Friedman, M. (1937). The use of ranks to avoid the assumption of normality implicit in the analysis of variance. Journal of the American Statistical Association, 32, 675-701.

Heim, A.W. , & Watts, K.P. (1967). An experiment on multiple-choice versus open-ended answering in a vocabulary test. British Journal of Educational Psychology, 37, 339-346.

Lord, F.M. (1971). Testing if two measuring procedures measure the same psychological dimension (Research Bulletin RB-71-36. Princeton NJ: Educational Testing Service.

Popham, W.J. (1981). Modern educational measurement. Englewood Cliffs NJ: Prentice-Hall.

Rimland, B. , & Zwerski, E. (1962). The use of open-end data as an aid in writing multiple-choice distractors : An evaluation with arithmetic reasoning and computation items. Journal of Applied Psychology, 46, 31-33.

Traub, R.E. , & Fisher, C.W. (1977). On the equivalence of constructed-response and multiple-choice tests. Applied Psychological Measurement, 1, 355-369.

Ward, W.C. (1982). A comparison of free-response and multiple-choice forms of verbal aptitude tests. Applied Psychological Measurement , 6, 1-11.

10.

Ward, W.C. , Frederiksen, N. , & Carlson, S.B. (1980). Construct validity of free-response and machine-scorable forms of a test. Journal of Educational Measurement, 17, 11-29.

11.

Wesman, A.G. (1971). Writing the test item. In R. L. Thorndike (Ed.), Educational measurement (2nd ed.). Washington DC : American Council on Education.

12.

Wesman, A.G. , & Bennett, G.K. (1946). The use of "none of these" as an option in test comparison. Journal of Educational Psychology, 37, 541—554.