The current study compared student performance between paper-and-pencil testing (PPT) and computer-based testing (CBT) on a large-scale statewide end-of-course English examination. Analyses were conducted at both the item and test levels. The overall results suggest that scores obtained from PPT and CBT were comparable. However, at the content domain level, a rather large difference in the reading comprehension section suggests that reading comprehension test may be more affected by the test administration mode. Results from the confirmatory factor analysis suggest that the administration mode did not alter the construct of the test.
Bridgeman, B., Lennon, M.L., & Jackenthal, A. (2003). Effects of screen size, screen resolution, and display rate on computer-based test performance. Applied Measurement in Education, 16, 191-205.
Choi, S.W., & Tinkler, T. (2002, April). Evaluating comparability of paper and computer-based assessment in a K-12 setting. Paper presented at the annual Meeting of the National Council on Measurement in Education, New Orleans, LA.
4.
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Ealbaum.
5.
Finney, S.J., & DiStefano, C. (2006). Dealing with nonnormal and categorical data in structural equation models. In G. R. Hancock & R. O. Mueller (Eds.), Structural equation modeling: A second course. Greenwich, CT: Information Age.
6.
Fitzpatrick, S., & Triscari, R. (2005, April). Comparability studies of the Virginia computer-delivered tests. Paper presented at the annual meeting of the American Educational Research Association, Montreal , Canada.
7.
Gallagher, A., Bridgeman, B., & Cahalan, C. (2002). The effect of computer-based tests on racial-ethnic and gender groups. Journal of Educational Measurement, 39, 133-147.
8.
Grissom, R.J., & Kim, J.J. (2005). Effect sizes for research: A broad practical approach . Mahwah, NJ: Lawrence Erlbaum.
9.
Higgins, J., Russell, M., & Hoffmann, T. (2005). Examining the effect of computer-based passage presentation on reading test performance. Journal of Technology, Learning, and Assessment, 3(4). Retrieved July 5, 2005, from http://escholarship.bc.edu/jtla
10.
Hu, L., & Bentler, P.M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6, 1-55.
11.
Huynh, H., & Barton, K. (2006). Performance of students with disabilities under regular and oral administrations for a high-stakes reading examination. Applied Measurement in Education, 19, 21-39.
12.
Jöreskog, K., & Sörbom, D. (2001). LISREL 8.5.1 for Windows [Computer Software]. Lincolnwood, IL: Scientific Software International .
13.
Linacre, J.M. (2005). WINSTEPS version 3.59.1 [Computer software]. Chicago: Author.
14.
Maryland State Department of Education (2005 ). 2005 MSA reading technical report. Retrieved September 2, 2006, from http://www.marylandpublicschools.org/msde/divisions/planningresultstest/2005+MSA+Reading+Technical+Report
15.
Mayes, D.K., Sims, V.K., & Koonce, J.M. (2001). Comprehension and workload differences for VDT and paper-based reading. International Journal of Industrial Ergonomics, 28, 367-378.
16.
Mead, A.D., & Drasgow, F. (1993). Equivalence of computerized and paper-and-pencil cognitive ability tests: A meta-analysis. Psychological Bulletin , 114, 449-458.
17.
Minnesota Department of Education (2004). Minnesota basic skills tests technical manual 2004 administration. Retrieved May 30, 2007, from http://education.state.mn.us/mdeprod/groups/Assessment/documents/Report/006696.pdf
18.
Nichols, P., & Kirkpatrick, R. (2005, April). Comparability of the computer-administered tests with existing paper-and-pencil tests in reading and mathematics tests. Paper presented at the annual Meeting of the American Educational Research Association, Montreal, Canada.
19.
Noyes, J.M., Garland, K.J., & Robbins, E.L. (2004). Paper-based versus computer-based assessment: Is workload another test mode effect? British Journal of Educational Technology, 35, 111-113.
20.
Poggio, J., Glasnapp, D.R., Yang, X., Beauchamp, A., & Dunham, M. (2005, April). Moving from paper and pencil to online testing: Findings from a state large scale assessment program. Paper presented at the annual meeting of the National Council on Measurement in Education, Montreal, Canada.
21.
Pommerich, M. (2004). Developing computerized versions of paper-and-pencil tests: Mode effects for passage-based tests. Journal of Technology, Learning, and Assessment, 2(6). Retrieved July 9, 2005, from http://escholarship.bc.edu/jtla
22.
Pomplun, M., Frey, S., & Becker, D.F. (2002). The score equivalence of paper-and-pencil and computerized versions of a speeded test of reading comprehension. Educational and Psychological Measurement, 62, 337-354.
23.
Rasch, G. (1960). Probabilistic models for some intelligence and attainment tests. Copenhagen, Denmark: Danish Institute for Educational Research.
24.
Russell, M., Goldberg, A., & O'Connor, K. (2003) Computer-based testing and validity: A look back into the future. Assessment in Education, 10, 278-293.
25.
SAS Institute Inc. (2004). SAS version 9.1.2 [Computer software]. Cary, NC: Author.
26.
South Carolina Department of Education. (2003 ). Technical documentation for the 2003 Palmetto Achievement Challenge Tests of English language arts, mathematics, science, and social studies. Columbia, SC: Author.
27.
SPSS Inc. (2004). SPSS version 13.0 for Windows [Computer software]. Chicago: Author.
28.
Swanson, C.B. (2006, May 4). Tracking U.S. trends. Education Week's Technology Counts, 25(35), 50-53.
29.
van de Vijver, F.J.R., & Harsveld, M. (1994). The incomplete equivalence of the paper-and-pencil and computer versions of the general aptitude test battery. Journal of Applied Psychology, 79, 852-859.
30.
Wright, B.D., & Stone, M.H. (1979). Best test design. Chicago: MESA Press.
