A Reliable and Valid Weighted Scoring Instrument for Use in Grading APA-Style Empirical Research Report

Abstract

The scoring instrument described in this article is based on a deconstruction of the seven sections of an American Psychological Association (APA)-style empirical research report into a set of learning outcomes divided into content-, expression-, and format-related categories. A double-weighting scheme used to score the report yields a final grade that reflects the relative importance of outcomes in each category and the differential contribution of each section of the report to the report as a whole. The scores produced by the instrument are reliable between and within raters and significantly correlated with students’ cumulative grade point averages. The author hopes the instrument can provide a useful framework for scoring any set of learning outcomes an instructor defines as the essential elements of an APA-style research report.

Keywords

grading scoring scoring rubrics

Get full access to this article

View all access options for this article.

References

American Psychological Association. (2001). Publication manual of the American Psychological Association (5th ed.). Washington, DC: Author.

American Psychological Association. (2010). Publication manual of the American Psychological Association (6th ed.). Washington, DC: Author.

Beins

B. C.

(2009). Research methods: A tool for life (2nd ed.). New York, NY: Pearson.

Christensen

L. B.

Johnson

R. B.

Turner

L. A.

(2011). Research methods: Design, and analysis (11th ed.). New York, NY: Pearson.

Goodwin

C. J.

(2008). Research in psychology: Methods and design (6th ed.). Hoboken, NJ: Wiley.

Gottfried

G. M.

Vosmik

J. F.

Johnson

K. E

. (2009). Assessing student learning: A collection of evaluation tools. Retrieved from http://teachpsych.org/otrp/resources/gottfried09.pdf

Green

K. H.

Emerson

. (2007). A new framework for grading. Assessment and Evaluation in Higher Education, 32(4), 495–511. doi:10.1080/02602930600896571

Kan

(2007). An alternative method in the new educational program from the point of performance-based assessment: Rubric scoring scales. Educational Sciences: Theory and Practice, 7(12) 144–152.

Malini

R. Y

. (2010). A review of rubric use in higher education. Assessment and Evaluation in Higher Education, 35(4), 435–448. doi:10.1080/02602930902862859

10.

Mertler

C. A

. (2001). Designing scoring rubrics for your classroom. Practical Assessment, Research, and Evaluation, 7(25). Retrieved from http://PAREonline.net/getvn.asp?v=7&n=25

11.

Moskal

B. M

. (2000). Scoring rubrics: What, when, and how? Practical Assessment, Research and Evaluation, 7(3). Retrieved from http://pareonline.net/getvn.asp?v=7&n=3

12.

Neuman

W. L.

(2011). Social research methods: Qualitative and quantitative approaches (7th ed.). New York, NY: Pearson.

13.

Peat

(2006). Integrating writing and research skills: Development and testing of a rubric to measure student outcomes. Journal of Public Affairs Education, 12(3), 295–311.

14.

Sadler

P. M.

Good

. (2006). The impact of self- and peer-grading on student learning. Educational Assessment, 11(1), 1–31. doi:10.1207/s15326977ea1101_1

15.

Stellmack

M. A.

Konheim-Kalkstein

Y. L.

Manor

J. E.

Massey

A. R.

Schmitz

J. A. P

. (2009). An assessment of reliability and validity of a rubric for grading APA-style introductions. Teaching of Psychology, 36, 102–107. doi:10.1080/00986280902739776

16.

Thaler

Kazemi

Huscher

. (2009). Developing a rubric to assess student learning outcomes using a class assignment. Teaching of Psychology, 36, 113–116. doi:10/1080/00986280902739305