Kappa SC : A Measure of Agreement on a Single Rating Category for a Single Item or Object Rated by Multiple Raters

Abstract

Kappa coefficient of agreement on a single rating category, K_SC, with its associated test of statistical significance, is proposed as a measure of the interrater agreement where a single item or object is rated by multiple raters. A goodness-of-fit approach is used to test the statistical significance of all Kappa_sc beyond chance agreement.

References

Fleiss

J. L.

(1981) Statistical methods for rates and proportions. New York: Wiley. P. 217.

Szalai

J. P.

(1993) The statistics of agreement on a single item or object by multiple raters. Perceptual and Motor Skills, 77, 377–378.