Chance Success Due to Guessing and Non-Independence of True Scores and Error Scores in Multiple-Choice Tests: Computer Trials with Prepared Distributions

The effect of chance success due to guessing upon the variance of multiple-choice test scores was estimated from prepared distributions of large numbers of scores. Each score consisted of an assumed “true score” component and an “error score” component generated by a computer. A large negative correlation was found between true scores and error scores and a positive correlation between error scores and error scores. The equation showing reliability in terms of components of variance was derived under the more restrictive assumption that there is a correlation between true scores and error scores, and the result r o 1 o 2 = 1 − [ ( s o 2 / s o 2 ) ( 1 − r o 1 o 2 ) ] was obtained. The fact that reliability can be positive even though error variance and observed variance are equal was discussed.