Temporal Interval and the Estimation of the Reliability of Work Performance Data

Abstract

The stability of estimates of test-retest reliability was studied in a situation in which the interval separating observation periods was permitted to vary. Weekly production data were collected from 24 women sewing-machine operators, and 16 women in folding and packaging jobs. Correlations between performance periods were assembled into arrays each containing weeks separated by a fixed interval, K. The variability of the distribution of coefficients, S_r, was determined for each value of K, and the regression of S_r on K was found to fit the general form Sr = K/(a + bK) for each group. Most of the increase in Sr occurred between the interval K = 1 to 25, and subsequent analyses yielded no significant relationship between the presence of aberrant reliability estimates and the time separating observation periods. Both magnitude of the reliability estimate and the accuracy with which the estimate is made vary as a function of the temporal dimension of the test-retest design.

References

Bittner

A. C.

(1979) Statistical tests for differential stability. Proceedings of the 23rd Annual Meeting of the Human Factors Society, 23, 541–545.

Hochhaus

(1981) Curve fitting by estimation and iteration. Behavior Research Methods & Instrumentation, 13, 753–754.

Humphreys

L. G.

(1960) Investigations of the simplex. Psychometrika, 25, 313–324.

Jones

M. B.

Stabilization and task definition in a performance test battery. Proceedings of the 23rd Annual Meeting of the Human Factors Society, 23, 536–540.

Lewis

(1960) Quantitative methods in psychology. New York: McGraw-Hill.

Pearson

E. S.

Hearley

H. O.

(Eds.) (1970) Biometricka tables for statisticians. Vol. 1. (3rd ed.) Cambridge, Eng.: Cambridge University Press.

Rambo

W. W.

Chomiak

A. M.

Price

J. M.

(1983) The consistency of performance under stable conditions of work. Journal of Applied Psychology, 68, 78–87.