A General Framework for Evaluating the Reliability of Medical Measurement Systems

Measurements of physical and social phenomena are often used in the decisionmakingprocess despite the error introduced by human and context variation. It would be advantageous to a decision maker to know the sources and magnitudes of such errors in using a measurement system or in choosing among alternative measurement systems. This paper describes an approach to determining the reliability of a measurement procedure called generalizability (G) theory. G theory uses the results of a random-effects analysis of variance to estimate the magnitude of error variability associated with each of several potential sources. These estimates can then be used to calculate a number of indices which typify the amount of precision provided by a measurement system in a particular context of usage. The major concepts of G theory are described and illustrated in the context of evaluating a radiologic total lung capacity measurement system.