A General Regression Methodology for ROC Curve Estimation

Abstract

A method for applying generalized ordinal regression models to categorical rating data to estimate and analyze receiver operating characteristic (ROC) curves is presented. These models permit parsimonious adjustment of ROC curve parameters for relevant covariates through two regression equations that correspond to location and scale. Particular shapes of ROC curves are interpreted in relation to the kind of covariates included in the two regressions. The model is shown to be flexible because it is not restricted to the assumption of binormality that is commonly employed in smoothed ROC curve estimation, although the binormal model is one particular form of the more general model. The new method provides a mechanism for pinpointing the effect that interobserver variability has on the ROC curve. It also allows for the adjustment of ROC curves for temporal variation and case mix, and provides a way to assess the incremental diagnostic value of a test. The new methodology is recommended because it substantially improves the ability to assess diagnostic tests using ROC curves. Key words: ROC curves; ordinal regression; technology assessment; diagnostic tests. (Med Decis Making 8:204-215, 1988)

Get full access to this article

View all access options for this article.

References

Alderson PO , Adams DF , McNeil BJ , et al: Computed tomography, ultrasound and scintigraphy of the liver in patients with colon or breast carcinoma: a prospective comparison. Radiology 149:225-230, 1983

Altham Pme : A non-parametric measure of signal discriminability. Br J Math Stat Psychol 26:1-12, 1973

Atkinson AC : Regression diagnostics, transformations and constructed variables (with discussion) . J R Stat Soc B, 44:1-36, 1982

Begg CB : Biases in the assessment of diagnostic tests. Stat Med 6:411-423, 1987

Begg CB , Greenes RA : Assessment of diagnostic tests when disease verification is subject to selection bias. Biometrics 39:207-215, 1983

Birdsall TG : The theory of signal detectability: ROC curves and their character . Dissertation Abstracts Int 281B, 1966

Breslow NE , Day NE (eds): Statistical Methods in Cancer Research, 1—The Analysis of Case-Control Studies. IARC Scientific Publications No. 32. Lyons, France, International Agency for Research on Cancer, 1972

Cox DR : Some procedures associated with the logistic qualitative response curve. In: David FN (ed): Research Papers in Statistics: Festschrift for J. Neyman. New York, John Wiley and Sons, 1966, pp 57-71

Diamond GA : ROC steady: a receiver operating characteristic curve that is invariant relative to selection bias. Med Decis Making 7:238-243, 1987

10.

Dorfinan DD , Alf E. : Maximum likelihood estimation of parameters of signal detection theory and determination of confidence intervals. Rating method data. J Math Psychol 6:487-496, 1969

11.

Doubilet P. , Begg CB , Weinstein MC , et al: Probabilistic sensitivity analysis using Monte Carlo simulation: a practical approach. Med Decis Making 5:157-178, 1985

12.

Doubilet P. , Herman PG : Interpretation of radiographs: effect of clinical history. AJR 137:1055-1058, 1981

13.

Green DM , Swets DA : Signal Detection Theory and Psychophysics. New York, John Wiley and Sons, 1966 (reprinted, New York, Krieger Huntington , 1974 )

14.

Hanley JA : The robustness of the "binormal" assumptions used in fitting ROC curves . Med Decis Making 8:197-203, 1988

15.

Hanley JA , Begg CB : Response to ROC steady. Med Decis Making 7:244-246, 1987

16.

McCullagh P. : Regression models for ordinal data (with discussion) . J R Stat Soc B 42:109-142, 1980

17.

McCullagh P. , Nelder JA : Generalized linear models. London, Chapman and Hall, 1983

18.

Metz CE : Basic principles of ROC analysis. Semin Nucl Med 8:283-298, 1978

19.

Metz CE : ROC methodology in radiologic imaging. Invest Radiol 21:720-733, 1986

20.

Ogilvie JC , Creelman CD : Maximum-likelihood estimation of receiver operating characteristic curve parameters . J Math Psychol 5:377-391, 1968

21.

Swets JA : ROC analysis applied to the evaluation of medical imaging techniques . Invest Radiol 14:109-121, 1979

22.

Swets JA : Indices of discrimination or diagnostic accuracy: their ROCs and implied models . Psychol Bull 99:100-107, 1986

23.

Swets JA : Form of empirical ROCs in discrimination and diagnostic tasks: implications for theory and measurement of performance. Psychol Bull 99:181-198, 1986

24.

Swets JA , Pickett RM : Evaluation of Diagnostic Systems. Methods from Signal Detection Theory . New York, Academic Press, 1982

25.

Swets JA , Pickett RM , Whitehead SF , et al: Assessment of diagnostic technologies. Advanced measurement methods are illustrated in a study of computed tomography of the brain. Science 205:753-759, 1979

26.

Wijesinha A. , Begg CB , Funkenstein HH , et al: Methodology for the differential diagnosis of a complex data set. Med Decis Making 3:133-154, 1983