Confidence Bands for Receiver Operating Characteristic Curves

Abstract

Receiver operating characteristic (ROC) curves are mapped out by the two types of errors that are generated by varying the decision threshold used to determine which subjects will be considered abnormal. Under the conventional binormal model for the ROC curve, two- sided and one-sided simultaneous confidence bands for an entire ROC curve, or for a portion of an ROC curve, are constructed by employing Working-Hotelling-type confidence bands for simple linear regression. Pointwise confidence bands are presented for comparison. The cases in which one has only asymptotically normally distributed estimates of the parameters of the ROC curve and consistent estimates of their variance-covariance matrix are em phasized. The methods extend beyond binormal models. Key words: confidence bands for linear regression; ROC curves. (Med Decis Making 1993;13:191-197)

Get full access to this article

View all access options for this article.

References

Metz CE Basic principles of ROC analysis. Semin Nucl Med. 1978;8:283-98.

Dorfman DD , Alf E. Maximum-likelihood estimation of parameters of signal-detection theory and determination of confidence intervals-rating-method data. J Math Psychol. 1969;6:487-96.

Metz CE , Wang P-L. , Kronman HB A new approach for testing the significance of differences between ROC curves measured from correlated data. In Deconick F , ed. Information processing in medical imaging VIII. The Hague: Martinus Nijhof, 1984 .

Centor RM , Keightley JH The ROC analyzer, ver. 5.2. University of Virginia, Charlottesville, Virginia, 1990.

McNeil BJ , Hanley JA Statistical approaches to the analysis of ROC curves. Med Decis Making. 1984;4:137-50.

Phelps CE , Mushlin AI Focusing technology assessment using medical decision theory. Med Decis Making. 1988;8:279-89.

McClish DK Analyzing a portion of ROC curve. Med Decis Making. 1989;9:190-5.

Hall WJ , Ma G. Confidence bands for regression revisited and applied. In preparation, 1992.

Working H. , Hotelling H. Application of the theory of error to the interpretation of trends. J Am Statist Assoc. 1929;24:73-85.

10.

Scheffé H. A method for judging all contrasts in the analysis of variance. Biometrika. 1953;40:87-194.

11.

Bohrer R. On sharpening Sheffé bounds. J R Statist Soc. 1967;B29:110-4.

12.

Haperin M. , Gurian J. Confidence bands in linear regression with constraints. J Am Statist Assoc. 1968;63:1020-7.

13.

Wynn HP , Bloomfield P. Simultaneous confidence bands in regression analysis. J R Statist Soc. 1971;B33:202-17.

14.

Bohrer R. On one-sided confidence bounds for response surfaces. Bull Int Statist Inst. 1969;43(Book 2):25-257.

15.

Bohrer R. , Francis GK Sharp one-sided confidence bounds over positive regions. Ann Math Statist. 1972;43:1541-8.

16.

Hochberg Y. , Quade D. One-sided simultaneous confidence bounds on regression surfaces with intercepts. J Am Statist Assoc. 1975;70:889-91.

17.

Nair VN Confidence bands for survival function with censored data: a comparative study . Technometrics. 1984;26:265-75.

18.

Grey DR , Morgan Bjt. Some aspects of ROC curve fitting: normal and logistic models. J Math Psychol. 1972;9:128-39.

19.

Hanley JA , McNeil BJ The meaning and use of the area under a ROC curve. Radiology. 1982;143:27-36.

20.

Mushlin AI , Phelps CE , Kido D. , et al. The accuracy of MRI in patients with suspected MS. Report to the Agency for Health Care Policy and Research, from the University of Rochester and the University of Toronto, 1992.

21.

Ma G. Two problems associated with ROC curves. PhD dissertation at the University of Rochester, Rochester, New York, 1992.

22.

Gafarian AV Confidence bands in straight line regression . J Am Statist Assoc. 1964;59:182-213.

23.

Bowden DC , Graybill FA Confidence bands of uniform and proportional width for linear models. J Am Statist Assoc. 1966;61:182-98.