Bootstrap-based procedures for inference in nonparametric receiver-operating characteristic curve regression analysis

Abstract

Prior to using a diagnostic test in a routine clinical setting, the rigorous evaluation of its diagnostic accuracy is essential. The receiver-operating characteristic curve is the measure of accuracy most widely used for continuous diagnostic tests. However, the possible impact of extra information about the patient (or even the environment) on diagnostic accuracy also needs to be assessed. In this paper, we focus on an estimator for the covariate-specific receiver-operating characteristic curve based on direct regression modelling and nonparametric smoothing techniques. This approach defines the class of generalised additive models for the receiver-operating characteristic curve. The main aim of the paper is to offer new inferential procedures for testing the effect of covariates on the conditional receiver-operating characteristic curve within the above-mentioned class. Specifically, two different bootstrap-based tests are suggested to check (a) the possible effect of continuous covariates on the receiver-operating characteristic curve and (b) the presence of factor-by-curve interaction terms. The validity of the proposed bootstrap-based procedures is supported by simulations. To facilitate the application of these new procedures in practice, an R-package, known as npROCRegression, is provided and briefly described. Finally, data derived from a computer-aided diagnostic system for the automatic detection of tumour masses in breast cancer is analysed.

Keywords

Receiver-operating characteristic curve generalised additive models bootstrap computer-aided diagnosis

Get full access to this article

View all access options for this article.

References

Krzanowski

Hand

. ROC curves for continuous data, 1st ed. Monographs on Statistics and Applied Probability, Boca Raton, FL: Chapman & Hall/CRC, 2009.

Pepe

. The statistical evaluation of medical tests for classification and prediction, New York: Oxford Statistical Sciences Series, Oxford University Press, 2003.

Zou

Obuchowski

McClish

. Statistical methods in diagnostic medicine, New York: Wiley, 2002.

Janes

Pepe

. Adjusting for covariates in studies of diagnostic, screening, or prognostic markers: an old concept in a new setting. Am J Epidemmiol 2008; 168: 89–97.

Janes

Pepe

. Adjusting for covariate effects on classification accuracy using the covariate-adjusted receiver operating characteristic curve. Biometrika 2009; 96: 371–382.

Pardo-Fernández

Rodríguez-Álvarez

Van Keilegom

. A review on ROC curves in the presence of covariates. REVSTAT-Stat J 2014; 12: 21–41.

Rodríguez-Álvarez

Tahoces

Cadarso-Suárez

et al.

Comparative study of ROC regression techniques – applications for the computer-aided diagnostic system in breast cancer detection. Comput Stat Data An 2011; 55: 888–902.

López-de-Ullibarri

Cao

Cadarso-Suárez

et al.

Nonparametric estimation of conditional ROC curves: application to discrimination tasks in computerized detection of early breast cancer. Comput Stat Data An 2008; 52: 2623–2631.

Inàcio de Carvalho

Jara

Hanson

et al.

Bayesian nonparametric ROC regression modeling. Bayesian Anal 2013; 8: 623–646.

10.

Faraggi

. Adjusting receiver operating curves and related indices for covariates. Statistician 2003; 52: 179–192.

11.

González-Manteiga

Pardo-Fernández

van Keilegom

. ROC curves in non-parametric location-scale regression models. Scand J Stat 2011; 38: 169–184.

12.

Pepe

. Three approaches to regression analysis of receiver operating characteristic curves for continuous test results. Biometrics 1998; 54: 124–135.

13.

Rodríguez-Álvarez

Roca-Pardiñas

Cadarso-Suárez

. ROC curve and covariates: extending induced methodology to the non-parametric framework. Stat Comput 2011; 21: 483–499.

14.

Rodríguez

Martínez

. Bayesian semiparametric estimation of covariate-dependent ROC curves. Biostatistics 2013; 15: 353–369.

15.

Zheng

Heagerty

. Semiparametric estimation of timedependent ROC curves for longitudinal marker data. Biostatistics 2004; 5: 615–632.

16.

Yao

Craiu

Reiser

. Nonparametric covariate adjustment for receiver operating characteristic curves. Can J Stat 2010; 38: 27–46.

17.

Alonzo

Pepe

. Distibution-free ROC analysis using binary regression techniques. Biostatistics 2002; 3: 421–432.

18.

Cai

. Semiparametric ROC regression analysis with placement values. Biostatistics 2004; 5: 45–60.

19.

Cai

Pepe

. Semiparametric receiver operating characteristic analysis to evaluate biomarkers for disease. J Am Stat Assoc 2002; 97: 1099–1107.

20.

Pepe

Cai

. The analysis of placement values for evaluating discriminatory measures. Biometrics 2004; 60: 528–535.

21.

McCullagh

Nelder

. Generalized linear models, Monographs on Statistics and Applied Probability, Boca Raton, FL: Chapman & Hall/CRC, 1989.

22.

Rodríguez-Álvarez

Roca-Pardiñas

Cadarso-Suárez

. A new flexible direct ROC regression model: application to the detection of cardiovascular risk factors by anthropometric measures. Comput Stat Data An 2011; 55: 3257–3270.

23.

Hastie

Tibshirani

. Generalized additive models, Monographs on Statistics and Applied Probability, Chapman & Hall/CRC, 1990.

24.

Cai

Zheng

. Model checking for ROC regression analysis. Biometrics 2007; 63: 152–163.

25.

R Core Team. R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria 2017. https://www.R-project.org/ (accessed 27 November 2017).

26.

Fan

Gijbels

. Local polynomial modelling and its applications, Monographs on Statistics and Applied Probability, Boca Raton, FL: Chapman & Hall/CRC, 1996.

27.

Dodd

Pepe

. Semiparametric regression for the area under the receiver operating characteristic curve. J Am Stat Assoc 2003; 98: 409–417.

28.

Dodd

Pepe

. Partial AUC estimation and regression. Biometrics 2003; 59: 614–623.

29.

Cai

Dodd

. Regression analysis for the partial area under the ROC curve. Stat Sinica 2008; 18: 817–836.

30.

Inàcio de Carvalho

de Carvalho

Alonzo

et al.

Functional covariate-adjusted partial area under the specificity–ROC curve with an application to metabolic syndrome diagnosis. Ann Appl Stat 2016; 10: 1472–1495.

31.

Youden

. Index for rating diagnostic tests. Cancer 1950; 3: 32–35.

32.

Wang

Fang

. A model-free estimation for the covariate-adjusted Youden index and its associated cut-point. Stat Med 2014; 33: 4963–4974.

33.

Fan

Marron

. Fast implementations of nonparametric curve estimators. J Comput Graph Stat 1994; 3: 35–56.

34.

Doi

. Computer-aided diagnosis in medical imaging: historical review current status and future potential. Comput Med Imag Grap 2007; 31: 198–211.

35.

Nishikawa

. Current status and future directions of computer-aided diagnosis in mammography. Comput Med Imag Grap 2007; 31: 224–235.

36.

Sickles

. Mammographic features of ‘early’ breast cancer. Am J Roentgenol 1984; 143: 461–464.

37.

Varela

Tahoces

Méndez

et al.

Computerized detection of breast masses in digitized mammograms. Comp Bio Med 2007; 37: 214–226.

38.

López-Ratón

Rodríguez-Álvarez

Cadarso-Suárez

et al.

OptimalCutpoints: an R package for selecting optimal cutpoints in diagnostic tests. J Stat Softw 2014; 61: 1–36. https://jstatsoft.org/index.php/jss/article/view/v061i08 .

39.

Hanley

Hajian-Tilaki

. Sampling variability of non-parametric estimates of the areas under receiver operating characteristic curves: an update. Acad Radiol 1997; 4: 49–58.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.38 MB