Confidence intervals for differences between volumes under receiver operating characteristic surfaces (VUS) and generalized Youden indices (GYIs)

Abstract

This article explores both existing and new methods for the construction of confidence intervals for differences of indices of diagnostic accuracy of competing pairs of biomarkers in three-class classification problems and fills the methodological gaps for both parametric and non-parametric approaches in the receiver operating characteristic surface framework. The most widely used such indices are the volume under the receiver operating characteristic surface and the generalized Youden index. We describe implementation of all methods and offer insight regarding the appropriateness of their use through a large simulation study with different distributional and sample size scenarios. Methods are illustrated using data from the Alzheimer's Disease Neuroimaging Initiative study, where assessment of cognitive function naturally results in a three-class classification setting.

Keywords

Box–Cox transformation comparison of diagnostic markers generalized pivotal quantities three-class receiver operating characteristic analysis

Get full access to this article

View all access options for this article.

References

Nakas

. Developments in ROC surface analysis and assessment of diagnostic markers in three-class classification problems. REVSTAT 2014; 12: 43–65.

Nakas

Yiannoutsos

. Ordered multiple-class ROC analysis with continuous measurements. Stat Med 2004; 23: 3437–3449.

Faraggi

Reiser

. Estimation of the area under the roc curve. Stat Med 2002; 21: 3093–3106.

Xiong

van Belle

Miller

et al.

Measuring and estimating diagnostic accuracy when there are three ordinal diagnostic groups. Stat Med 2006; 25: 1251–1273.

Bantis

Nakas

Reiser

. Construction of confidence regions in the ROC space after the estimation of the optimal Youden index-based cut-off point. Biometrics 2014; 70: 212–223.

Nakas

Alonzo

Yiannoutsos

. Accuracy and cut-off point selection in three-class classification problems using a generalization of the Youden index. Stat Med 2010; 29: 2946–2955.

Mueller

Weiner

Thal

et al.

The Alzheimer's disease neuroimaging initiative. Neuroimaging Clin N Am 2005; 15: 869–877.

Luo

Xiong

. Youden index and associated cut-points for three ordinal diagnostic groups. Commun Stat Simul Comput 2013; 42: 1213–1234.

Luo

Xiong

. Diagtest3grp: an R package for analyzing diagnostic tests with three ordinal groups. J Stat Softw 2012; 51: 1.

10.

Tian

Xiong

Lai

et al.

Exact confidence interval estimation for the difference in diagnostic accuracy with three ordinal diagnostic groups. J Stat Plann Inference 2011; 141: 549–558.

11.

Tsui

Weerahandi

. Generalized p-values in significance testing of hypotheses in the presence of nuisance parameters. J Am Stat Assoc 1989; 84: 602–607.

12.

Weerahandi

. Generalized confidence intervals. J Am Stat Assoc 1993; 88: 899–905.

13.

Weerahandi

. Exact statistical methods for data analysis, New York: Springer Verlag, 2003. .

14.

Krishnamoorthy

. Inferences on the common mean of several normal populations based on the generalized variable method. Biometrics 2003; 59: 237–247.

15.

Weerahandi

Berger

. Exact inference for growth curves with intraclass correlation structure. Biometrics 1999; 55: 921–924.

16.

Yin

Tian

. Joint confidence region estimation for area under ROC curve and Youden index. Stat Med 2014; 33: 985–1000.

17.

Yin

Tian

. Joint inference about sensitivity and specificity at the optimal cut-off point associated with youden index. Comput Stat Data Anal 2014; 77: 1–13.

18.

Lin

Lee

. Generalized inferences on the common mean of several normal populations. J Stat Plann Inference 2005; 134: 568–582.

19.

Hanley

. The use of the 'binormal' model for parametric ROC analysis of quantitative diagnostic tests. Stat Med 1996; 30: 1585–1595.

20.

Fluss

Faraggi

Reiser

. Estimation of the Youden index and its associated cutoff point. Biometrical J 2005; 47: 458–472.

21.

Molodianovitch

Faraggi

Reiser

. Comparing the areas under two correlated roc curves: parametric and non-parametric approaches. Biometrical J 2006; 48: 745–757.

22.

Schisterman

Reiser

Faraggi

. ROC analysis for markers with mass at zero. Stat Med 2006; 25: 623–638.

23.

Bantis

Nakas

Reiser

et al.

Construction of joint confidence regions for the optimal true class fractions of receiver operating characteristic (ROC) surfaces and manifolds. Stat Methods Med Res 2017; 26: 1429–1442.

24.

Kang

Tian

. Estimation of the volume under the roc surface with three ordinal diagnostic categories. Comput Stat Data Anal 2013; 62: 39–51.

25.

Silverman

. Density estimation for statistics and data analysis, London: Chapman & Hall/CRC, 1986.

26.

Zou

Tempany

Fielding

et al.

Original smooth receiver operating characteristic curve estimation from continuous data: statistical methods for analyzing the predictive value of spiral CT of ureteral stones. Acad Radiol 1998; 5: 680–687.

27.

Nakas

Dalrymple-Alford

Anderson

et al.

Generalization of Youden index for multiple-class classification problems applied to the assessment of externally validated cognition in Parkinson disease screening. Stat Med 2013; 32: 995–1003.

28.

Bäckman

Jones

Berger

et al.

Multiple cognitive deficits during the transition to Alzheimer's disease. J Intern Med 2004; 256: 195–204.

29.

Greicius

Srivastava

Reiss

et al.

Default-mode network activity distinguishes Alzheimer's disease from healthy aging: evidence from functional MRI. Proc Natl Acad Sci U S A 2004; 101: 4637–4642.

30.

Hansson

Zetterberg

Buchhave

et al.

Association between CSF biomarkers and incipient Alzheimer's disease in patients with mild cognitive impairment: a follow-up study. Lancet Neurol 2006; 5: 228–234.

31.

Morris

. The clinical dementia rating (CDR): current version and scoring rules. Neurology 1993; 43: 2412–2414.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.09 MB

0.00 MB

0.06 MB

0.24 MB