Bootstrap-based testing approaches for the assessment of the diagnostic accuracy of biomarkers subject to a limit of detection

Abstract

Assessment of the diagnostic accuracy of biomarkers through receiver operating characteristic curve analysis frequently involves a limit of detection imposed by the laboratory analytical system precision. As a consequence, measurements below a certain level are undetectable and ignoring these is known to lead to negatively biased estimates of the area under the receiver operating characteristic curve. In this article, we introduce two receiver operating characteristic curve-based parametric approaches that tackle the issue of correct assessment of diagnostic markers in the presence of a limit of detection. Proposed approaches are simulation-based utilising bootstrap methodology. Non-parametric alternatives that are naively used in the literature do not solve the inherent problem of limit of detection values which are treated as censored observations. However, the latter seems to perform adequately well in our simulation study. Nonparametric bootstrap was consistently used throughout, while other bootstrap alternatives performed similarly in our pilot simulation study. The simulation study involves the comparison of parametric and non-parametric options described here versus alternative strategies that are routinely used in the literature. We apply all methods to a study-setting resembling a chemical quasi-standard situation, where compound tumour biomarkers were searched within a multi-variable set of measurements to discriminate between two groups, namely colorectal cancer and controls. We focus in the assessment of glutamine and methionine.

Keywords

Receiver operating characteristic curve analysis area under the receiver operating characteristic curve two-sample tests limit of detection colorectal cancer biomarkers

Get full access to this article

View all access options for this article.

References

Pepe

. The statistical evaluation of medical tests for classification and prediction, New York: Oxford University Press, 2003.

Zhou

Obuchowski

McClish

. Statistical methods in diagnostic medicine, New York: Wiley & Sons Interscience, 2011.

Perkins

Schisterman

Vexler

. Receiver operating characteristic curve inference from a sample with a limit of detection. Am J Epidemiol 2007; 165: 325–333.

Lubin

Colt

Camann

et al.

Epidemiological evaluation of measurement data in the presence of detection limits. Environ Health Perspect 2004; 112: 1691–1696.

Lyles

Williams

Chuachoowong

. Correlating two viral load assays with known detection limits. Biometrics 2001; 57: 1238–1244.

Mumford

Schisterman

Vexler

et al.

Pooling biospecimens and limits of detection: effects on ROC curve analysis. Biostatistics 2006; 7: 585–598.

Schisterman

Reiser

Faraggi

. ROC analysis for markers with mass at zero. Stat Med 2006; 25: 623–638.

Vexler

Liu

Eliseeva

et al.

Maximum likelihood ratio tests for comparing the discriminatory ability of biomarkers subject to limit of detection. Biometrics 2008; 64: 895–903.

Bantis

Yan

Tsimikas

et al.

Estimation of smooth ROC curves for biomarkers with limits of detection. Stat Med 2017; 36: 3830–3843.

10.

Bloch

. Comparing two diagnostic tests against the same “gold standard” in the same sample. Biometrics 1997; 53: 73–85.

11.

Molodianovitch

Faraggi

Reiser

. Comparing the areas under two correlated ROC curves: parametric and non-parametric approaches. Biometric J 2006; 48: 745–757.

12.

Leichtle

Nuoffer

Ceglarek

et al.

Serum amino acid profiles and their alterations in colorectal cancer. Metabolomics 2012; 8: 643–653.

13.

Halligan

Altman

Mallett

. Disadvantages of using the area under the receiver operating characteristic curve to assess imaging tests: a discussion and proposal for an alternative approach. Eur Radiol 2015; 25: 932–939.

14.

Nakas

Yiannoutsos

Bosch

et al.

Assessment of diagnostic markers by goodness-of-fit tests. Stat Med 2003; 22: 2503–2513.

15.

Pardo

Franco-Pereira

. Non parametric ROC summary statistics. REVSTAT 2017; 15: 583–600.

16.

Klein

Moeschberger

. Survival analysis: techniques for censored and truncated data, New York: Springer, 2003.

17.

Gupta

. Estimation of the mean and standard deviation of a normal population from a censored sample. Biometrika 1952; 39: 260–73.

18.

Wieand

Gail

James

et al.

A family of nonparametric statistics for comparing diagnostic markers with paired or unpaired data. Biometrika 1989; 76: 585–592.

19.

Metz CE, Wang PL and Kronman HB. A new approach for testing the significance of differences between ROC curves measured from correlated data. In: Deconinck F (ed.) Information processing in medical imaging. The Hague: Nijhoff, p.43

20.

Martínez-Camblor

. Comparing k-independent and right censored samples based on the likelihood ratio. Comput Stat 2010; 25: 363–374.

21.

Gehan

. A generalized Wilcoxon test for comparing arbitrarily singly-censored samples. Biometrika 1965; 52: 203–223.

22.

Venkatraman

Begg

. A distribution-free procedure for comparing receiver operating characteristic curves from a paired experiment. Biometrika 1996; 83: 835–848.

23.

Senn

Holford

Hockey

. The ghosts of departed quantities: approaches to dealing with observations below the limit of quantitation. Stat Med 2012; 31: 4280–4295.

24.

Longford

. Handling the limit of detection by extrapolation. Stat Med 2012; 31: 3133–3146.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.11 MB

0.00 MB

0.21 MB