Construction,visualization and application of neutral zone classifiers

Abstract

When the potential for making accurate classifications with a statistical classifier is limited, a neutral zone classifier can be constructed by adding a no-decision option as a classification outcome. We show how a neutral zone classifier can be constructed from a receiving operating characteristic (ROC) curve. We extend the ROC curve graphic to highlight important performance characteristics of a neutral zone classifier. Additional utility of neutral zone classifiers is illustrated by showing how they can be incorporated into the first stage of a two-stage classification process. At the first stage, a classification is attempted from easily collected or inexpensive features. If the classification falls into the neutral zone, additional relatively more expensive features can be obtained and used to make a definitive classification at the second stage. The methods discussed in the paper are illustrated with an application pertaining to prostate cancer.

Keywords

Statistical classification logistic regression neutral zones ROC curve prostate cancer

Get full access to this article

View all access options for this article.

References

James

Witten

Hastie

, et al. An introduction to statistical learning with applications in R, New York: Springer, 2013.

Jeske

. Metrics used when evaluating the performance of statistical classifiers. Biostat Biom Open Access J 2018; 8: 555728.

Rao

. The problem of classification and distance between two populations. Nature 1947; 18: 30–31.

Habbema

JDF

Hermans

Van Der Brugt

. Cases of doubt in allocation problems. Biometrika 1947; 61: 313–324.

Jeske

Ruegger

, et al. A three-class neutral classifier using a decision-theoretic approach with application to DNA array analysis. J Agric Biol Environ Stat 2010; 15: 474–490.

Benecke

Jeske

Reugger

, et al. Bayes neutral zone classifiers with applications to nonparametric unsupervised settings. J Agric Biol Environ Stat 2013; 18: 39–52.

Swartz RJ. Applications of Bayesian sequential decision-theory to medical decision-making. PhD Dissertation, Rice University, 2002.

Chow

. An optimum character recognition system using decision functions. IEEE Trans Comput 1957; 4: 247–254.

Anderson

. Constrained discrimination between k populations. J R Stat Soc Series 1969; 31: 123–139.

10.

Jeske

Smith

. Maximizing the usefulness of statistical classifiers for two populations with illustrative applications. Stat Methods Med Res 2018; 27: 2344–2358.

11.

Pepe MS. The statistical evaluation of medical tests for classification and prediction. Oxford Statistical Science Series 28. Oxford: Oxford University Press, 2003.

12.

Zhou X-H, Obuchowski NA and McClish DK. Statistical methods in diagnostic medicine. New York: Wiley, 2002.

13.

Zou KH, Liu A, Bandos AI, et al. Statistical evaluation of diagnostic performance: topics in ROC analysis. Boca Raton: CRC Press, 2011.

14.

Saito

Rehmsmeier

. The precision-recall plot is more informative than the ROC plot when evaluating binary classifiers on imbalanced data sets. PLoS One 2015; 10: e0118432.

15.

Davis J and Goadrich M. The relationship between precision-recall and ROC curves. In: Proceedings of the 23rd international conference on machine learning, 2006, pp. 232–240. New York: Association for Computing Machinery.

16.

Shiu

S-Y

Gatsonis

. The predictive receiver operating characteristic curve for the joint assessment of the positive and negative predictive values. Philos Trans A Math Phys Eng Sci 2008; 366: 2313–2333.

17.

Jeske

Linehan

, et al. Two-stage classifiers that minimize PCA3 and the PSA proteolytic activity testing in the prediction of prostate cancer recurrence after radical prostatectomy. Can J Urol 2017; 24: 9089–9097.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.34 MB