A limitation of the diagnostic-odds ratio in determining an optimal cut-off value for a continuous diagnostic test

Abstract

The article considers the diagnostic odds ratio, a special summarising function of specificity and sensitivity for a given diagnostic test, which has been suggested as a measure of diagnostic discriminatory power. In the situation of a continuous diagnostic test a cut-off value has to be chosen and it is a common practice to choose the cut-off value on the basis of the maximised diagnostic odds ratio. We show that this strategy is not to be recommended since it might easily lead to cut-off values on the boundary of the parameter range. This is illustrated by means of some examples. The source of the deficient behaviour of the diagnostic odds ratio lies in the convexity of the log-diagnostic odds ratio as a function of the cut-off value. This can easily be seen in practice by plotting a non-parametric estimate of the log-DOR against the cut-off value. In fact, it is shown for the case of a normal distributed diseased and a normal distributed non-diseased population with equal variances that the log-DOR is a convex function of the cut-off value. It is also shown that these problems are not present for the Youden index, which appears to be a better choice.

Get full access to this article

View all access options for this article.

References

Youden D. Index for rating diagnostic tests. Cancer 1950; 3: 32-5.

Pepe MS The statistical evaluation of medical tests for classification and prediction . University Press, Oxford; 2003.

Greiner M. Serodiagnostische tests. Springer, Berlin; 2003.

Le CT A solution for the most basic optimisation problem associated with an ROC curve. Statistical Methods in Medical Research 2006; 15: 571-84.

Glas AS , Lijmer JG , Prins MH , Bonsel GJ , Bossuyt PMM. The diagnostic odds ratio: a single indicator of test performance . Journal of Clinical Epidemiology 2003; 56: 1129-35.

Leeflang MMG , Moons KGM , Reitsma JB , Zwinderman AH Bias in sensitivity and specificity caused by data-driven selection of optimal cutoff values: mechanisms, magnitude, and solutions. Clinical Chemistry 2008; 54: 729-37.

Greiner M. , Pfeiffer DU , Smith, RD Principles and practical applications of the receiver operating characteristic (ROC) for diagnostic test. Preventive Veterinary Medicine 2000; 45: 23-41.

Magder LS , Fix AD Optimal choice of a cut point for a quantitative diagnostic test performed for research purposes. Journal of Clinical Epidemiology 2003; 56: 956-62.

Lotrakul M. , Sumrithe S. , Saipanish R. Reliability and validity of the Thai version of the PHQ-9. BMC Psychiatry 2008; 8: 46.

10.

Fischer JE , Bachmann LM , Jaeschke R. A readers’ guide to the interpretation of diagnostic test properties: clinical example of sepsis. Intensive Care Medicine 2003; 29: 1043-51.

11.

Wei LH , Chiu JS , Chang SY , Wang YF Predicting bone metastasis in prostate cancer patients: value of prostate specific antigen. Tzu Chi Medical Journal 2008; 20: 291-95.

12.

Brasil P. The DiagnosisMed Package. Available at: http://cran.r-project.org/web/packages/DiagnosisMed/DiagnosisMed.pdf , (accessed 9 March 2010).

13.

Schisterman EF , Perkins N. Confidence intervals for the Youden index and corresponding optimal cut-point. Communications in Statistics - Simulation and Computation 2007; 36: 549-63.

14.

Perkins N. , Schisterman EF The Youden index and the optimal cut-point corrected for measurement error . Biometrical Journal 2005; 47: 428-41.

15.

Fluss R. , Faraggi D. , Reiser B. Estimation of the Youden index and its associated cutoff point. Biometrical Journal 2005; 47: 458-72.

16.

Norton SJ , Gorga MP , Widen JE , et al. Identification of neonatal hearing impairment: Evaluation of transient evoked optoacoustic emission, distortion product optoacoustic emission, and auditory brain stem response test performance. Ear and Hearing 2000; 21: 508-28.

17.

Pepe MS , Janes H. , Longton G. , Leisenring W. , Newcomb P. Limitations of the odds ratio in gauging the performance of a diagnostic, prognostic, or screening marker. American Journal of Epidemiology 2004; 159: 882-90.

18.

Hasselblad V. , Hedges LV Meta-analysis of screening and diagnostic tests. Psychological Bulletin 1995 ; 117: 167-78.

19.

Edwards JH Some taxonomic implications of a curious feature of the bivariate normal surface . British Journal of Prevention and Social Medicine 1966; 20: 42.

20.

Deeks JJ Systematic reviews of evaluations of diagnostic and screening tests . British Medical Journal 2001; 323: 157-62.

21.

Hunink M. , Glasziou P. , Siegel J. , et al. Decision making in health and medicine - integrating evidence and values. University Press, Cambridge ; 2001.

22.

Hand DJ , Krzanowski W. ROC curves for continuous data. CRC Press, Boca Raton; 2009.