Sensitivity and Specificity of Machine Learning Classifiers and Spectral Domain OCT for the Diagnosis of Glaucoma

Abstract

Purpose.

To investigate the sensitivity and specificity of machine learning classifiers (MLC) and spectral domain optical coherence tomography (SD-OCT) for the diagnosis of glaucoma.

Methods.

Sixty-two patients with early to moderate glaucomatous visual field damage and 48 healthy individuals were included. All subjects underwent a complete ophthalmologic examination, achromatic standard automated perimetry, and RNFL imaging with SD-OCT (Cirrus HD-OCT; Carl Zeiss Meditec, Inc., Dublin, California, USA). Receiver operating characteristic (ROC) curves were obtained for all SD-OCT parameters. Subsequently, the following MLCs were tested: Classification Tree (CTREE), Random Forest (RAN), Bagging (BAG), AdaBoost M1 (ADA), Ensemble Selection (ENS), Multilayer Perceptron (MLP), Radial Basis Function (RBF), Naive-Bayes (NB), and Support Vector Machine (SVM). Areas under the ROC curves (aROCs) obtained for each parameter and each MLC were compared.

Results.

The mean age was 57.0±9.2 years for healthy individuals and 59.9±9.0 years for glaucoma patients (p=0.103). Mean deviation values were −4.1±2.4 dB for glaucoma patients and −1.5±1.6 dB for healthy individuals (p<0.001). The SD-OCT parameters with the greater aROCs were inferior quadrant (0.813), average thickness (0.807), 7 o'clock position (0.765), and 6 o'clock position (0.754). The aROCs from classifiers varied from 0.785 (ADA) to 0.818 (BAG). The aROC obtained with BAG was not significantly different from the aROC obtained with the best single SD-OCT parameter (p=0.93).

Conclusions.

The SD-OCT showed good diagnostic accuracy in a group of patients with early glaucoma. In this series, MLCs did not improve the sensitivity and specificity of SD-OCT for the diagnosis of glaucoma.

Keywords

Machine learning classifiers Optical coherence tomography Sensitivity Specificity Spectral domain

Get full access to this article

View all access options for this article.

References

Sommer

, Katz

, Quigley

H.A.

, Clinically detectable nerve fiber atrophy precedes the onset of glaucomatous field loss. Arch Ophthalmol 1991; 109: 77–83.

Johnson

C.A.

, Cioffi

G.A.

, Liebmann

J.R.

, Sample

P.A.

, Zangwill

L.M.

, Weinreb

R.N.

. The relationship between structural and functional alterations in glaucoma: a review. Semin Ophthalmol 2000; 15: 221–33.

Huang

, Swanson

E.A.

, Lin

C.P.

, Optical coherence tomography. Science 1991; 254: 1178–81.

Schuman

J.S.

, Hee

M.R.

, Arya

A.V.

, Optical coherence tomography: a new tool for glaucoma diagnosis. Curr Opin Ophthalmol 1995; 6: 89–95.

Schuman

J.S.

, Hee

M.R.

, Puliafito

C.A.

, Quantification of nerve fiber layer thickness in normal and glaucomatous eyes using optical coherence tomography. Arch Ophthalmol 1995; 113: 586–96.

Harwerth

R.S.

, Vilupuru

A.S.

, Rangaswamy

N.V.

, Smith

III . The relationship between nerve fiber layer and perimetry measurements. Invest Ophthalmol Vis Sci 2007; 48: 763–73.

Zangwill

L.M.

, Bowd

, Berry

C.C.

, Discriminating between normal and glaucomatous eyes using the Heidelberg retina tomograph, GDx nerve fiber analyzer, and optical coherence tomograph. Arch Ophthalmol 2001; 119: 985–93.

Guedes

, Schuman

J.S.

, Hertzmark

, Optical coherence tomography measurement of macular and nerve fiber layer thickness in normal and glaucomatous human eyes. Ophthalmology 2003; 110: 177–89.

Chang

R.T.

, Knight

O.J.

, Feuer

W.J.

, Budenz

D.L.

. Sensitivity and specificity of time-domain versus spectral-domain optical coherence tomography in diagnosing early to moderate glaucoma. Ophthalmology 2009; 116: 2294–9.

10.

Jeoung

J.W.

, Park

K.H.

. Comparison of Cirrus OCT and Stratus OCT on the ability to detect localized retinal nerve fiber layer defects in preperimetric glaucoma. Invest Ophthalmol Vis Sci 2010; 51: 938–45.

11.

Park

S.B.

, Sung

K.R.

, Kang

S.Y.

, Kim

K.R.

, Kook

M.S.

. Comparison of glaucoma diagnostic Capabilities of Cirrus HD and Stratus optical coherence tomography. Arch Ophthalmol 2009; 127: 1603–9.

12.

Moreno-Montañés

, Olmo

, Alvarez

, García

, Zarranz-Ventura

. Cirrus high-definition optical coherence tomography compared with Stratus optical coherence tomography in glaucoma diagnosis. Invest Ophthalmol Vis Sci 2010; 51: 335–43.

13.

Knight

O.J.

, Chang

R.T.

, Feuer

W.J.

, Budenz

D.L.

. Comparison of retinal nerve fiber layer measurements using time domain and spectral domain optical coherent tomography. Ophthalmology 2009; 116: 1271–7.

14.

Bizios

, Heijl

, Bengtsson

. Trained artificial neural network for glaucoma diagnosis using visual field data: a comparison with conventional algorithms. J Glaucoma 2007; 16: 20–8.

15.

Burgansky-Eliash

, Wollstein

, Chu

, Optical coherence tomography machine learning classifiers for glaucoma detection: a preliminary study. Invest Ophthalmol Vis Sci 2005; 46: 4147–52.

16.

Bowd

, Medeiros

F.A.

, Zhang

Relevance vector machine and support vector machine classifier analysis of scanning laser polarimetry retinal nerve fiber layer measurements. Invest Ophthalmol Vis Sci 2005; 46: 1322–9.

17.

Bizios

, Heijl

, Hougaard

J.L.

, Bengtsson

. Machine learning classifiers for glaucoma diagnosis based on classification of retinal nerve fibre layer thickness parameters measured by Stratus OCT. Acta Ophthalmol 2010; 88: 44–52.

18.

Townsend

K.A.

, Wollstein

, Danks

, Heidelberg Retina Tomograph 3 machine learning classifiers for glaucoma detection. Br J Ophthalmol 2008; 92: 814–8.

19.

Bowd

, Hao

, Tavares

I.M.

, Bayesian machine learning classifiers for combining structural and functional measurements to classify healthy and glaucomatous eyes. Invest Ophthalmol Vis Sci 2008; 49: 945–53.

20.

Horn

F.K.

, Lämmer

, Mardin

C.Y.

, Combined evaluation of frequency doubling technology perimetry and scanning laser ophthalmoscopy for glaucoma detection using automated classification. J Glaucoma 2012; 21: 27–34.

21.

Boden

, Chan

, Sample

P.A.

, Assessing visual field clustering schemes using machine learning classifiers in standard perimetry. Invest Ophthalmol Vis Sci 2007; 48: 5582–90.

22.

Mardin

C.Y.

, Peters

, Horn

, Jünemann

A.G.

, Lausen

. Improving glaucoma diagnosis by the combination of perimetry and HRT measurements. J Glaucoma 2006; 15: 299–305.

23.

Huang

M.L.

, Chen

H.Y.

. Development and comparison of automated classifiers for glaucoma diagnosis using Stratus optical coherence tomography. Invest Ophthalmol Vis Sci 2005; 46: 4121–9.

24.

Horn

F.K.

, Brenning

, Jünemann

A.G.

, Lausen

. Glaucoma detection with frequency doubling perimetry and short-wavelength perimetry. J Glaucoma 2007; 16: 363–71.

25.

Tucker

, Vinciotti

, Liu

, Garway-Heath

. A spatio-temporal Bayesian network classifier for understanding visual field deterioration. Artif Intell Med 2005; 34: 163–77.

26.

Mitchell

T.M.

. Machine Learning. Singapore: McGraw-Hill, 1997; 81–126; 154–99.

27.

Quinlan

J.R.

. Programs for Machine Learning. San Mateo: Morgan Kaufmann, 1993; 17–25.

28.

Quinlan

J.R.

. Induction of decision trees. Mach Learn 1986; 1: 81–106.

29.

Breiman

. Random forests. Mach Learn 2001; 45: 5–32.

30.

Breiman

. Bagging predictors. Mach Learn 1996; 24: 123–40.

31.

Haykin

. Neural Networks: A Comprehensive Foundation. Upper Saddle River, NJ: Prentice Hall, Inc., 1999; 183–282.

32.

Keerthi

S.S.

, Shevade

S.K.

, Bhattacharyya

, Improvements to Platt's SMO algorithm for SVM classifier design. Neural Comput 2001; 13: 637–49.

33.

Burges

C.J.C.

. A tutorial on support vector machines for pattern recognition. Data Mining Knowledge Discovery 1998; 2: 121–67.

34.

Witten

I.H.

, Frank

. Data Mining: Practical Machine Learning Tools and Techniques, 2nd ed. San Francisco: Morgan Kaufmann, 2005; 365–8.