Abstract
Abstract
Purpose:
To investigate the effectiveness of gradient boosting to classify endophthalmitis versus uveitis and lymphoma by intraocular cytokine levels.
Method:
Patient diagnoses and aqueous and vitreous levels of interleukin (IL)-6 and IL-10 were retrospectively extracted from a National Eye Institute Histopathology Core database and compared by Kruskal–Wallis and post hoc Dunn tests. A gradient-boosted decision tree classifier was trained to differentiate endophthalmitis versus uveitis and lymphoma from vitreous IL-6 and IL-10, vitreous IL-6 only, and aqueous IL-6 only data sets; and was tested with 80-20 train-test split and 3-fold cross-validation of the training set.
Results:
Seven endophthalmitis, 29 lymphoma, and 49 uveitis patients were included. IL-6 was higher in endophthalmitis than uveitis (P = 0.0713 aqueous, 0.0014 vitreous) and lymphoma (P = 0.0032 aqueous, 0.0001 vitreous). IL-10 was significantly higher in lymphoma than uveitis (P = 0.0017 aqueous, 0.0014 vitreous). Three-fold cross validation demonstrated 95% ± 5%, 95% ± 4%, and 97% ± 5% predictive accuracy for vitreous IL-6 and IL-10, vitreous IL-6 only, and aqueous IL-6 only data sets. Upon validation with the testing set, vitreous IL-6 and IL-10 and aqueous IL-6 only data sets achieved 100% predictive accuracy and vitreous IL-6 only data achieved 93% predictive accuracy with 100% sensitivity, 92% specificity, and an area under the receiver operating characteristic curve (ROC/AUC) of 96%.
Conclusions:
With limited sample size, gradient boosting can differentiate endophthalmitis from uveitis and lymphoma by IL-6 and IL-10 with high sensitivity and specificity; however, a larger cohort is needed for further validation.
Introduction
V
The gold standard for diagnosis of intraocular lymphoma is cytology; however, it has a low detection rate (44.5% in one large Japanese study) as lymphoma cells are rapidly necrotic after they leave the eye and are frequently infiltrated with large numbers of reactive immune cells. 2 As a result, adjunctive diagnostic tests including flow cytometry and immunohistochemistry for monoclonality, polymerase chain reaction (PCR) analysis for IgH or T-cell receptor rearrangements, and intraocular cytokine analysis are used to improve diagnostic yield.3–6 Several studies have found that vitreous interleukin (IL)-10 is elevated in lymphoma, while vitreous IL-6 is elevated in uveitis and endophthalmitis.7,8 Furthermore, an IL-10/IL-6 ratio >1 has been shown to be sensitive for lymphoma versus an IL-10/IL-6 ratio <1, which is more indicative of uveitis.2,4,9
For suspected endophthalmitis, workup and management consist of a vitreous tap and empiric intravitreal broad-spectrum antibiotics. 1 Unfortunately, vitreous Gram stain is poorly sensitive, positive in only 7.5%–36% of endophthalmitis patients.10–14 Vitreous culture is better but still relatively insensitive, being positive in roughly 70% of endophthalmitis patients. 15 PCR has emerged as a promising complementary diagnostic test with one recent study reporting a positive predictive value of 99% and a negative predictive value of 93%; but further validation is required. 14 Additionally, availability of the latest PCR assays remains limited in the United States and worldwide. As in uveitis, endophthalmitis stimulates a strong inflammatory response within the eye and is associated with high levels of proinflammatory cytokines such as IL-6.16–19 However, while cytokine analysis has been extensively studied in differentiating lymphoma from uveitis, its utility has not been investigated for differentiating endophthalmitis from uveitis or lymphoma. In this study, we demonstrate that intraocular IL-6 and IL-10 levels can help differentiate endophthalmitis from uveitis and lymphoma using machine learning, specifically gradient boosting.
Methods
Data extraction
All patients in the study were enrolled in IRB approved clinical protocols at the National Eye Institute, and informed consent was obtained from all patients. Patient entries with aqueous or vitreous (undiluted and/or diluted samples) IL-6 or IL-10 levels were extracted retrospectively from a National Eye Institute (NEI) Histopathology Core database using regular expressions in the Python pandas library. The NIH electronic medical record was then queried for associated clinical records to establish definitive diagnoses of endophthalmitis, uveitis, or lymphoma. To focus on the initial diagnosis of endophthalmitis versus uveitis/lymphoma, only the first cytokine measurements associated with each patient were included in the analysis.
Statistical analysis
Aqueous and vitreous IL-6 and IL-10 levels between disease groups (endophthalmitis, uveitis, and lymphoma) were compared by the nonparametric Kruskal–Wallis test and post hoc Dunn test due to non-normal distributions and unequal variances between groups. The nonparametric Mann–Whitney U Test was performed to compare differences between aqueous and vitreous IL-6 and IL-10 levels for each group to assess whether aqueous and vitreous cytokine levels could be pooled for machine learning classification. All statistical analyses and plots were done in R and in GraphPad (GraphPad Software, Inc., La Jolla, CA). A 2-tailed P value of less than 0.05 was considered statistically significant.
Machine learning classifier
A gradient-boosted decision tree classifier was implemented using the Python scikit-learn library to classify endophthalmitis versus uveitis/lymphoma from aqueous and vitreous IL-6 and IL-10 levels and validated with 80/20 train-test split and 3-fold cross validation within the training set. Receiver operating characteristic (ROC) curves were generated for the testing set of each data set to assess the sensitivity and specificity of the machine learning classifier.
Results
Study population
A total of 2339 patient entries from October 5, 1999 to September 16, 2015 were extracted from the NEI ocular immunopathology database. Five hundred thirty-nine entries (corresponding to 386 patients) reported IL-6 levels and 391 entries (corresponding to 296 patients) reported both IL-6 and IL-10 levels. All patient entries with IL-10 levels also reported IL-6 levels. Of these entries, 185 (corresponding to 116 patients) were associated with patients in the NIH electronic medical record and 158 (corresponding to 87 patients) were associated with a definitive diagnosis of endophthalmitis, uveitis, or lymphoma. The 85 patients included in this study include 7 endophthalmitis patients, 29 lymphoma patients, and 49 uveitis patients (11 anterior, 10 intermediate, 15 panuveitis, and 13 posterior uveitis). Forty-nine had vitreous specimens taken and 36 had aqueous specimens taken (Table 1).
IL, interleukin.
Statistical analysis
IL-6 levels between diagnoses
IL-6 was higher in endophthalmitis compared to uveitis, though only significantly so in vitreous samples (P = 0.0713 aqueous, 0.0014 vitreous), and was significantly higher in endophthalmitis compared to lymphoma (P = 0.0032 aqueous, 0.0001 vitreous). IL-6 levels were not significantly different between uveitis and lymphoma (P = 0.069 aqueous, 0.239 vitreous) (Fig. 1A, C).
Aqueous and vitreous IL-6 and IL-10 levels between diagnoses. Comparisons between endophthalmitis, uveitis, and lymphoma patients are shown for
IL-10 level between diagnoses
IL-10 levels were significantly elevated in lymphoma compared to uveitis in both aqueous (P = 0.0017) and vitreous (P = 0.0014) samples. However, vitreous IL-10 levels in endophthalmitis were not significantly different from uveitis (P = 0.1854) or lymphoma (P = 0.8679). There were no aqueous IL-10 levels measured in endophthalmitis patients (Fig. 1B, D).
Vitreous IL-10/IL-6 ratio between diagnoses
Because vitreous samples included both diluted and undiluted specimens, IL-10/IL-6 ratio, which normalizes between diluted and undiluted specimens, was also compared between diagnoses. IL-10/IL-6 ratio was significantly elevated in lymphoma compared to uveitis (P = 0.0166) and endophthalmitis (P = 0.0018), and also significantly higher in uveitis compared to endophthalmitis (P = 0.0459).
Aqueous versus vitreous cytokine levels
Vitreous and aqueous cytokine levels were not significantly different except in the case of aqueous versus vitreous IL-6 levels in uveitis patients where aqueous IL-6 levels were significantly higher than vitreous IL-6 levels (P = 0.0004) (Fig. 2A–E). Of note, aqueous IL-6 was markedly higher than vitreous IL-6 in cases of anterior uveitis, while aqueous and vitreous IL-6 were roughly similar for intermediate, posterior, and panuveitis, possibly due to the anteriorly directed flow of aqueous humor in the eye. Because of the potential bias that this could introduce, we did not pool aqueous and vitreous cytokine data but rather trained separate machine learning classifiers for aqueous and vitreous cytokine data.
Aqueous versus vitreous IL-6 and IL-10 levels for each diagnosis. Comparisons between aqueous and vitreous cytokine levels are shown for
Machine learning classifier
The gradient-boosted decision tree classifier demonstrated 95% ± 5%, 95% ± 4%, and 97% ± 5% accuracy, respectively, on 3-fold cross validation of the training set with vitreous IL-6 and IL-10 data, vitreous IL-6 data alone, and aqueous IL-6 data alone. In the testing set, vitreous IL-6 and IL-10, and aqueous IL-6 only data sets achieved 100% predictive accuracy, while vitreous IL-6 only data achieved 93% predictive accuracy with 100% sensitivity, 92% specificity, and an area under the receiver operating characteristic curve of 0.96. A scatter plot of vitreous IL-6 and IL-10 data is shown with the gradient-boosted decision tree classifier's decision boundary in Fig. 3. ROC curves for classification of each data set are shown in Fig. 4.
Gradient boosting decision boundary for vitreous IL-6 and IL-10 data.
Receiver operating characteristic curves of
Discussion
In this study, we found a near significant to significant difference in aqueous and vitreous IL-6 levels between endophthalmitis and uveitis and between endophthalmitis and lymphoma. However, there was no significant difference in vitreous IL-10 between endophthalmitis and uveitis or between endophthalmitis and lymphoma. Uveitis and lymphoma had similar aqueous and vitreous IL-6 but had significantly different aqueous and vitreous IL-10 levels. Normalizing diluted and undiluted vitreous measurements by IL-10/IL-6 ratio demonstrated a significant difference between lymphoma, uveitis, and endophthalmitis, with IL-10/IL-6 ratio highest in lymphoma and lowest in endophthalmitis. These findings corroborate previous reports that intraocular IL-10 and IL-10/IL-6 ratio can aid in the differentiation between lymphoma and uveitis and suggest that IL-6 can be used to differentiate endophthalmitis from uveitis and lymphoma. We then demonstrated that a gradient-boosted decision tree classifier can differentiate endophthalmitis from uveitis and lymphoma with high sensitivity and specificity using vitreous IL-6 and IL-10 data, vitreous IL-6 data alone, and aqueous IL-6 data alone.
This study has several limitations, the most significant of which are its retrospective nature and small sample size due to the low incidence of endophthalmitis requiring diagnostic vitrectomy. There was a significant dropout of patients from inclusion in the study due to lack of corresponding electronic medical records. This is most likely because the NEI Histopathology Core has historically processed samples from both intramural and extramural patients. Further dropout occurred due to lack of definitive diagnoses recorded in the medical record. Because of the small sample size, study populations were not normally distributed and had unequal variances, necessitating nonparametric statistical testing. Small sample size may also have contributed to the 100% predictive accuracy of gradient boosting in differentiating endophthalmitis from uveitis and lymphoma in the vitreous IL-6 and IL-10 data and aqueous IL-6 only data sets. More data are needed to further validate these results.
Despite these limitations, this study suggests that IL-10 and IL-6 can indeed be used to differentiate endophthalmitis, uveitis, and lymphoma, and highlights machine learning classification with gradient boosting based on intraocular cytokine data as a promising approach. Additionally, marked elevation of IL-6 in ocular fluid may suggest an infectious etiology. As machine learning usually becomes more accurate with more data, reliable classification of endophthalmitis, uveitis, and lymphoma by intraocular cytokine analysis may be achievable. Thus, with more data to improve predictive accuracy; and with continuing advances in laboratory techniques for cytokine analysis that enable greater sensitivity and specificity with smaller sample size requirements, intraocular cytokine analysis may become a valuable diagnostic test for ophthalmologists caring for patients with ocular inflammation of unclear etiology. Further studies with larger cohorts are recommended.
Footnotes
Acknowledgments
This research was made possible through the National Eye Institute Intramural Research Program, and the National Institutes of Health (NIH) Medical Research Scholars Program, a public-private partnership supported jointly by the NIH and generous contributions to the Foundation for the NIH from the Doris Duke Charitable Foundation, American Association for Dental Research, Howard Hughes Medical Institute, and the Colgate-Palmolive Company, as well as alumni of student research programs and other individual supporters.
Author Disclosure Statement
No competing financial interests exist.