Reliability of Stereoscopic Nonmydriatic Retinography for Assessment of Diabetic Macular Edema when Performed by Endocrinologists

Abstract

Background:

Diabetic macular edema (DME) is the main cause of moderate vision loss in type 2 diabetes. Diagnosis is achieved by dilated fundus examination or by measuring retinal thickness. However, it can be identified in nonmydriatic retinography (NMR) with hard exudates as a surrogate marker or macular thickness under stereoscopic vision. To date, few studies have focused on interobserver reliability for DME with this technique.

Methods:

Fity-three type 2 diabetes patients with known diabetic retinopathy were studied. We obtained 182 pairs of stereoscopic retinographs with a nonmydriatic camera. Photographic options were 30° or 45° macula-centered retinal field and spontaneous or pharmacological dilation using tropicamide. An endocrinologist with a minimum of training and another with no specific training in this kind of examination diagnosed the images. DME was assumed if retinal thickness was identified within one disc diameter around the fovea.

Results:

The κ index agreement between both endocrinologists for all the data was 0.16 (P = 0.02). Depending on the photographic options, all the κ indices were below 0.25, except for the 45° retinal field under spontaneous mydriasis (0.58) where the number of samples analyzed was reduced to 22.

Conclusions:

In our study, endocrinologists with a low level of training did not reach a suitable level of agreement regarding the reliability of stereoscopic NMR as a technique for diagnosing DME. We feel that, as NMR can be performed by various different health providers, it would be advisable to establish generally agreed upon criteria for training staff in this technique.

Introduction

G iven the relevance and magnitude of the problem in public health, diabetic retinopathy (DR) is an issue of great importance in clinical research; it is the main cause¹ of irreversible loss of vision in developed countries among adults of between 20 and 74 years of age. In the case of type 2 diabetes mellitus (T2DM), the lesion responsible for this is generally the macular edema.² Up to 21% of patients with T2DM already suffer from DR at the moment when diabetes is diagnosed.³

Diabetic macular edema (DME) is the macular fluid collection caused by tissue and vascular injuries due to diabetes, usually accompanied by hard exudates. Based on the heterogeneity of the studies published regarding not only the population under study but also the diagnostic method used, clinically significant macular edema (CSME) prevalence values between 2% and 10% were reported in international studies.⁴ The incidence of macular edema found in the regional population on which this work focuses, as diagnosed by biomicroscopy, previously stood at 16%.⁵

Patients are usually observed to be asymptomatic during the initial phases of retinopathy, as long as the macula remains unaltered. However, in advanced stages where the macula is affected, quality of life becomes considerably worse, and we know that this results in an increase in health costs. Morbidity caused by macular injuries is mainly represented by clumsiness and trauma in daily life and worse results in work. The economic burden of partial visual lost due to DME was previously calculated to be $3,150 U.S. per year.⁶

The Early Treatment Diabetic Retinopathy Study proved the effectiveness of focal photocoagulation in diabetic macular edema; loss of moderate vision was reduced by 50%, dropping from 24% to 12% during the following 3 years.⁷ Until now, the main reason for implementing screening programs was to be able to detect vision-threatening retinopathy early in the follow-up, given the proven effectiveness of laser treatment. However, nowadays it is becoming even more important to detect DR in its early stages as new therapeutic strategies are currently being developed for that moment.

Various scientific organizations have made recommendations regarding the frequency and methods to be used for DR screening. The American Diabetes Association suggests that a complete first evaluation should be made by an ophthalmologist with mydriasis, when T2DM is diagnosed, and should then be repeated annually; moreover, it is proposed that specialists interpret fundus photographs, with or without mydriasis, although direct examination of the patient must be kept up for the follow-up of the detected lesions and also when the quality of the images is poor.¹ Biomicroscopy and stereoscopic mydriatic retinography are generally considered to be the most suitable diagnostic tests for macular edema as they enable us to detect retinal thickening and hard exudates.^8,9 Currently the optic systems of optical coherence tomography (OCT), retinal thickness analysis, and Heidelberg retinal tomography provide us with promising results; that is why they are already in use in specialized healthcare services.^10,11 On the other hand, the nonmydriatic retinography (NMR) technique has already proved its usefulness in DR screening, even when it is performed by unskilled health providers and interpreted by doctors who are not ophthalmologists.^12,13 Its main drawback is the proportion of nondiagnostic images due to their poor quality, because of the advanced age of the patient, the media opacity, the experience of the person handling the equipment and the technical characteristics of the system.^14,15 The staff performing the retinography must receive a minimum of training, but no clinical guidelines have yet been established in this regard. It has been generally agreed, however, that the degree of technical skill needed to obtain images of a sufficient quality in NMR is less than what is needed for performing mydriatic retinography.

Fewer studies have been published for evaluating NMR in DME than in DR. Most of the data come from DR screening studies where DME is also observed, although they were not specifically designed to detect this lesion, and, as a result, the number of affected eyes is normally quite low. With standard 35-mm retinography being the reference test, the identification of hard exudates for diagnosing DME in NMR has provided us with values that are somewhat erratic regarding sensitivity: 73.3–91.7%. However, in that study specificity was high: 90.7–91.5%.¹⁶ When indirect ophthalmoscopy is taken as the reference test, the results are 43% sensitivity and 100% specificity.¹⁷ In order to obtain stereoscopic images of a suitable quality with which we can then identify macular thickening, pharmacological mydriasis should be used. However, taking into account the advantages NMR offers us in other aspects, stereoscopic vision without mydriasis has also been evaluated; using the telemedicine system from the Joslin Vision Network and taking the standard retinographs as reference test, the following data regarding DME were obtained: 62% sensitivity, 95% specificity, 62% positive predictive value, and 95% negative predictive value. Data regarding CSME gave 27% sensitivity and 98% specificity. The proportion of eyes with DME, according to the standard retinographs, was 16% of the sample.¹⁸

There are data about interobserver agreement with nonstereoscopic images that show relatively low values when they have been interpreted by health providers who are not ophthalmologists specialized in the retina.¹⁹ The data published for stereoscopic images achieve quite a high degree of agreement for CSME.¹⁸ In both cases the study was not designed for detecting DME as a main variable, but rather as one more of multiple items to be evaluated. As for DR, we should know whether DME can be screened with NMR in clinical settings without the care of an ophthalmologist. When reviewing the studies published, we did not find any work specifically designed to evaluate the degree of agreement among endocrinologists with a low level of training in the detection of DME by means of stereoscopic NMR, only taking into account retinal thickening and not hard exudates. Therefore, in this investigation it is our aim to find more data regarding the reliability of NMR with the stereoscopic viewfinder.

Subjects and Methods

A sample of diabetes patients with DR attending the endocrinology and ophthalmology departments of a tertiary hospital was taken. The potential candidates were identified during their outpatient clinic follow-up and using the hospital database. Then they were contacted either directly in the clinic or by telephone. Consent was obtained from all the patients. The criteria for including them were as follows: patients with T2DM with any degree of diagnosed DR, with or without previous treatment for retinopathy, and residing in the province of Zaragoza, Spain. Exclusion criteria were as follows: retinal detachment, vitreous hemorrhage or neovascular glaucoma, retinal dystrophy, retinal arterial occlusion or venous thrombosis, prior cranioencephalic trauma, treatment with toxic drugs for the retina, and difficult approach to hospital due to limited mobility. Advanced age was not considered to be a criterion for exclusion; neither were the duration or type of diabetes treatment considered. Data confidentiality was maintained all the time. With a 95% confidence level, maximum error of 0.05–0.1, and DME prevalence of 16% taken from previous local research, a range of about 50–200 researching data was deemed to be necessary. It was thus decided that a sample of 100 eyes should be studied. Fifty-three patients took part in the study.

Images of a macular retinal field were obtained at 30° and 45°, with spontaneous dilation in a dark room and following tropicamide. The equipment used was a Topcon TRC- NW6S nonmydriatic retinograph (Topcon S.A., Barcelona, Spain) with Imaginet 2000 Lite software and a Topcon Screen-Vu stereoscopic viewfinder. A technical control was performed by the manufacturer before the study.

An endocrinologist (examiner 1) with previous basic training studying 15 retinographs supervised by an ophthalmologist (identifying anatomical structures, characteristic DR lesions, retinal thickening, and common artifacts), performed the retinographs and later interpreted them. Another endocrinologist (examiner 2) with no specific previous training also interpreted the images in order to calculate the degree of agreement between them. Both doctors used the same equipment in the same conditions. Poor quality images (comprising single and stereoscopic pairs) were excluded. Technical failure rate was calculated as the percentage of failed pairs of images over the whole expected sample. Of the 238 pairs of stereoscopic images evaluated by the endocrinologists, 182 were selected by them for diagnosis. This selection was done under the criteria of both endocrinologists, expressed as “sufficient photographic quality.” Cases were classified in excellent quality when image clarity and focus allowed identifying main and secondary vessels. The possible categories were as follows: “edema” when retinal thickening was detected less than one disc diameter far from the center of the macula, whether hard exudates were present or not, or “no edema” in the opposite case. Basic data from clinical files and basic physical examinations were compiled. Later, definitive diagnosis of DME performed by OCT was searched in the hospital database for each patient. All patients had reports of the OCT examination in the previous 3 months. DR severity data were collected from the previous routine fundoscopic examination, following the International Severity Scale. The statistical calculations were made using SPSS 15.0 version (SPSS, Inc., Chicago, IL). Unweighted κ index agreement was calculated for each image option.

Results

The κ values that we found show in general low agreement among endocrinologists with a low level of training for this kind of examination.

The baseline characteristics of the cohort are presented in Table 1. We found a prevalence of DME diagnosed by OCT in our sample group of 68% (53 patients, 99 eyes, 67 cases of DME). From the images obtained with spontaneous dilations, 13% were of excellent quality, whereas of those obtained following tropicamide instillation, 59% achieved that same level of quality. The technical failure rate was calculated to be 40.5%. The results of the agreement index values were as follows: κ index for the total number of images analyzed, 0.16 (n = 182, P = 0.02); for 30° retinal field images with spontaneous mydriasis, 0.21 (n = 21, P = 0.22); for 45° retinal field images with spontaneous mydriasis, 0.58 (n = 22, P = 0.009); for 30° retinal field images using tropicamide, 0.14 (n = 68, P = 0.23); and for 45° retinal field images using tropicamide, 0.01 (n = 71, P = 0.88). Complete data are shown in Table 2.

Table 1.

Characteristics of the Population

Characteristic	Mean	Ratio
Sex	—	48% women, 52% men
Age	66 years
Body mass index	32 kg/m²
HbA1c	7.9%
Diabetes' duration	18 years
Antidiabetes treatment	—	56% oral treatment, 83% insulin, 39% both
Total cholesterol	177 mg/dL
HDL-cholesterol	51 mg/dL
LDL-cholesterol	95 mg/dL
Triglycerides	144 mg/dL
Dyslipidemia	—	59%
High blood pressure	—	72%
Retinopathy severity
Mild nonproliferative	—	65%
Moderate nonproliferative	—	15%
Severe nonproliferative	—	10%
Proliferative	—	10%

HbA1c, glycated hemoglobin; HDL, high-density lipoprotein; LDL, low-density lipoprotein.

Table 2.

Crosstabs from Agreement Index Analysis

	Examiner 2
Examiner 1	No DME	DME	Total
Total number of images analyzed
No DME	92	21	113
DME	46	23	69
Total	138	44	182
30° retinal field images with spontaneous mydriasis
No DME	10	1	11
DME	7	3	10
Total	17	4	4
45° retinal field images with spontaneous mydriasis;
No DME	9	1	10
DME	4	8	12
Total	13	9	22
30° retinal field images using tropicamide
No DME	43	10	53
DME	10	5	15
Total	53	15	68
45° retinal field images using tropicamide
No DME	31	8	39
DME	25	7	32
Total	56	15	71

Discussion

We believe that the sample group obtained for our study is representative enough of the type 2 diabetes patients from our clinical settings. The proportion of eyes with DME is much higher than that shown in previous works, probably because the patients were controlled in the outpatient clinic of a tertiary hospital, with a longer duration of diabetes and greater incidence of complications. In other studies patients are enrolled from general practice care, representing less complicated diabetes. Also, they all had to be suffering from DR in order to be included in the study, and DME prevalence was calculated from OCT findings, much more sensitive than other tests.

Analysis was performed under spontaneous and pharmacological dilation because in case we had found good agreement for both situations, the next objective would have been to evaluate which one is better. However, except for the 45° field without tropicamide, agreement was poor. Regarding the situation of the 45° field using tropicamide, the main difference appears when examiner 1 detects DME and examiner 2 does not. Poor agreement for pharmacological dilation was unexpected. We chose different retinal fields to determine if image magnification changed interobserver variation. It seems to be an improvement when considering spontaneous dilation.

In a previous study¹⁹ with nonstereoscopic images, the diagnoses of four groups were compared: photographers and nurses working in ophthalmology departments who had been given 2 days' training 1 month previously, general ophthalmologists, and ophthalmologists specialized in the retina. The authors observed the following results: an agreement index for all the examiners of 0.27 and for the ophthalmologists specialized in the retina of 0.58. The group of photographers was the one that best agreed with the specialists in the retina (reference group), with an index of 0.36. Pharmacological mydriasis was required in 33% of the sample in order to obtain images with sufficient quality. The authors suggested that the low level of agreement among the general ophthalmologists could be due to the fact that the diagnosis differs in the cases where no hard exudates are present, thus making the detection of thickening more difficult. They recommended reinforcing the continuous training of these professionals. In our case, the level of agreement for the total of the images analyzed does not reach the agreement that was observed for any of the four groups of professionals or for each group compared to the reference one. However, great methodological differences exist between both studies.

Digital stereoscopic images have been evaluated using the Joslin Vision Network as mentioned before,¹⁸ and a κ interobserver agreement value for CSME (according to the criteria of the Early Treatment of Diabetic Retinopathy Study) of 0.57 was found. It was very similar to the value corresponding to the standard 35-mm retinographs. In the previous article, the kind of specialist who interpreted the images is not specified. When we analyzed our results depending on the kind of retinography, we found a κ index value of 0.58 for the images in spontaneous mydriasis with a 45° retinal field. However, there was an unexpectedly greater reduction in the number of data analyzed, and therefore this information must be considered with caution. One important methodological difference between the two works is the difference in the size of edema taken into account; our definition of DME was wider than that of CSME.

The technical failure rate that we obtained is much higher than what is usually accepted for including NMR in a screening program.²⁰ Nevertheless, given that the interobserver reliability depends on the diagnoses from the images of suitable quality, it does not invalidate our results as they describe the behavior of the technique with valid data. Both parameters—the technical failure rate and the reliability—together with the rest of the diagnostic ability indicators and the economic evaluation of the procedure, provide us with the information needed for implementing NMR in screening programs. Regarding the training necessary for achieving images of a sufficient quality with NMR, it has been noted that there are no substantial differences among a professional photographer specialized in ophthalmologic imaging, a photographer with 2 days' training and experience on 50 eyes, and another with 1 days' training and experience on 10 eyes.²¹ All the patients had diabetes. No differences were found taking half the patients who were dilated by protocol. Our case could correspond to the training of the third photographer, although methods differ and quality for stereoscopic vision was not evaluated in that work. An experienced imager in our team may have helped to improve the results.

The interpretation of the images by endocrinologists has been studied previously,¹³ with physicians receiving more intensive training than in our study by a doctor specialized in the retina and taking as the gold standard the diagnosis performed by ophthalmologists on the same monoscopic images. The proportion of cases with maculopathy detected as hard exudates improved following dilation, with values that were quite similar for the two endocrinologists. When reviewing the issue, we found no articles in which the results of endocrinologists were contrasted with those of ophthalmologists for stereoscopic images.

Conclusions

We must therefore conclude that the use of NMR for DME detection does not achieve a suitable degree of agreement among endocrinologists minimally trained in this procedure, regardless of the pharmacological dilation and the angle of the retinal field examined. NMR is a diagnostic test of interest for screening complications of a highly prevalent disease, which makes it likely to be used by different health providers with different qualification depending on their geographical area. Moreover, with the technical development of nonmydriatic cameras and adapted software, performance features currently available are likely to be improved upon. Given the above-mentioned reasons, we think that criteria for determining the type and level of training needed for acquiring and interpreting this kind of retinography should be developed, standardized, or accepted by the scientific communities involved in, in order to then consider a screening program that will be properly carried out regarding DME.

Footnotes

Author Disclosure Statement

No competing financial interests exist.

References

American Diabetes Association. Standards of medical care in diabetes 2009 (position statement) Diabetes Care, 2009; 32,Suppl 1:34–35.

Girach

, Lund-Andersen

. Diabetic macular oedema: a clinical overview. Int J Clin Pract, 2007; 61:88–97.

Fong

, Aiello

, Gardner

, King

, Blankenship

, Cavallerano

, Ferris

, Klein

. Retinopathy in diabetes. Diabetes Care, 2004; 27,Suppl 1:84–87.

Cunha-Vaz

. Retinopatía diabética. LXXXII Conference of Spanish Society of Ophthalmology. Spanish Society of Ophthalmology. Madrid, 2006; 182–183.

Mateo Orobia

. Utilidad de los retinógrafos no midriáticos en el diagnóstico y seguimiento de la retinopatía diabética [Doctoral Thesis] Zaragoza, Spain: University of Zaragoza, 2006.

Dasbach

, Fryback

, Newcomb

, Klein

. Cost-effectiveness of strategies for detecting diabetic retinopathy. Med Care, 1991; 29:20–39.

Photocoagulation for diabetic macular edema. Early Treatment Diabetic Retinopathy Study report number 1. Early Treatment Diabetic Retinopathy Study research group. Arch Ophthalmol, 1985; 103:1796–1806.

Rudnisky

, Tennant

MTS

, de Leon

, Hinz

, Greve

MDJ

. Benefits of stereopsis when identifying clinically significant macular edema via teleophthalmology. Can J Ophthalmol, 2006; 41:727–732.

Bresnick

, Mukamel

, Dickinson

, Cole

. A screening approach to the surveillance of patients with diabetes for the presence of vision-threatening retinopathy. Ophthalmology, 2000; 107:19–24.

10.

Massin

, Girach

, Erginay

, Gaudric

. Optical coherence tomography: a key to the future management of patients with diabetic macula oedema. Acta Ophthalmol Scand, 2006; 84:466–474.

11.

Virgili

, Menchini

, Dimastrogiovanni

, Rapizzi

, Menchini

, Bandello

, Chiodini

. Optical coherence tomography versus stereoscopic fundus photography or biomicroscopy for diagnosing diabetic macular edema: a systematic review. Invest Ophthalmol Vis Sci, 2007; 48:4963–4973.

12.

Ahmed

, Ward

, Bursell

, Aiello

, Cavallerano

, Vigersky

. The sensitivity and specificity of nonmydriatic digital stereoscopic retinal imaging in detecting diabetic retinopathy. Diabetes Care, 2006; 29:2205–2209.

13.

Deb-Joardar

, Germain

, Thuret

, Manoli

, Garcin

, Millot

, Gavet

, Estour

, Gain

. Screening for diabetic retinopathy by ophthalmologists and endocrinologists with pupillary dilation and a nonmydriatic digital camera. Am J Ophthalmol, 2005; 140:814–821.

14.

Scanlon

, Foy

, Malhotra

, Aldington

. The influence of age, duration of diabetes, cataract, and pupil size on image quality in digital photographic retinal screening. Diabetes Care, 2005; 28:2448–2453.

15.

Murgatroyd

, Cox

, Ellingford

, Ellis

, MacEwen

, Leese

. Can we predict which patients are at risk of having an ungradeable digital image for screening for diabetic retinopathy? Eye, 2008; 22:344–348.

16.

Hansen

, Sander

, Larsen

, Kleener

, Borch-Johnsen

, Klein

, Lund-Andersen

. Screening for diabetic retinopathy using a digital non-mydriatic camera compared with standard 35-mm stereo colour transparencies. Acta Ophthalmol Scand, 2004; 82:656–665.

17.

Ruamviboonsuk

, Wongcumchang

, Surawongsin

, Panyawatananukul

, Tiensuwan

. Screening for diabetic retinopathy in rural area using single-field, digital fundus images. J Med Assoc Thai, 2005; 88:176–180.

18.

Bursell

, Cavallerano

, Clermont

, Birkmire-Peters

, Aiello

. Joslin Vision Network Research Team. Stereo nonmydriatic digital-video color retinal imaging compared with Early Treatment Diabetic Retinopathy Study seven standard field 35-mm stereo color photos for determining level of diabetic retinopathy. Ophthalmology, 2001; 108:572–585.

19.

Ruamviboonsuk

, Teerasuwanajak

, Tiensuwan

, Yuttiham

. Interobserver agreement in the interpretation of single-field digital fundus images for diabetic retinopathy screening. Ophthalmology, 2006; 113:826–832.

20.

British Diabetic Association. Retinal Photographic Screening for Diabetic Eye Disease. A British Diabetic Association Report. London: British Diabetic Association, 1997.

21.

Maberley

, Morris

, Hay

, Chang

, Hall

, Mandava

. A comparison of digital retinal image quality among photographers with different levels of training using a non-mydriatic fundus camera. Ophthalmol Epidemiol, 2004; 11:191–197.