Abstract
Background:
Studies have shown that teleophthalmology programs using a nonmydriatic camera in primary care settings can improve rates of diabetic retinopathy (DR) screening. However, such programs are not yet widespread due to common challenges in sustainability.
Purpose:
To comprehensively evaluate clinical and operational measures of an urban primary care clinic's 1-year pilot teleophthalmology DR evaluation program.
Materials and Methods:
This retrospective analysis used five metrics to evaluate the program: clinical diabetic retinal exam (DRE) rate, visual acuity and pathology, camera utilization, billing and insurance reimbursements, and outcomes of follow-up referrals.
Results:
Two hundred eleven patients were screened over 14 months. The DRE rate had more than doubled (34–75%). Of the patients, 55.9% had vision better than 20/50 in each eye and 21% with at least 1 eye worse than or equal to 20/70. DR was noted in 11% of patients. The program's first few months saw greatest camera use. Government and Medicare Advantage insurers were significantly (p < 0.001) less likely to reimburse than commercial insurers. Twenty-seven percent of patients screened had documented follow-up with an eye care provider within 16 months of their screening. Patients diagnosed with DR or recommended follow-up within 1 month were significantly (p < 0.001) more likely to schedule an appointment.
Discussion:
Challenges to program sustainability include efficient utilization, reimbursement from governmental insurers, and adherence to follow-up recommendations.
Conclusions:
Assessing teleophthalmology programs with the aforementioned five metrics allows for a comprehensive evaluation of impact and sustainability. This may be utilized to standardize the implementation and evaluation of such programs across diverse settings.
Introduction
Diabetes mellitus affects 29.1 million in the United States, 1 and 40–45% are estimated to have some form of diabetic retinopathy (DR), 2 the leading cause of blindness in the United States working-age population. 3 An annual dilated retinal examination has been standard of care for decades to avoid vision loss from DR. 4,5 Despite the risks, many health care systems, especially in underserved areas, struggle to meet nationally defined benchmarks for screening for DR. 6,7
Teleophthalmology, using nonmydriatic camera-based assessment for retinopathy, has good specificity and sensitivity for remote detection of retinopathy and other eye diseases without having to travel to see an eye doctor. 8 –14 Several evaluations have shown that placing a retinal camera in a primary care office can increase annual screening for DR, adequately stage the level of retinopathy, and identify vision-related pathology such as optic nerve issues, maculopathy, and cataract. 15 –18 However, few discuss if such programs increase the rate of annual retinal examinations from a known baseline 19 –21 as per the Healthcare Effectiveness Data and Information Set (HEDIS) standards. 22 Furthermore, tracking of eye care follow-up has not been reported in most prior evaluations. 23 –25 Understanding the business model and financial sustainability of such programs is also key to evaluating implementation. 26 –30
In 2015, Silva and Aiello noted the importance of reporting on real-world outcomes of telemedicine used for eye examinations, 31 and the recent impact of COVID-19 has only increased the relevance of teleophthalmology programs. Our evaluation presents the outcomes of a teleophthalmology program for DR detection over a 14-month period in a U.S. primary care setting of largely minority, low income patients with diabetes.
Materials and Methods
This retrospective evaluation received exemption from the Research Subjects Review Board (RSRB) at the University of Rochester.
This analysis used five measures to evaluate the teleophthalmology program at a university owned, neighborhood based, urban primary care practice serving a low-income population in Rochester, NY. The outcome evaluation followed recommendations outlined in a framework proposed to review similar teleophthalmology programs for the Veterans Affairs in 2006. 32 Operational details of the program have been previously published. 33
In brief, patients with diabetes who did not have a documented eye examination as per HEDIS criteria were identified by clinic staff a week before their regularly scheduled primary care visit. These patients were flagged by clinic staff as individuals who should be informed about the program and asked to participate when they showed to their clinic appointment by the staff member who performed their clinic intake and who had all been educated about the program by the ophthalmology teleophthalmology team and by the program's clinic champion, a nurse practitioner. The ophthalmology team also worked with the clinic to make and post flyers advertising the program in the clinic waiting room and examination area and gave educational lectures to all staff members and clinic providers.
An ophthalmic photographic technician from the University of Rochester trained the nurse practitioner program champion, two registered nurses, and two patient care technician individuals to operate a camera, check distance visual acuity, and to use the online portal over three training sessions, and each staff member had to pass a written and practical evaluation with images graded by the ophthalmic photographer and a retina specialist (R.S.R.) to be internally certified as “trained” to take photos and conduct the teleophthalmology evaluation. The retina specialist gave an educational lecture on the program and DR at the start of and two more times during the 14 months of the program evaluation period to clinic staff. As the champion for the program, the nurse practitioner was the “super-user.” She received additional training from the ophthalmic photographer and retina specialist and achieved the level of proficiency to be able to train others in how to operate the camera and check vison. She also ensured that there were sufficient available staff trained to conduct the program.
Over the course of the 14 months of this evaluation, there was always a trained staff available in clinic who could operate the camera and perform the workflow of the teleophthalmology program. Staff always could and did regularly contact the program manager for the teleophthalmology program with any questions or technical help needed. For patients participating in the program, the trained technicians or nursing clinic staff recorded the patient's information and captured four digital photos of each eye using a Topcon NW400 nonmydriatic fundus camera, three posterior and one anterior. These images were uploaded to a secure cloud server to be read by a single retina specialist (R.S.R.) from the University eye institute. The ophthalmologist used the International Clinical Diabetic Retinopathy Disease Severity Scale grading standard for DR, noted any other pathology that may have been present in the posterior and anterior photographs, and then made recommendations in an electronically-generated report through the cloud server for follow-up time to eye care as per the guidelines in the updated 2015 Preferred Practice Pattern from the AAO and clinical judgment. The protocol used follows that used and published by EyePACS, Inc. (Berkeley, CA), 15 for which the retina specialist (R.S.R.) also serves as a grader, and is classified at ATA category 3. 34 Distance visual acuity and the use of corrective lenses were also measured and recorded by trained primary care staff. Examination results and recommended eye care follow-up duration given by the one retina specialist (R.S.R.), who reviewed all images and data, were shared with the patient within a few days by the primary care clinic care managers, who placed a referral for an examination in the Electronic Medical Record (EMR).
Ophthalmology staff called patients based on this referral and helped them to schedule appointment at the University clinic if the patient agreed to do so. Follow-up to eye care for this cohort of patients was tracked through the EMR until the end of 2017 or one and half years after the last participant during the period evaluated.
Description Of The Five Measured Outcomes
Clinic diabetic retinal examination rate
The percentage of the clinic's patients with diabetes receiving an annual retinal examination either through the camera-based program or by seeing an eye doctor and having a report of a dilated eye examination was tracked in the EMR population health dashboard by the clinic as part of monthly quality metrics reported by the clinic. Throughout the 14-month evaluation period, the clinic's diabetic retinal exam (DRE) rate was regularly checked to observe changes due to the camera implementation.
Visual acuity and pathology
Visual acuity, the level of DR, and the presence of other ocular pathology detected for each patient were obtained from the program's cloud-based database for each patient.
Camera utilization
The frequency of camera use was noted on a monthly basis.
Billing and insurance reimbursement
Fee for service (FFS) insurance billing was utilized by both primary care and ophthalmology using the 92250 Current Procedural Terminology (CPT) code for ophthalmic photography. Using the modifiers technical fee (TC) and 26 enabled the primary care site to capture the TC and the ophthalmology department to capture the professional fee (26). Health insurance coverage for patients at the time the code was billed was noted along with the amount collected for both the technical and professional components by primary care and ophthalmology from FFS billing records.
Referral and follow-up
Recommended eye care follow-up ranged from less than 1 to 15 months. Using the University's EMR and scheduling system, follow-up appointments were tracked and analyzed to see what percentage of patients were scheduled and actually followed-up within the recommended time frame to the university eye clinic, which was the primary eye care provider for this population. Patients who followed up were classified as either new patients, if not seen within the last 3 years, or as established patients. To account for patients choosing to seek care outside of the university, a study team member searched for notes from outside eye doctors in the EMR that would indicate the patient had eye care follow-up.
Statistical Analysis
Data for those who participated in the teleophthalmology screening were available and analyzed. Characteristics of study measures were summarized through ordinary descriptive statistics. For binary variables, percentages and frequencies were reported; for continuous variables, sample means and standard errors were reported. Appropriate statistical tests were performed to quantify the strength of association between categorical variables, such as the visual acuity and pathology found, billing to insurance, and follow-up outcomes based on the recommendation. Specifically, Fisher's exact test was used to compare two categorical variables; two-sample test of proportions was conducted to determine if there was a significant difference between two proportions. For continuous measures, Welch's two-sample t-test was used to test whether the means of two groups are equal. In cases where there were more than two groups, the one-way ANOVA test was used to test the global null hypothesis of equal means across all groups. A p-value <0.05 was considered statistically significant.
All analyses were performed in R 4.0.1 (R Foundation for Statistical Computing, Vienna, Austria) and in SAS 9.3 (SAS Institute, Cary, NC).
Results
Over a 14-month period, 222 visits from 211 unique patients with diabetes were screened for retinopathy using the clinic's nonmydriatic fundus camera during normally scheduled visits. As presented in Supplementary Table S1, the average age was 54 years with a majority being female, African American, and covered by government-funded health insurance.
Clinic Dre Rate
The clinic's DRE rate with an eye care provider as shown in Supplementary Table S1 increased from 34% at baseline to 75% at 14 months. Ninety-seven percent of the increase in the DRE metric was directly due to the teleophthalmology program as only 14 more patients met this metric by having an eye examination with an eye doctor.
Visual Acuity And Pathology
The majority of patients (118 [56%]) had 20/40 or better vision in each eye, with 20 (10%) having binocular vision worse than or equal to 20/70 (Supplementary Table S2). Forty-four patients (21%) had at least 1 eye worse than or equal to 20/70. DR was noted in images of 23 (11%) patients, with 4 patients having apparent diabetic macular edema as determined by finding hard exudates within the macula on the retinal images. Age, race, ethnicity, and level of visual acuity were not associated with having DR, and 74% of patients with DR had vision of 20/40 or better in each eye. Other ocular pathology was noted in 53 (25%) patients, and image quality for 31 (15%) patients was inadequate to accurately assess for DR, with about a quarter of these patients having a cataract. Older age, but not sex, race, or ethnicity, was significantly associated (p < 0.001) with the presence of non-DR pathology, especially cataract (p < 0.001), poorer vision, and inadequate retinal image quality imaging controlling for the presence of having a cataract (p < 0.001).
Camera Utilization
Greater frequency of use, with over 100 evaluations, occurred in the first few months of initiating the program, but decreased steadily to an average of 8 visits after month 4 (Fig. 1). The proportion of patients with visually significant findings or inadequate quality imaging was not significantly different over time. Eleven patients underwent a second imaging session during the 14 months.

Camera utilization versus frequency of pathology: A high initial screening rate was followed by a decrease in subsequent months for an average of eight screenings per month. Eleven patients underwent a second evaluation during the 14-month period.
Billing And Insurance Reimbursement
FFS billing using the 92250 CPT code was done separately by ophthalmology and the primary care for the professional and technical components, respectively. Commercial insurers were more likely (p < 0.001) to reimburse the professional and technical component than government-funded insurances, including Medicare Advantage (Fig. 2). Billing claims were not submitted for patients with images that could not be graded. Ophthalmology billed significantly (p = 0.006) more professional component charges (131 [69%] patients) than the number of technical component charges submitted by primary care (102 [53%] patients). Ophthalmology collected on significantly less, 67 (51%), of these claims than primary care, which collected on 83 (81%) claims (p < 0.001). There was no significant association between the number of claims submitted/reimbursed and a diagnosis of retinopathy or other ocular pathology. While the collection rate was higher for primary care, primary care also submitted fewer claims to the government insurers and Medicare Advantage (p < 0.001).

Billed visits by insurance class: The eye care center and the primary care clinic did separate fee for service billing using the Current Procedural Terminology code 92250. The eye care center submitted higher numbers of claims, especially to government insurers. *Collection rate was significantly higher (p < 0.001) for primary care compared to ophthalmology.
Referral And Follow-Up
Seventy-five of 211 patients (35.5%) scheduled appointments with an eye care provider within 16 months of their camera screening, and 57 (76%) of these patients saw an eye doctor within 16 months for a total follow-up rate of 27% (Supplementary Table S3). Of the patients that followed up, 35 (61%) did so within the recommended time frame. Those diagnosed with any level of DR were significantly more likely to schedule a follow-up appointment (65%, p < 0.001), but not more likely to keep their appointment or follow-up within the recommended time frame. Those asked to follow-up within 1 month had visually threatening disease (23 of 211 patients [11%]) and were more likely to schedule an appointment (p < 0.001). However, only 10 of these patients actually follow-up to eye care with only 4 doing so within the recommended time frame. Age, gender, race, ethnicity, insurance class, having non-DR pathology, and having poorer vision were not associated with being more or less likely to make or keep a follow-up eye care appointment. Of the 57 patients who had a documented follow-up eye doctor visit in the EMR, 40 (70%) followed with the University ophthalmology department and 28 of 40 (70%) were new patients to the department.
Discussion
Implementing a teleophthalmology program in an urban and largely minority safety net clinic affiliated with an academic medical center, in conjunction with an academic ophthalmology department, demonstrated a significant increase in annual DRE rate from 34% to 75%. Most patients evaluated through teleophthalmology had good vision, and non-DR pathology was more commonly detected than DR pathology. In addition to the program's uniqueness in incorporating visual acuity as part of the screening, our evaluation was distinctive in its tracking of camera utilization, reimbursement of billing for the examination, and follow-up to recommended eye care.
Our doubling of the DRE rate in 1 year by introducing the camera compares favorably to the 1-year Vine Hill Study (23–59%) 10 and a larger scale study's improvement of DRE from 41% to 57% in a 2-year study of a program in LA County. 19 However, not all studies had drastic DRE advancement, as a similarly sized single clinic pilot program only saw a 3% DRE increase over 2 years. 35
As reported by other programs, our pilot identified both DR and non-DR pathology. Although the number identified with DR was lower than in some studies, 19,36 the rate of detecting non-DR was similar to other studies. 10 Older patients were more likely to have non-DR pathology, with predictably higher likelihood of cataract. Our program is one of the few to measure distance visual acuity. A majority with DR had vision better than 20/40, demonstrating that DR is a silent disease early on and sometimes even much later in the disease course. This only increases the value of screening in non-eye care settings as patients may not seek eye care even for a routine examination until they experience a problem with their vision. 33,37 –40
As seen in previous studies, the number of eligible patients for screening in our pilot decreased significantly after the first few months of implementation. Very few studies have looked at longitudinal utilization but, like our findings, those that did have realized camera utilization is a major challenge of DR teleophthalmology programs. 35,41,42 Our program did have a dedicated staff member to champion recruiting patients needing an eye examination and overseeing the imaging of these patients by ensuring that there were enough trained staff available on each clinic day to carry out the workflow of the program. This may be why nearly 80% of patients had been assessed by the teleophthalmology program or had a documented eye examination with an eye doctor in the first year after implementing our program. The clinic staff were very aware of who needed imaging before implementing the program and were very proactive at encouraging these individuals to make primary care appointments and participate right after the program started as the majority of the clinic's eligible patients with diabetes were imaged within the first few months of operation.
Although not specifically assessed, the lower utilization of the camera after the first few months may have been because the majority of those individuals requiring the evaluation who regularly showed to the primary clinic were assessed in the first few months and those that were assessed later may have belonged to a pool of patients that did not as frequently see their primary care provider. Clinic staff did report that most of the patients not captured by the teleophthalmology program did not show to the clinic during the evaluation period of the program. They reported that many individuals who did not show to clinic are home bound for physical and psychological reasons. They also asked if our program could expand to include a mobile service. Since the implementation of our program, local visiting nurse services have incorporated teleophthalmology evaluation for DR using handheld cameras during home visits in partnerships with commercial entities.
Larger scale programs had fewer issues with utilization as cameras could be shared between multiple clinics. Phan et al. mention the importance of planning a strategic workflow for optimal utilization, and this is vital for program sustainability. 27 Utilization of the camera is important to justify the cost of these programs, where the purchase of a nonmydriatic table-top camera currently ranges from $10 to $20K. In addition, 31 patients (15%) were unable to be successfully screened due to nongradable images, similar to the findings of other previous studies. 8,35,43 This was likely due to either small pupils or media opacities, although, initially, also possibly related to the learning curve of operating the camera. 44
While our program used the 92250 CPT code, other programs have used the 92227, which has a different payment amount and likelihood of reimbursement. Several studies have focused on the costs associated with implementing a screening program, 26,45,46 but very few reported on collecting reimbursements. 30 Of the patient population in our primary care clinic, 56% had Medicare and/or Medicaid, which is typical of an inner-city primary care clinic and of the population at highest risk for DR vision loss. 47 –51 Commercial insurers not only reimbursed a higher dollar amount on average, they also were much more likely to reimburse than Medicare and Medicaid. Thus, only about a quarter of those screened with government funded insurance had their screening reimbursed. This difference in reimbursements was similarly noted by Mamillapalli et al. 35 One other study looked at the cost-benefit analysis of a county hospital teleophthalmology program, concluding that they failed to break even on investment 4 years after screening implementation and that coordination of workflow before implementation on the selection of screened patients is critical for success. 27 Clinics with higher rates of noncommercial payers who do not often recognize the ophthalmic photography codes such as 92250 when used for teleophthalmology based evaluation of eyes without DR may need to find alternative ways to financial support teleophthalmology programs for DR.
We found that teleophthalmology prioritizes patients in the queue for appointments based on pathology detected. Most studies that tracked follow-up only tracked appointment scheduling. 10,19,52 Follow-up to eye care based on teleophthalmology assessment is essential to improving care. Our results showed that tracking appointment scheduling as opposed to actual follow-up may overstate value of the program, as only 17% of the 211 patients had initially scheduled and completed their appointment within the recommended time frame from a total of 27% of the patients having a documented visit with an eye care provider within 16 months of their screening. One evaluation that focused on tracking actual follow-up demonstrated similar problems, with over 50% having no recorded follow-up 2 years after screening. 23 Another study surveyed patients on the barriers to referral adherence, finding that the cumulative effect of multiple barriers proved most likely to cause incomplete referral. 53 Soliman et al. mentioned that patient awareness of the need to follow-up to eye care was key for increasing the likelihood of referral adherence in patients having retinopathy, as seen in our pilot program. 25 Patient education and follow-up adherence within recommended time frames is still a challenge that future research needs to focus on. 54 It is noteworthy that 70% of the patients who followed up were new patients to the ophthalmology clinic, another value for ophthalmology departments and eye care centers championing such initiatives with primary care.
Strengths of this pilot program evaluation are the comprehensiveness of the clinical and operational metrics measured and reported on, including ocular findings, billing outcomes, camera utilization, and patient follow-up. The pilot clinic in this evaluation is comparable to many other low-income, urban primary care clinics that face similar challenges. Such clinics may use this replicable framework either to evaluate current teleophthalmology programs or to plan workflows for implementing a new teleophthalmology program.
Our evaluation was limited to one university affiliated, neighborhood-based primary care clinic with a single year of data and follow-up of 16 months after the last patient went through the program. Although a pilot program is not meant to be generalizable, further longitudinal data would have demonstrated the sustainability of the programs' operation and ability to consistently achieve the increased DRE rate in subsequent years. A study by Mansberger et al. showed such sustainability to be problematic in their 5-year investigation. 8 Furthermore, those who were not evaluated in the teleophthalmology program were not tracked to understand how they differed. In addition, follow-up to eye care may have been underestimated due to the unavailability of tracking some patients seeking follow-up eye care outside the university system.
Conclusions
Urban primary care clinics face the most challenges in implementation, and their patient population is the least likely to follow-up to ophthalmology. 55 Evaluating programs on our five metrics allows for a standardized way to characterize a program's success, sustainability, and value to population health. This evaluation approach can be applied to similar programs and used to inform and standardize evaluation metrics in teleophthalmology across diverse U.S. settings. Future studies should focus on the issues still impeding program sustainability, including efficient utilization of the camera, creating the right business model, the struggle for reimbursement especially from governmental insurers, and improving patient adherence to follow-up recommendations.
Footnotes
Disclosure Statement
No competing financial interests exist.
Funding Information
This study was not funded by any outside organizations apart from the University of Rochester.
Supplementary Material
Supplementary Table S1
Supplementary Table S2
Supplementary Table S3
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
