Patient Satisfaction with the Hybrid Telemedicine Model for Ophthalmology

Abstract

Background:

The purpose of this research was to compare patient satisfaction between hybrid ophthalmology telemedicine and standard-of-care in-person visits. A retrospective, cross-sectional, case–control analysis of patient satisfaction based on survey data was used.

Methods:

Responses to the National Research Council Health Patient Survey were retrieved for randomly sampled hybrid ophthalmology telemedicine and in-person visits between March 11, 2020 and December 31, 2021 at a hospital-based eye clinic in Boston, Massachusetts. The primary outcome was based on the question “How likely would you be to recommend this provider to your family and friends?” (0–10 scale) with a score of 9 or 10 coded as satisfied. Two-sample t-tests, Pearson's chi-square tests, and bivariate logistic regressions were used to compare patient satisfaction scores between the hybrid and in-person cohorts. Demographic data, including age, sex, language, and self-reported race and ethnicity, were used as potential predictors of patient satisfaction in a multivariable logistic regression model.

Results:

There were 49 surveys from hybrid visits and 3,390 surveys from in-person visits. Hybrid visit patients reported high satisfaction scores without significant differences compared to in-person visit patients (hybrid 79% satisfied, in-person 82% satisfied, p = 0.728). Age was significantly associated with satisfaction in the hybrid cohort with the 65+ age group reporting lower satisfaction (below 65 years 100% satisfied, 65+ years 60% satisfied, p = 0.003). No association with age was observed in the in-person cohort.

Conclusions:

The hybrid ophthalmology telemedicine model can provide effective care without sacrificing patient satisfaction. Older patients may benefit from targeted interventions in future telemedicine models.

Introduction

Many studies have described the benefits of telemedicine in eye care.^1

–4 One model, the hybrid ophthalmology telemedicine model, was developed during the COVID-19 pandemic to provide clinical care to patients with stable nonprocedural complaints.⁵ It consists of an in-person imaging appointment with a trained ophthalmic technician followed by a virtual appointment with a clinician within 14 days. During the imaging appointment, the technician collects basic ophthalmic vitals (e.g., visual acuity and intraocular pressure) and a protocol-driven array of imaging and testing (e.g., fundus photography, ocular coherence tomography, and visual fields) based on the chief complaint.⁵ The objective data gathered during the imaging appointment are designed to help clinicians more accurately assess and manage ocular diseases compared to a virtual-only telemedicine visit. All appointments were conducted with trained or family interpreters in the patient's preferred language. An extensive review of this model and the testing protocols used are discussed in our previous publication.⁵

Patient satisfaction is a key quality measure in patient-centered care and may guide payments from the Centers for Medicare and Medicaid Services (CMS).^6
–8 A systematic review of 44 telemedicine studies found that patient satisfaction with telemedicine was associated with improved outcomes, better communication, and lower cost.^9,10 While there have been studies investigating patient satisfaction with telemedicine in nonophthalmic medical specialties,^11,12 very few have looked at patient satisfaction with telemedicine in ophthalmology,¹³ and to our knowledge, there have been no prior studies investigating patient satisfaction with any hybrid ophthalmic telemedicine model.

In this study, we compared patient satisfaction scores between hybrid ophthalmology telemedicine and standard in-person visits at our hospital-based eye clinic at Boston Medical Center, which is the largest safety-net hospital in the Northeastern United States (U.S.). We also investigated whether demographic factors, such as age, sex, primary language, and self-reported race and ethnicity, were associated with patient satisfaction. As telemedicine use continues to grow, it is imperative that emerging telemedicine models be evaluated for their utility, equitability, and acceptance by patients.

Methods

This was a retrospective, cross-sectional, single-center analysis of patient satisfaction survey data following hybrid and standard in-person visits to the Department of Ophthalmology of Boston Medical Center from the beginning of the COVID-19 pandemic on March 11, 2020 to December 31, 2021. This study was approved by the Boston University Institutional Review Board (IRB No.: H40455) and adhered to the tenets of the Declaration of Helsinki.

NATIONAL RESEARCH COUNCIL HEALTH PATIENT SURVEY

The National Research Council (NRC) Health Patient Survey is a patient satisfaction tool used in 75% of the 200 largest U.S. hospital chains and over 9,000 U.S. health care organizations, including our study site.¹⁴ The survey was approved by the Hospital Consumer Assessment of Health Care Providers and Systems (HCAHPS) to measure patient satisfaction across inpatient and outpatient settings in compliance with the CMS requirements.⁶ At the Boston Medical Center, the NRC survey was sent to randomly selected patients with valid email addresses or phone numbers, who had no other pending NRC visit surveys or previous NRC survey responses within 180 days and whose primary language was indicated as English, Haitian Creole, Spanish, or Portuguese. Patients received the NRC survey in their primary language through phone or email and had to respond within 18 days of the visit per survey requirement.

The validated NRC survey¹⁵ consisted of 20 questions about patient experience with clinicians, clinic staff, and nurses, as well as time spent at the facility (Table 1). Six questions were identified as the most relevant for assessing patient satisfaction with the clinician: (1) likelihood of patient referral of the clinician to family and friends, (2) patient trust in the clinician with their care, (3) patient perception that the clinician listened carefully to them, (4) patient satisfaction with time spent with the clinician, (5) patient perception that the clinician knew their medical history, and (6) patient perception that the clinician treated them with courtesy and respect.

Table 1.

National Research Council Health Patient Satisfaction Survey

	SURVEY QUESTION	ANSWER FORMAT
Provider-specific	How likely would you be to recommend this provider to your family and friends?	0 to 10 Ordinal Scale 0—Not at all 10—Extremely
	Did you trust this provider with your care?	1 to 4 4-point Likert Likelihood Scale 1—No 2—Yes, somewhat 3—Yes, mostly 4—Yes, definitely
	Did this provider listen carefully to you?
	Did this provider spend enough time with you?
	Did this provider seem to know your medical history?
	Did this provider treat you with courtesy and respect?
	Did this provider give you enough information about your health and treatment?
Staff-specific	Was there good communication between the different doctors and nurses?
	Were you told when you could expect your test results?
	Were you able to schedule an appointment with this provider as soon as you needed?
	Was the staff who interacted with you friendly/courteous?
Nurse-specific	Did you have confidence and trust in the nurses treating you?
	Were you comfortable talking with nurses about your worries or concerns?
	Did nurses treat you with courtesy and respect?
	Did nurses listen carefully to you?
	Did nurses explain things in a way you could understand?
	Did you see a nurse during this visit?	Yes No
Other	How long did you wait beyond your appointment time?	Under 5 min 5–10 10–20 20–30 More than 30
	Would you be willing to join our online patient community and provide periodic feedback that will help us continue improving the care we deliver to patients?	Yes, I would No thanks
	What else would you like to say about your experience?	Open question

The table lists the 20 questions comprising the National Research Council survey that was administered to the outpatient eye clinic patients. The dark shaded cells highlight the six provider-specific questions selected in this study to assess patient satisfaction between visit types.

STUDY DATA

This study included all hybrid and standard in-person patients seen at the Boston Medical Center Eye Clinic who completed an NRC survey from the beginning of the COVID-19 pandemic on March 11, 2020, to December 31, 2021. Patient-level and visit-level data were retrieved from electronic medical records.

Patient-level data included age, sex, primary language, and self-reported race and ethnicity. Sex and race and ethnicity data were categorized based on the mutually exclusive classifications available through electronic medical records and were collected to evaluate possible demographic associations with patient satisfaction.

Visit-level data included visit type (hybrid or standard in-person), visit dates, and name of the attending ophthalmologist or optometrist. A standard in-person visit consists of a standard office appointment with a provider at the clinic. The corresponding NRC data were excluded from the analysis for a hybrid visit if an NRC survey was completed following the imaging appointment but before the virtual appointment. Only the first survey response was included if the same patient completed more than one NRC survey during the study period from March 11, 2020 to December 31, 2021.

OUTCOME METRICS

The primary outcome of this analysis was patient satisfaction with the clinician based on the question “How likely would you be to recommend this provider to your family and friends?” (0–10 ordinal scale, “Recommend”). This was selected as the primary outcome based on other NRC-based studies that consistently used this metric for assessing patient satisfaction.^9,11,16 Five additional patient satisfaction metrics were analyzed as secondary outcomes using a 1–4 Likert scale: (1) patient trust in provider (“Trust”), (2) patient perception that provider listened (“Listened”), (3) patient satisfaction with time spent with provider (“Time”), (4) patient perception that provider knew their medical history (“History”), and (5) patient perception that provider treated them with courtesy and respect (“Courtesy”).

STATISTICAL ANALYSIS

Summary statistics was reported as medians with interquartile ranges (IQRs) for continuous variables and percentages for categorical variables. For the analysis, the primary patient satisfaction outcome was modeled as both a continuous variable (0–10) and as a dichotomized binary variable using the top-box approach where responses of 0–8 represent a lack of patient satisfaction with the provider, while responses of 9 or 10 represent satisfaction with the provider. Similarly, the five secondary patient satisfaction outcomes were modeled as continuous variables (1–4) and as dichotomized variables (not satisfied [1–3] and satisfied [4]). Although the results of both continuous and dichotomized satisfaction score analyses are typically consistent, top-box results are considered more interpretable because of the significant rightward skew to patient responses typically observed in HCAHPS surveys.^7,17 The top-box approach also reflects how CMS publicly reports hospital results.¹⁸

For the comparison of demographic characteristics between hybrid and standard in-person visit types, Pearson's chi-square tests were used for categorical variables, and Mann–Whitney U tests were used for continuous variables. Two-sample t-tests were used to compare numeric patient satisfaction outcomes between the visit types. Pearson's chi-square tests and a bivariate logistic regression were used to compare dichotomized patient satisfaction outcomes between visit types. Separate chi-square tests were used to examine the effects of age, sex, primary language, and race and ethnicity within and between visit types. Fisher's exact test was performed in cases with few observations to confirm statistical significance.

A multivariable logistic regression model was applied to estimate the association between each visit type variable and the primary outcome while simultaneously controlling for demographic variables. Subgroup analyses were performed using the following categories: age (0–17, 18–44, 45–64, 65+), sex (male, female), primary language, and race and ethnicity. Statistical significance was set at p < 0.05. Data analyses were conducted using BlueSky Statistics Version 10.0 (R Package Version 8.0) and IBM SPSS Version 27.0.

Results

STUDY SAMPLE

From the 13,011 ophthalmic or optometric patients surveyed using the NRC from March 11, 2020, to December 31, 2021, a total of 4,977 responses were collected, corresponding to an overall response rate of 38% (compared to a 34% average response rate across all surveyed services at the hospital, Boston Medical Center). After removing repeat survey responses from identical patients (389 responses), surveys with missing data (913 responses), inapplicable visit type (147 responses), or visit date (89 responses), 3,439 unique patient response surveys were identified, corresponding to 49 (1%) hybrid visits and 3,390 (99%) standard in-person visits occurring between March 11, 2020, and December 31, 2021. Patient-level demographic characteristics of the hybrid and standard cohorts are summarized in Table 2.

Table 2.

Patient Demographics by Visit Type for National Research Council Health Survey responses from March 11, 2020 to December 31, 2021 (n = 3,439)

	HYBRID		STANDARD IN-PERSON		p^a
	N = 49	%	N = 3,390	%	p^a
Sex
Female	29	59	1,959	58	0.844
Male	20	41	1,431	42
Age
Median (IQR)	65	(56–73)	60	(46–70)	0.003^*
Age group					0.013^**
0–17	1	2	200	6	0.253
18–44	2	4	580	17	0.016
45–64	18	37	1,323	39	0.744
65+	28	57	1,287	38	0.0061^***
Race and ethnicity
Non-Hispanic Black/African American	27	55	1,473	43	0.448
Hispanic/Latino	7	14	641	19
Non-Hispanic White	7	14	603	18
Asian	0	0	61	2
Other	0	0	99	3
Declined/unavailable	8	16	513	15
Language
English	33	67	2,074	61	0.747
Spanish	11	22	843	25
Haitian Creole	4	8	264	8
Portuguese	1	2	138	4
Other	0	0	71	2

p < 0.01.

p < 0.05.

***

p < 0.05/8 = 0.00625 (Bonferroni-corrected).

p Values were calculated using Mann–Whitney U tests for continuous variables (age) and chi-square tests for categorical variables (sex, age group, race and ethnicity, and language). A Bonferroni-corrected significant level of 0.05/8 = 0.00625 was used for subcategories of age group. Language category Other included languages primarily spoken by <20 respondents each (Cape Verdean, Arabic, French, Vietnamese, Amharic, Tigrinya, and others). Brazilian Portuguese was coded as Portuguese. Race and ethnicity category Other comprised races and ethnicities that each had a sample size of fewer than 13 respondents (American Indian/Native American, Middle Eastern, Native Hawaiian/Pacific Islander, and multiple race and ethnicity groups).

A comparison between the hybrid and standard in-person cohorts showed no statistically significant differences in sex, primary language, or race and ethnicity. However, the distribution of age varied significantly between the two cohorts, with the median age of hybrid respondents (65, IQR 56–73) being significantly older than standard in-person respondents (60, IQR 46–70), (p = 0.003). Among the age groups, there were significantly more respondents aged 65 years or older in the hybrid cohort (57%) than in the standard in-person cohort (38%), (p = 0.013).

PATIENT SATISFACTION BETWEEN VISIT TYPES

Evaluating patient satisfaction as continuous variables showed no significant differences between the hybrid and standard in-person cohorts for the primary and secondary outcomes. For the primary patient satisfaction outcome (likelihood of recommending the ophthalmologist or optometrist to friends or family), the hybrid cohort responded with a mean score of 8.72 out of 10 ± 2.82, and the standard in-person cohort responded with a mean score of 9.01 ± 2.24. These averages reflected high patient satisfaction in the Recommend satisfaction outcome, which did not significantly differ between visit types (p = 0.427). The five secondary patient outcomes on a 1–4 Likert scale also did not show a difference between visit types. Further details are provided in Table 3.

Table 3.

Patient Satisfaction Outcomes by Visit Type for National Research Council Health Survey responses from March 11, 2020 to December 31, 2021 (n = 3,439)

	HYBRID			STANDARD IN-PERSON			p ^a
	N	MEAN	SD	N	MEAN	SD	p ^a
Recommend (0–10)	39	8.72	2.82	2,934	9.01	2.24	0.427
Trust (1–4)	46	3.20	1.03	3,265	3.36	0.92	0.220
Listened (1–4)	48	3.13	1.10	3,323	3.31	0.95	0.184
Time (1–4)	44	3.39	0.95	3,154	3.37	0.90	0.910
History (1–4)	45	3.27	1.01	3,202	3.23	0.94	0.769
Courtesy (1–4)	45	3.40	0.94	3,182	3.48	0.86	0.538

p Values were calculated using two-sample t-tests.

SD, standard deviation.

Evaluating patient satisfaction as dichotomized measures according to the top-box approach yielded similar results for the primary and secondary outcomes. On the Recommend primary outcome, hybrid visits were rated similarly to standard in-person visits: 31 of the 39 (79%) hybrid visit patients who answered the Recommend question indicated that they were highly likely to recommend their provider to family or friends (9 or 10 on a 0–10 scale), compared to 2,396 of the 2,934 (82%) standard in-person visit patients. A bivariate logistic regression revealed no statistical differences in any patient satisfaction outcomes between hybrid and standard in-person visits. Further details are provided in Table 4.

Table 4.

Dichotomized Patient Satisfaction Outcomes by Visit Type for National Research Council Health Survey Responses from March 11, 2020 to December 31, 2021 (n = 3,439)

	HYBRID ( N = 49)		STANDARD IN-PERSON ( N = 3,390)		BIVARIATE LOGISTIC REGRESSION WITH STANDARD AS REFERENCE
	SATISFIED (%)	NOT SATISFIED (%)	SATISFIED (%)	NOT SATISFIED (%)	p	OR (95% CI)
Recommend (0–10)	31 (79)	8 (21)	2,396 (82)	538 (18)	0.728	0.870 (0.418–2.042)
Trust (1–4)	26 (57)	20 (43)	2,062 (63)	1,203 (37)	0.356	0.758 (0.423–1.380)
Listened (1–4)	28 (58)	20 (42)	2,027 (61)	1,296 (39)	0.707	0.895 (0.505–1.616)
Time (1–4)	30 (68)	14 (32)	1,969 (62)	1,185 (38)	0.435	1.290 (0.694–2.516)
History (1–4)	27 (60)	18 (40)	1,706 (53)	1,496 (47)	0.371	1.315 (0.727–2.437)
Courtesy (1–4)	31 (69)	14 (31)	2,218 (70)	964 (30)	0.906	0.962 (0.520–1.873)

CI, confidence interval; OR, odds ratio.

Counts and percentages for satisfied and unsatisfied patients using top-box approach are presented in the table. Satisfied corresponds to a score of 9 or 10 on a 0–10 scale outcome or a score of 4 on a 1–4 Likert outcome. Not satisfied corresponds to a score of 0–8 on a 0–10 scale or a score of 1–3 on a 1–4 Likert scale. p Values, ORs, and 95% CIs were calculated from a bivariate logistic regression on each dichotomized patient satisfaction outcome with standard in-person visit type as reference.

PATIENT SATISFACTION BY DEMOGRAPHICS

We calculated the correlation of the demographic variables with a patient recommending the clinician to family and friends within each visit type cohort (Table 5).

Table 5.

Descriptive Analysis of the Dichotomized Primary Outcome (Recommend Patient Satisfaction) by Patient-Level Characteristics (n = 2,973)

	HYBRID ( N = 39)			STANDARD IN-PERSON ( N = 2,934)
	SATISFIED (%)	NOT SATISFIED (%)	p HYBRID	SATISFIED (%)	NOT SATISFIED (%)	p STANDARD	p HYBRID VS. STANDARD
Sex			1.000			0.280
Female	18 (78)	5 (22)		1,373 (81)	322 (19)		0.788
Male	13 (81)	3 (19)		1,023 (83)	216 (17)		0.750
Age group			0.012^*			0.902
0–17	1 (100)	0 (0)	1.000	133 (80)	33 (20)		1.000
18–44	2 (100)	0 (0)	1.000	413 (82)	88 (18)		1.000
45–64	16 (100)	0 (0)	0.012	948 (82)	210 (18)		0.092
65+	12 (60)	8 (40)	0.003^*	902 (81)	207 (19)		0.037^*
Race and ethnicity			0.490			0.008^**
Non-Hispanic Black or African American	14 (70)	6 (30)		983 (79)	261 (21)	<0.001^**	0.404
Hispanic/Latino	6 (86)	1 (14)		474 (85)	86 (15)	0.041	1.000
Non-Hispanic White	5 (100)	0 (0)		472 (85)	83 (15)	0.021	1.000
Asian	0 (0)	0 (0)		41 (79)	11 (21)	0.591	NA
Other	0 (0)	0 (0)		70 (81)	16 (19)	0.941	NA
Language			1.000			<0.001^***
English	18 (78)	5 (22)		1,508 (81)	353 (19)	0.244	0.788
Spanish	9 (82)	2 (18)		630 (86)	104 (14)	<0.001^**	0.662
Haitian Creole	3 (75)	1 (25)		128 (72)	50 (28)	<0.001^**	1.000
Portuguese	1 (100)	0 (0)		96 (82)	21 (18)	0.912	1.000
Other	0 (0)	0 (0)		34 (77)	10 (23)	0.448	NA

p < 0.05, ^** p < 0.01, ^*** p < 0.001 for categories (Bonferroni-corrected for subcategories).

For the hybrid cohort, p values were calculated from Fisher's exact tests to compare satisfaction across subgroups of each demographic category. For the standard in-person cohort, p values were calculated from chi-square tests. For any categories within each visit cohort showing a significant difference, a post hoc comparison with Bonferroni adjustment was used. The Bonferroni-corrected significant level was 0.05/8 = 0.00625 for age group and 0.05/10 = 0.005 for race and ethnicity and language. For hybrid versus standard, p values were calculated from Fisher's exact tests to compare satisfaction between the cohorts among patients of each demographic subcategory. Race and ethnicity category Declined/Unavailable was excluded from the analysis.

Age was significantly associated with satisfaction in the hybrid cohort, with 19 of the 19 (100%) hybrid visit patients in the below-65 age groups providing a high satisfaction score compared to only 12 of the 20 (60%) hybrid visit patients in the 65+ age group (p = 0.003, Fisher's exact test). Among all patients aged 65 and above, satisfaction was significantly lower with hybrid visits than with standard in-person visits (χ ² [1, N = 1,129] = [5.800], p = 0.037, confirmed by Fisher's exact test). No association with age was observed in the standard in-person cohort.

In the standard in-person cohort, race and ethnicity and primary language were significantly associated with differences in the Recommend satisfaction outcome. Specifically, 983 of the 1,244 (79%) in-person patients who identified as non-Hispanic Black or African American were highly likely to recommend their clinician, compared to 472 of the 555 (85%) in-person patients who identified as non-Hispanic White (p = 0.008).

In addition, 128 of the 178 (72%) in-person patients with Haitian Creole as primary language were highly likely to recommend their provider, compared to 630 of the 734 (86%) in-person patients with Spanish as primary language (p < 0.001). No associations with race and ethnicity or primary language reached statistical significance in the hybrid cohort.

Finally, sex was not significantly associated with the Recommend patient satisfaction outcome regardless of visit type.

To further elucidate racial/ethnic and language differences, a multivariate logistic regression model (Table 6) that simultaneously controlled for visit type and all demographic patient-level data showed that patients were more likely to provide a positive Recommend satisfaction score if they were non-Hispanic White (odds ratio [OR], 1.399; 95% confidence interval [CI], 1.060–1.862, p = 0.019) and less likely to provide a positive Recommend satisfaction score if their primary language was Haitian Creole (OR, 0.663; 95% CI, 0.456–0.976, p = 0.033). Patients were more likely to provide a positive Recommend satisfaction score if their primary language was Spanish with a strong trend that did not reach statistical significance (OR, 1.485; 95% CI, 0.973–2.276, p = 0.068).

Table 6.

Factors Associated with the Dichotomized Recommend Patient Satisfaction Outcome Among Patients Completing Any Visit Type for National Research Council Health Survey Responses from March 11, 2020 to December 31, 2021 (n = 3,439)

	TOTAL POPULATION ( N = 2,973)		MULTIVARIATE LOGISTIC REGRESSION, OR (95% CI) ADJUSTED	p
	SATISFIED (%)	NOT SATISFIED (%)	MULTIVARIATE LOGISTIC REGRESSION, OR (95% CI) ADJUSTED	CATEGORY	SUBCATEGORY
Visit type				0.612
Hybrid	31 (79)	8 (21)	Ref.
Standard in-person	2,396 (82)	538 (18)	0.841 (0.378–2.132)		0.691
Sex				0.310
Female	1,391 (81)	327 (19)	Ref.
Male	1,036 (83)	219 (17)	1.105 (0.899–1.360)		0.344
Age				0.590
0–17	134 (80)	33 (20)	Ref.
18–44	415 (83)	88 (17)	1.388 (0.809–2.322)		0.222
45–64	964 (82)	210 (18)	1.220 (0.739–1.946)		0.419
65+	914 (81)	215 (19)	1.138 (0.69–1.814)		0.599
Race and ethnicity				0.173
Black or African American	997 (79)	267 (21)	Ref.
Hispanic/Latino	480 (85)	87 (15)	1.018 (0.67–1.563)		0.934
Non-Hispanic White	477 (85)	83 (15)	1.399 (1.060–1.862)		0.019^*
Asian	41 (79)	11 (21)	0.923 (0.479–1.928)		0.819
Other	70 (81)	16 (19)	0.993 (0.574–1.818)		0.981
Language				0.002^**
English	1,526 (81)	358 (19)	Ref.
Spanish	639 (86)	106 (14)	1.485 (0.973–2.276)		0.068
Haitian Creole	131 (72)	51 (28)	0.663 (0.456–0.976)		0.033^*
Portuguese	97 (82)	21 (18)	1.108 (0.550–2.486)		0.787
Other	34 (77)	10 (23)	0.903 (0.424–2.156)		0.804

p < 0.05, ^** p < 0.01.

A multivariate logistic regression controlled for visit type and all four demographic characteristics simultaneously. Category p values correspond to an analysis of deviance. Subcategory p-values correspond to regression coefficients.

Discussion

Overall, we found that patients using hybrid ophthalmology telemedicine visits reported high patient satisfaction scores without significant differences compared to patients using standard-of-care in-person visits, as measured by the NRC Health Patient Survey in an academic hospital-based eye clinic in Boston, Massachusetts. However, age-based disparities were noted in the hybrid cohort, with older patients reporting lower satisfaction than younger patients. Our results also suggest that there are race/ethnicity- and language-based disparities in patient satisfaction in the in-person cohort, which did not reach statistical significance in the hybrid cohort.

To our knowledge, no patient satisfaction studies have used the NRC or a similar patient satisfaction survey tool in ophthalmic telemedicine. Literature in nonophthalmic specialties shares our findings of high patient satisfaction scores but varied in results when comparing telemedicine with in-person care. Telemedicine services in voice therapy (average likelihood of recommending the provider to friends or family of 9.3/10 compared to our finding of 8.7)⁹ and pediatric urology (90% highly and extremely likely to recommend the provider compared to our finding of 79%)¹⁶ both demonstrated high patient satisfaction scores based on NRC surveys. Another NRC-based study focusing on otolaryngology clinics¹¹ reported a significantly higher preference for in-person care compared to telemedicine, as measured by our primary outcome (73% satisfied telemedicine patients and 93% satisfied in-person patients, compared to our finding of 79% satisfied hybrid patients and 82% satisfied in-person patients).

One possible explanation for the smaller difference we found in patient satisfaction between the hybrid and standard in-person visits is that our hybrid ophthalmology telemedicine model pairs the surveyed virtual appointment with a prior in-person imaging appointment with a trained technician, which more closely resembles a standard in-person visit than a virtual-only telemedicine visit.

Our study found age-based differences in the hybrid cohort (older patients reported lower satisfaction than younger patients), which were not observed in the in-person cohort. This is consistent with prior studies in both ophthalmic and nonophthalmic specialties that found similar disparities in the older population's use of telemedicine.^10,19
–21 Although some studies reported a lower likelihood of older patients completing an ophthalmic video visit but not a phone-based visit,^19,20 our findings show that even with a hybrid model that offers a phone-based option, the disparity in satisfaction was still notable among older patients. Higher barriers to telemedicine accessibility are associated with older age, including technology access, telemedicine literacy, and more severe motor and sensory impairment, which might help explain this age-based disparity.

Among older patients, efforts could be made to include caregivers on hybrid visits who could help bridge this accessibility gap. In addition, beyond infrastructural or physical barriers, some studies have attributed this disparity to generational differences in attitudes toward telemedicine and inertia against change.^{10,20,22
–24} Targeted promotion and education of telehealth services could help increase older patients' satisfaction with the hybrid model.²⁰

Demographic factors other than age have a mixed association with patient satisfaction in the literature.^19
–21,25 In our diverse patient population, race/ethnicity- and language-based differences were observed in the in-person cohort but did not reach statistical significance in the hybrid cohort. Further studies with larger hybrid sample sizes could help elucidate these associations.

When considering all demographic variables and visit type, non-Hispanic White patients reported higher patient satisfaction than non-Hispanic Black or African American patients, and patients who primarily speak Haitian Creole reported lower patient satisfaction than primary English speakers. Previous studies have found that barriers to telemedicine care, including language, digital literacy, and infrastructural barriers, disproportionally impact vulnerable populations.^19,21,26 Of note, Hispanic or Latino patients reported higher patient satisfaction both in the in-person cohort and across the total patient population, which has been previously found in one study set in an urban pediatric emergency department²⁵ but not in telemedicine ophthalmology.²¹ These findings suggest the efficiency of implementing interpreter and translation services in telemedicine and reinforce the recent calls to prioritize adequate language interpretation services on ophthalmic telemedicine platforms.²⁷

Our study was limited by missing data and the sample size. Missing data included incomplete NRC surveys. The small number of surveyed hybrid patients restricted our ability to further analyze the effects of visit type and certain demographic factors and did not allow for a separate multivariable analysis of the hybrid cohort. Because of the difference between the hybrid and in-person sample sizes, the multivariable logistic regression model was disproportionally powered by the larger in-person cohort. An additional weakness is that the NRC Patient Health Survey has not been specifically validated as a measure of patient satisfaction with telemedicine, and as such some of the survey questions we analyzed may not apply as well to hybrid visits or fully elucidate their advantages and disadvantages.

In addition, we were limited by the single-center, retrospective design focusing on a hospital-based eye clinic in Boston, Massachusetts; therefore, generalizability to other health care systems may be limited. As telemedicine continues to be utilized for health care delivery,²⁸ we encourage other health care systems to review their patient satisfaction data to ensure quality care and patient satisfaction.

This hybrid ophthalmology telemedicine model was developed during the COVID-19 pandemic to provide clinical care to patients with stable nonprocedural complaints. While our previously published study showed that this model can provide good care with limited adverse events in patients with a variety of chief complaints,⁵ our current study shows that the model is also associated with high patient satisfaction on par with standard in-person visits. A positive doctor-patient relationship improves patient outcomes; therefore, even though the COVID-19 pandemic is waning, a hybrid ophthalmology telemedicine model may serve as an effective option for the delivery of care for certain patients and eye clinics.

Footnotes

Acknowledgments

The authors thank Frank Vavrek, MPH (Boston Medical Center, Boston, MA) and Abbie Grim (NRC Health, Lincoln, NE) for their assistance in providing the raw data for the project.

Authors' Contributions

M.D.: writing—original draft, methodology, formal analysis, data curation; M.M.A.: methodology, data curation, formal analysis; N.S.: methodology, data curation; H.C.: methodology, formal analysis; D.C.M.: methodology, data curation; M.L.S.: writing—review and editing; S.N.: writing—review and editing; N.H.S.: writing—review and editing; M.D.: writing—review and editing; X.C.: supervision, methodology, formal analysis, conceptualization, writing—review and editing, project administration.

Disclosure Statement

No competing financial interests exist.

Funding Information

This research was supported by the Medical Student Summer Research Program at the Boston University School of Medicine.

References

Parikh

, Armstrong

, Liou

, et al. Advances in telemedicine in ophthalmology. Semin Ophthalmol, 2020; 35:210–215.

Sommer

, Blumenthal

. Telemedicine in ophthalmology in view of the emerging COVID-19 outbreak. Graefes Arch Clin Exp Ophthalmol, 2020; 258:2341–2352.

Raparia

, Husain

. COVID-19 launches retinal telemedicine into the next frontier. Semin Ophthalmol, 2021; 36:258–263.

Portney

, Zhu

, Chen

, et al. COVID-19 and use of teleophthalmology (CUT Group): Trends and diagnoses. Ophthalmology, 2021; 128:1483–1485.

Sanayei

, Albrecht

, Martin

, et al. Outcomes of a hybrid ophthalmology telemedicine model for outpatient eye care during COVID-19. JAMA Netw Open, 2022; 5:e2226292.

HCAHPS: Patients' Perspectives of Care Survey |

CMS

. Available from: https://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/HospitalQualityInits/HospitalHCAHPS (Last accessed: October 5, 2022).

Davis-Dao

, Ehwerhemuepha

, Chamberlin

, et al. Keys to improving patient satisfaction in the pediatric urology clinic: A starting point. J Pediatr Urol, 2020; 16:377–383.

Hospital Value-Based Purchasing Program |

CMS

. Available from: https://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/HospitalQualityInits/Hospital-Value-Based-Purchasing- (Last accessed: October 5, 2022).

Gray

, Michael

, Hoffmeister

, et al. Patient satisfaction with virtual vs in-person voice therapy. J Voice, 2022;S0892199722000819. [Epub ahead of print]; doi: 10.1016/j.jvoice.2022.03.011

10.

Kruse

, Krowski

, Rodriguez

, et al. Telehealth and patient satisfaction: A systematic review and narrative analysis. BMJ Open, 2017; 7:e016242.

11.

Itamura

, Tang

, Higgins

, et al. Comparison of patient satisfaction between virtual visits during the COVID-19 pandemic and in-person visits pre-pandemic. Ann Otol Rhinol Laryngol, 2021; 130:810–817.

12.

Pappot

, Taarnhøj

, Pappot

. Telemedicine and e-health solutions for COVID-19: Patients' perspective. Telemed J E Health, 2020; 26:847–849.

13.

Newman-Casey

, De Lott

, Cho

, et al. Telehealth-based eye care during the COVID-19 pandemic: Utilization, safety, and the patient experience. Am J Ophthalmol, 2021; 230:234–242.

14.

NRC Health Partners. NRC Health. Available from: https://nrchealth.com/partners (Last accessed: July 14, 2022 ).

15.

NRC Health Consumer Assessment of Healthcare Providers and Systems. NRC Health. Available from: https://nrchealth.com/solutions/cahps (Last accessed: October 5, 2022 ).

16.

Holzman

, Davis-Dao

, Khoury

, et al. Telemedicine and patient satisfaction during the COVID-19 pandemic: A case–control study of outpatient pediatric urology patients. J Child Health Care, 2021;13674935211058272. [Epub ahead of print]; doi: 10.1177/13674935211058272.

17.

Jha

, Orav

, Zheng

, et al. Patients' perception of hospital care in the United States. N Engl J Med, 2008; 359:1921–1931.

18.

Elliott

, Zaslavsky

, Goldstein

, et al. Effects of survey mode, patient mix, and nonresponse on CAHPS^® Hospital Survey Scores. Health Serv Res, 2009; 44:501–518.

19.

Aziz

, Moon

, Parikh

, et al. Association of patient characteristics with delivery of ophthalmic telemedicine during the COVID-19 pandemic. JAMA Ophthalmol, 2021; 139:1174–1182.

20.

Chen

, Andoh

, Nwanyanwu

. Socioeconomic and demographic disparities in the use of telemedicine for ophthalmic care during the COVID-19 pandemic. Ophthalmology, 2022; 129:15–25.

21.

Adeli

, Bloom

. Implementing telemedicine visits in an underserved ophthalmology clinic in the COVID-19 era. J Prim Care Community Health, 2021; 12:2150132721996278.

22.

Anthony

, Campos-Castillo

, Lim

. Who isn't using patient portals and why? Evidence and implications from a national sample of US adults. Health Aff (Millwood), 2018; 37:1948–1954.

23.

Fischer

, David

, Crotty

, et al. Acceptance and use of health information technology by community-dwelling elders. Int J Med Inf, 2014; 83:624–635.

24.

Yusif

, Soar

, Hafeez-Baig

. Older people, assistive technologies, and the barriers to adoption: A systematic review. Int J Med Inf, 2016; 94:112–116.

25.

Heyming

, Donaldson

, Ehwerhemuepha

, et al. Multivariable analysis of patient satisfaction in the pediatric emergency department. Pediatr Emerg Care, 2022; 38:e544–e549.

26.

VanderBeek

. Telemedicine and the exacerbation of health care disparities. JAMA Ophthalmol, 2021; 139:1182–1183.

27.

Schmuter

. Language barriers in teleophthalmology. Eye, 2022; 36:1335.

28.

Mosenia

, Li

, Seefeldt

, Seitzman

, Sun

, Kim

. Longitudinal use of telehealth during the COVID-19 pandemic and utility of asynchronous testing for subspecialty-level ophthalmic care. JAMA Ophthalmol, 2023; 141(1):56–61.