Abstract
Objective
The objective of this study is to evaluate patient and provider satisfaction with telemedicine encounters across 3 otolaryngology practices.
Study Design
Cross-sectional survey.
Setting
A military community hospital, an academic military hospital, and a nonmilitary academic center.
Methods
A telephone-based survey of patients undergoing telemedicine encounters for routine otolaryngology appointments was performed between April and July 2020. Patients were asked about their satisfaction, the factors affecting care, and demographic information. A provider survey was emailed to staff otolaryngologists. The survey asked about satisfaction, concerns for reimbursement or liability, encounters best suited for telemedicine, and demographic information. The results were analyzed with descriptive statistics and a multivariable logistic linear regression model to determine odds ratios.
Results
A total of 325 patients were surveyed, demonstrating high satisfaction with telemedicine (average score, 4.49 of 5 [best possible answer]). Patients perceived “no negative impact” or “minor negative impact” on the encounter due to the lack of a physical examination or face-to-face interaction (1.86 and 1.95 of 5, respectively). High satisfaction was consistent across groups for distance to travel, age, and reason for referral. A total of 25 providers were surveyed, with an average satisfaction score of 3.44 of 5. Providers reported “slight” to “somewhat” concern about reimbursement (40%) and liability (32%).
Conclusion
Given patients’ and providers’ levels of satisfaction, there is likely a role for telemedicine in otolaryngology practice that may benefit patient care independent of the COVID-19 pandemic.
The year 2020 has been marked by the COVID-19 pandemic, creating a worldwide public health emergency. In response, the American Academy of Otolaryngology–Head and Neck Surgery, in a March statement, recommended against routine, elective, and nonurgent procedures to prevent exposure to patients, providers, and assistants while preserving resources during the pandemic. 1 As a result, telemedicine was widely adopted to facilitate access to care. This posed significant challenges to clinicians, as many otolaryngology practices were not prepared to implement telemedicine and had to develop protocols quickly. 2
Telemedicine has been demonstrated to increase provider efficiency in otolaryngology practices and is convenient for patients. 3 Otolaryngology relies on testing that may de-emphasize physical examination, such as audiometry and radiographic imaging. Therefore, certain types of encounters may favor telemedicine with certain types of encounters. However, examinations such as otoscopy and endoscopy would remain difficult in the telemedicine environment. 3 McCool and Davies found that 62% of otolaryngologic encounters were eligible for telemedicine, with the remainder of patients requiring in-clinic diagnostic or therapeutic procedures. 4 Rimmer et al evaluated patient satisfaction for telemedicine in a head and neck cancer practice; however, the majority (70%) of the study consisted of postoperative telephone consults and did not assess provider satisfaction. 5 The results of their study revealed that 95% of patients were satisfied with telephone consults for postoperative visits, although patient dissatisfaction with wait times and technical issues remained.
To our knowledge, no studies have examined patient- and provider-specific satisfaction with telemedicine in otolaryngology practice. The primary aim of this study is to assess satisfaction of otolaryngology telemedicine encounters among all patient encounters. Secondary aims were to assess variables that predicted higher or lower satisfaction, including distance to travel, age, and reason for referral, as well as to assess for concerns for diagnostic concordance and reimbursement.
Method
Institutional Review Board approval was obtained from the Department of Research Programs at the Walter Reed National Military Medical Center (WRNMMC-EDO-2020-0484) and at the Ochsner Clinic Foundation (2020.200). Study participants consisted of patients presenting for routine clinical care for telemedicine consultation at 1 of 3 tertiary otolaryngology practices between March 2020 and July 2020.
Surveys were obtained across 3 distinct practices: a military community hospital (Fort Belvoir Community Hospital), an academic military hospital (Walter Reed National Military Medical Center), and a nonmilitary academic center (Ochsner Medical Center). Respondents were asked to complete a 10-question survey to assess patient satisfaction with telemedicine encounters. The survey was conducted telephonically by an investigator not involved directly in that patient’s care, with results obtained either at the time of the encounter or within 1 month of the appointment. The patient survey consisted of demographic questions (age, sex, geographic location, and reason for the visit) and questions regarding satisfaction with the telemedicine platform (see Figure 1 ). The encounter was categorized by patient-presenting complaint: ear (including hearing and dizziness), head and neck, nose, sinus, and other. For questions related to satisfaction, patients were asked to utilize a 5-point Likert scale corresponding to very dissatisfied, dissatisfied, neither satisfied nor dissatisfied, satisfied, or very satisfied. For questions rating impact, patients used a 5-point Likert scale corresponding to no negative impact, minor negative impact, neutral, moderate negative impact, and major negative impact. At the conclusion of the survey, patients were given the opportunity to provide feedback or comments on telemedicine.
Patient satisfaction survey.
The 12-question provider survey was distributed by email to all otolaryngologists practicing at Fort Belvoir Community Hospital, Walter Reed National Military Medical Center, and Ochsner Medical Center. Included in the provider survey were demographic questions: number of years in clinical practice since the completion of training, subspecialty fellowship training, geographic location, age, and sex (see Figure 2 ). Provider respondents were queried about their satisfaction with telemedicine, impact on decision making, concern for reimbursement, concern for increased risk of malpractice or misdiagnosis, likelihood to have performed a procedure, patient’s access to at-home interface, and which patients would one consider offering telemedicine encounters.
Provider satisfaction survey.
Survey responses were collected electronically, organized, and analyzed with Microsoft Excel. No personal identifying information was collected, ensuring that the surveys remained anonymous for data analysis.
Statistical Analysis
Demographics and survey results were described with means and standard deviations or frequencies and percentages. Multivariable logistic regression was used to evaluate potential independent predictors for high patient satisfaction, defined as a response of “very satisfied” with the telehealth visit. A P value <.05 was considered statistically significant. Analyses were done with R version 3.6.1 (R Foundation).
Results
A total of 325 patient satisfaction surveys and 25 provider satisfaction surveys were completed. Characteristics of the patient population are presented in Table 1 . Patients treated at Fort Belvoir Community Hospital accounted for 44.6% of the surveys; Walter Reed National Military Medical Center, 18.8%; and Ochsner Medical Center, 36.6%. A plurality of patients was between 40 and 59 years old (45.5%). The sex distribution was evenly distributed, with 50.2% of respondents being male. The majority of patients (68.9%) lived <20 miles from their provider’s office, and few patients (12.0%) lived >50 miles from their provider’s office.
Characteristics of Patient Study Population (N = 325).
Patient responses to the survey are presented in Table 2 . Patients cited an overall satisfaction with telemedicine between “satisfied” and “very satisfied” (4.49 of 5 [best possible answer]).
Patient Responses to Telehealth Survey.
Patients felt “no negative impact” to a “minor negative impact” on their care by not seeing their provider face-to-face (1.86 of 5 [best possible answer, 1]). Similarly, patients felt no negative impact to a minor negative impact on their care due to a lack of physical examination (1.95 of 5). On average, patients felt no negative impact to a minor negative impact on their ability to access their provider (1.46 of 5). The majority (78.8%) of patients felt that their provider had all the information needed to make a diagnosis and treatment.
Nearly two-thirds of patients (210, 64.6%) indicated that they were “very satisfied” with the telemedicine encounter. Those patients who did not rate their experience as very satisfied (n = 115, 35.4%) reported an average satisfaction score of 3.56 of 5, corresponding to “neutral” or “satisfied.” Patients demonstrating lower satisfaction scores perceived a minor negative impact to neutral impact on their care from a lack of face-to-face interaction (score, 2.49 of 5 [best possible answer, 1]), as compared with no negative impact to a minor negative impact (score, 1.52 of 5) in very satisfied patients. Patients in the lower satisfaction group felt that their provider had all the information needed to make a diagnosis and treatment plan less frequently (n = 74, 64.3%) than the “very satisfied” cohort (n = 182, 86.7%).
Patients were given the opportunity to make comments or provide general, open-ended feedback. A total of 66 patients provided comments: 10 felt that telemedicine should continue after the resolution of COVID-19; 7 commented on increased access to care; and 4 remarked that they had difficulty hearing.
Table 3 details the characteristics of the provider population. A total of 25 otolaryngologists completed the survey. The length of time that providers had been in practice varied, with the majority reporting <5 years of experience (n = 9, 36%). The majority of respondents were male (n = 17, 68%) and between the ages of 30 and 39 years (n = 13, 52%). Most providers practiced at Ochsner Medical Center (n = 18, 72%). All otolaryngology subspecialties were reflected in the survey, with the exception of sleep surgery. The largest proportion of respondents represented general otolaryngology (n = 9, 36%).
Characteristics of Provider Study Population (N = 25).
Results of the provider survey are presented in Table 4 . On average, providers responded that they were “neither satisfied nor dissatisfied” to “satisfied” (3.44 of 5 [best possible answer, 5]) with telemedicine. Providers felt a “neutral impact” on decision making because of the lack of physical touch or examination during these encounters (3.12 of 5 [best possible answer, 1]). Providers were “slightly” to “somewhat concerned” about reimbursement (2.28 of 5 [best possible answer, 1]) and increased risk for malpractice and misdiagnosis (2.56 of 5). Most providers felt a “likely chance” that they would have performed a procedure (eg, flexible laryngoscopy) if the encounter had been face-to-face (3.84 of 5). A total of 96% (n = 24) of providers stated that patients did not have access to an at-home interface (eg, video otoscope). The majority of providers felt a role for telemedicine going forward (n = 24, 96%), with 40% (n = 10) responding that it would be appropriate for all patients, including new, established, and postoperative patients.
Provider Responses to Telehealth Survey (N = 25).
Table 5 shows a multivariable logistic regression model with patient characteristics as the independent variables and with highest satisfaction with the telehealth visit (“very satisfied”) as the dependent variable. Variance inflation factors <2 indicated no concern of multicollinearity. High satisfaction was consistent across age groups, sex, reason for visit, and clinic location.
Multivariable Logistic Regression Model of Patient Characteristics (Independent Variables) and Highest Satisfaction With the Telehealth Visit (Dependent Variable). a
Abbreviations: OR, odds ratio; Ref, reference.
The reference group for the dependent variable is defined as a response <5 on question 2 (satisfaction); the highest satisfaction score is 5.
Patients reporting a distance from their provider of 21 to 50 miles demonstrated an association with decreased overall satisfaction (odds ratio, 0.44; 95% CI, 0.24-0.82; P = .01) when compared with patients 0 to 20 miles and ≥50 miles from their provider.
Discussion
Our study shows that patients had an overall positive experience with telemedicine, with only minor negative impacts related to not being able to meet with their providers in person, not receiving a physical examination, or feeling that there was a limited ability to access providers. Current literature has shown that telemedicine has multiple benefits to patients and providers. According to a study by Doshi et al, “healthcare systems leveraging telemedicine for patient care will gain several advantages, including workforce sustainability, reduction of provider burnout, limitation of provider exposure, and reduction of personal protective equipment (PPE) waste.” 6 Sandberg et al found telemedicine to have many benefits outside the context of a global pandemic; their study showed that patients had “opportunities for more frequent contact . . . and increased accessibility by those frequently underserved.” 7
Satisfaction
In several studies, patients have reported high satisfaction with telemedicine consults.8-10 Made et al used a questionnaire to assess patient satisfaction; on a 6-point scale, surveyed patients had an average rating of 5.7, corresponding to high satisfaction. 8 Similarly, Mishra et al used a questionnaire among a cohort of 34 patients after undergoing thyroidectomy and parathyroidectomy and found that all patients reported telemedicine satisfaction as “very high” or “excellent.”9,10
Our study found that high levels of satisfaction persisted across age groups. Van den Brink et al had a similar finding in which an older population with decreased computer literacy was not deterred from telemedicine. 11 These patients reported increased access to care, which improved their feeling of security and quality of life as compared with a control group. 11
Several patients who completed our survey remarked that telemedicine should continue even after the COVID-19 pandemic has subsided. In a study examining telemedicine consultation where general practitioners performed endoscopic examinations that were later forwarded to otolaryngologists, most patients were satisfied with telemedicine and open to further teleconsultations from otolaryngologists. 12 This may be due to feeling more connected and engaged during these visits, as described by Stalfors et al. In a direct comparison between face-to-face encounters and telemedicine for patients with head and neck cancer, the face-to-face group reported that it felt “as if everybody was talking about me, but not to me.” 9
The results of the patient survey found high levels of satisfaction across all chief complaints. Data from Van den Brink et al study corroborate this, showing that patients with head and neck complaints had high odds of being very satisfied with their encounter. 11
Providers had a less enthusiastic response to telemedicine, corresponding to an answer between “satisfied” and “neither satisfied nor dissatisfied.” Similar results were seen in studies by Van Den Brink et al and Whitten et al.13,14 The former study used an electronic information system to provide telemedicine communications between patients with head and neck cancer and general practitioners involved on a multidisciplinary team. With this form of telemedicine, the general practitioners’ satisfaction rating was 5.6 of 10. While our study did not use this model, it still demonstrated a level of reluctance toward telemedicine. There are many potential reasons for providers to look unfavorably upon telemedicine. Telemedicine reduces the interpersonal interactions that a face-to-face encounter has. Counseling changes as providers can no longer rely on facial and body language cues to ensure that patients understand information they are receiving. There were likely technical and logistic issues; many departments did not have a strong telemedicine infrastructure prior to the current pandemic. This may have led to more time troubleshooting as opposed to patient care. Interestingly, 76% of providers would have been likely or very likely to perform a procedure had they seen the patient in person; this likely affected providers’ satisfaction as telemedicine limited their resources for evaluation. Many providers likely are concerned about reimbursement; without the ability to code for a comprehensive physical examination, the value of telemedicine encounters will likely reflect that if telemedicine continues.
Concordance
In our study, all encounters followed a synchronous model, meaning that both parties were present at the time of the encounter, as opposed to recording patient information and forwarding it to the provider. The majority of patients (78.8%) reported that they felt that their provider was able to provide a diagnosis and treatment plan; similarly, the average response to impact on care due to a lack of physical examination corresponded to “no negative impact” to “minor negative impact.” Despite having no physical examination, patients reported receiving adequate workup and treatment. In a study by Smith et al in a pediatric otolaryngology practice, surgical decision making was the same for 93% of synchronous telemedicine appointments as compared with face-to-face appointments. 15 Similarly, Sclafani et al demonstrated high rates of diagnostic concordance with synchronous telemedicine (85%). 16 There are conflicting data with concordance rates based on an asynchronous model. 17 Per Wu et al, using clinical context in asynchronous telemedicine encounters increased concordance from 94.4% to 100%. 18
Given that there are conflicting data about diagnostic concordance in the literature,16-18 this may contribute to provider fear for malpractice or misdiagnosis. Providers reported an average response of 2.56 of 5 when asked about their concern for malpractice, corresponding to “slightly concerned” to “somewhat concerned.” However, the average response to impact on decision making was neutral, suggesting that physical examination may have been used as confirmatory rather than eliciting the diagnosis. Additionally, the likelihood of performing a procedure had an average response between “neutral” and “likely,” again possibly indicating the use of these tests for confirmation.
Reimbursement
Prior to this study, it was hypothesized that the change to reimbursement with telemedicine visits would be a significant concern for providers. Providers reported an average answer of “slightly concerned” to “somewhat concerned” (2.28 of 5). Prior to the COVID-19 public health emergency, Medicare and other payer policies were considered a roadblock for telemedicine, 19 likely because, for telemedicine encounters, Medicare typically provides payment for fee-for-service consultations. 19 However, since March 30, 2020, the Centers for Medicare and Medicaid Services have changed its policies such that Medicare will make payment for virtual services with relaxed requirements. 20 Because of these policy changes, Medicare beneficiaries may receive needed services from otolaryngologists without risking exposure. 20
It is possible that the Medicare policy has affected the level of concern that providers have for reimbursement, though it is unclear how telemedicine reimbursement will be affected once the public health emergency has subsided. Additionally, 2 of the sites where providers were surveyed are military hospitals (Walter Reed and Fort Belvoir Community Hospital). At these institutions, all patients have insurance through Tricare, and these providers do not receive reimbursement through Medicare; it is likely this also lowered the average level of concern for this survey question.
Limitations
Our study had multiple limitations. Patients were surveyed during the government shutdown, when nonurgent medical visits were canceled. Patient bias may have been introduced when the alternative to telemedicine was no medical appointment. Some patients performed the survey up to 1 month after their encounter; this introduces recall bias when compared with the group of patients who were surveyed soon after their appointment. There was likely also participant bias; patients who participated may have been influenced to give answers to appease the physicians performing the survey. This was mitigated by telling each patient that the survey was confidential. The treating providers did not survey their patients. The sample size of the provider satisfaction survey was fairly small (n = 25), which is also a limitation. The majority of respondents (72%) to the provider survey were from the nonmilitary practice, as compared with 35.7% civilian respondents for the patient survey. This is the result of a limited number of military providers at Fort Belvoir Community Hospital and Walter Reed National Military Medical Center as compared with the civilian counterpart, Ochsner Medical Center.
Fort Belvoir Community Hospital and Walter Reed National Military Medical Center are military hospitals that primarily serve active duty service members, retirees, and dependents. This means that the 206 patients and 7 providers surveyed from these locations represent a niche community, which may not apply to the civilian population. At these locations, all patients have Tricare insurance; thus, concern for reimbursement may differ as compared with civilian counterparts. Patient expectations between military and civilian health systems remain unknown and warrant further study.
Conclusions
The patient satisfaction survey demonstrated high levels of satisfaction with limited confounding factors, indicating that satisfaction was consistent across each demographic listed. Ear complaints were associated with the lowest satisfaction scores. Providers had lower levels of satisfaction as compared with patients, though telemedicine had a neutral effect on providers’ decision making and caused a “slight” to “somewhat” higher concern for malpractice and misdiagnosis. Overall, telemedicine appears to be a viable option for patients and providers, though further study is necessary to elicit which patients are at high risk for poorer outcomes.
Author Contributions
Disclosures
Footnotes
The contents of this publication are the sole responsibility of the authors and do not necessarily reflect the views, opinions, or policies of Uniformed Services University of the Health Sciences, the Department of Defense, or the departments of the army, navy, or air force. Mention of trade names, commercial products, or organizations does not imply endorsement by the US government.