Clinician Satisfaction with Rapid Adoption and Implementation of Telehealth Services During the COVID-19 Pandemic

Abstract

Background:

To examine clinician perspectives on the acceptability, appropriateness/suitability, and feasibility of video telehealth as a way to deliver mental health services during the COVID-19 pandemic.

Materials and Methods:

Mental health clinicians were surveyed with 27 Likert questions, using previously validated measures, on satisfaction and implementation experience with video telehealth visits between March and June 2020.

Results:

A total of 112 of 193 clinicians completed the survey (58.0%), including psychiatrists, psychologists, trainees (i.e., residents and fellows), advanced practice providers, and licensed mental health counselors. Clinicians reported high levels of acceptability, feasibility, and appropriateness of video telehealth; they also reported high levels of satisfaction with video telehealth visits. Seventy-nine and a half (79.5%) reported that their patients seemed highly satisfied with video telehealth visits, and 107 (95.5%) of clinicians responded that they would like video telehealth visits to represent at least 25% of their practice in the future.

Discussion:

Mental health clinicians showed positive attitudes toward the implementation of video telehealth visits, high levels of satisfaction with this care, and indicated strong interest in continuing this modality as a significant portion of clinical practice.

Conclusions:

This study demonstrates the ability of mental health clinicians to embrace new technology to expand access to care during the COVID-19 pandemic. Results indicate that telemental health is likely to be an integral part of clinic practice in the future.

Background

Telehealth is a mode of health care delivery that provides care remotely through the use of electronic systems and telecommunications technology. Telehealth has been shown to improve access to health care, offer a high quality of care, reduce costs, and it is now being used to protect patients and providers from exposure during the COVID-19 pandemic.¹ Telemental health (TMH), also known as telepsychiatry, telepsychology, telebehavioral health, or teletherapy, is especially well suited to deliver care at a distance because mental health encounters primarily involve audiovisual communication and rely less on physical examination. Studies have found that TMH is highly effective across a comprehensive range of mental health diagnoses and is well accepted by patients.² There is a growing body of literature demonstrating that clinical outcomes with TMH are equivalent to in-person services.^3,4 Despite the immense potential of TMH, its widespread implementation has lagged behind expectations.^5,6 Unfortunately information technology systems, when used in health care settings, encounter many challenges with as many as 75% of programs failing during the operational stage.⁷

Multiple barriers may account for why telemedicine has not yet lived up to its potential. A 2018 literature review of barriers to telemedicine described 33 barriers to the adoption of telemedicine worldwide; some of which included cost, technically challenged staff, and resistance to change.⁸ Medicare restrictions on service codes and originating site restriction have also limited broader implementation of telehealth programs, particularly for disabled and older adults.⁶ A recent review of barriers to TMH found that provider concerns about TMH represented one of the primary barriers to broader implementation of these new technologies.⁵ Clinicians' acceptance of telehealth and ability to adapt to new technology has been identified as a critical factor in the long-term success of TMH programs.⁹ Previous research indicates that clinicians report lower satisfaction with telehealth in comparison to patients who generally report high levels of satisfaction.¹⁰ It has been suggested that younger, more technologically savvy clinicians are likely to be more accepting of TMH practice.⁹

The COVID-19 pandemic thrust the U.S. health care system into a crisis of providing care to patients while maintaining social distancing to reduce risk of infection. This led to governmental changes to allow a rapid transition from in-person to virtual care. Many health care systems, such as Mayo Clinic, had started using telehealth over the past decade; however, COVID-19 triggered the need for its rapid expansion. The Veterans Administration, which has a well-established telehealth service, noted a > 500% increase in TMH visits at the start of the COVID-19 pandemic.¹¹ Despite the VA's prior experience with telehealth, qualitative interviews with clinicians during the COVID-19-related expansion note clinician concerns regarding quality of care, technological issues, and administrative difficulties.¹²

The Mayo Clinic Department of Psychiatry and Psychology began developing telemedicine services in 2015. From 2015 to 2020, these services were gradually expanded; yet only a limited number of providers (5 of 70 psychiatrists) participated in telehealth, and as a whole they only averaged 5–6 visits per week across the health care system. As a result of the COVID-19 pandemic, the number of video telehealth visits in the Mayo Clinic Department of Psychiatry and Psychology escalated from the baseline average of 17 per month to 1,478 in the month of April (Fig. 1). This involved a rapid implementation and training process for hundreds of clinicians, across disciplines and levels of experience. This study evaluates the experience of mental health clinicians at an academic medical center, specifically their perceptions of acceptability, appropriateness, and feasibility of video telehealth.

Figure 1.

Completed video appointments 2020.

Methods

Study Design

This study was a cross-sectional descriptive survey of mental health clinicians who provided telehealth services during the time of the survey between March and June 2020. Inclusion criteria were: the completion of at least one video telehealth visit during the time period based on data pulled from the electronic medical record (EMR). Additional non-billing providers involved in video telehealth visits were identified by clinical program leaders. The study was approved by the Mayo Clinic Institutional Review Board (study ID: 20-0046090), and participants were provided an email consent form. Participants indicated their consent by completing and returning the survey.

Outcome Measures and Survey

The survey included demographic information, prior experience with video telehealth visits, and perceived provider acceptability, feasibility, and appropriateness of video telehealth implementation. The survey utilizes three implementation science outcome measures with permission from the developers, Weiner et al. (1) the Acceptability of Intervention Measure (AIM), (2) Intervention Appropriateness Measure (IAM), and (3) Feasibility of Intervention Measure (FIM).¹³

These measures have Cronbach alphas from 0.85 to 0.91, and test–retest reliability coefficients ranged from 0.73 to 0.88.¹³ In addition to these validated measures, the study survey has additional items tailored specifically for the Department of Psychiatry and Psychology, with particular focus on the role of the COVID-19 pandemic in the perceptions of telehealth. The full survey was not re-assessed for validity or internal consistency. In total, the 3-page survey took less than 10 minutes to complete and was composed of 27 Likert questions on a five-point continuum framed from “Completely Disagree” to “Completely Agree.” The areas were: (1) acceptability (AIM) (e.g., telehealth is appealing or welcome; four questions); (2) appropriateness (IAM) (e.g., suitable, good match to provide care; four questions); (3) feasibility (FIM) (e.g., doable, comfortable; four questions); and (4) 17 additional questions about the clinician's experience with the telehealth implementation process, reflections on the COVID-19 window of time and attitudes going forward with continued use of video telehealth.

Settings/Participants

Participants were identified by using the EMR to create a list of mental health providers who completed video visits between March 2020 and June 2020. Other non-billing providers were also identified by contacting clinical programs to identify other clinical staff participating in video visits. Participants were clinical staff within the Department of Psychiatry and Psychology and were located at one of two primary clinical sites within the practice in Rochester, Minnesota. Invitations to complete the survey were distributed by email. Overall, the approach aligned with the checklist reporting results of online surveys, in terms of a description of the purpose, time to complete, voluntary participation, and data protection and confidentiality.¹⁴

Procedures for Telehealth Implementation

In March 2020, when the State of Minnesota enacted a stay-at-home order, the Mayo Clinic institutional and departmental leadership team decided to significantly scale back in-person clinical activities to reduce COVID-19 risks for patients and staff. A departmental work group, consisting of telehealth experts, clinical staff, administrators, scheduling, billing, and compliance experts, was formed to facilitate the rapid transition to telehealth visits. The transition plan addressed the following:

Technology telehealth needs: A needs assessment was performed of the outpatient areas and all clinical offices were equipped with web cameras and headsets to facilitate communication and privacy. In order to support staff to perform video visits from home, the institution equipped all clinical staff with individual laptops with web cameras. Due to the number of clinicians working from home, there were increased demands on the virtual private network, requiring rapid system upgrades. The Zoom platform was already in use within the institution and was chosen as the platform for this expansion. The Zoom platform was embedded within the EMR and linked to the patient encounter. The Zoom visit link and passcode was provided to patients via the patient portal component of the EMR along with instructions for connecting to video telehealth visits. Patients were encouraged to check their technologic capabilities before their scheduled appointment by contacting a Connected Care support line where they could initiate a Zoom link and trouble shoot any technical difficulties.

Clinician readiness: The Mayo Clinic Office of Connected Care developed a self-service toolkit with links to training videos and other online resources to support the transition to both video and telephone visits. Safety protocols for potential psychiatric emergencies during a telehealth visit were developed by the telehealth workgroup and distributed to all clinicians.

Scheduling: To reduce nonessential in-person visits, clinicians were asked to prioritize upcoming scheduled visits by categorizing them as elective, semi-urgent, or urgent. Urgent visits were performed in-person with increased infection-control procedures in place. Elective and semi-urgent visits were rescheduled for telephone or video visits depending on clinician and patient capabilities and preferences.

Data Collection and Statistical Analysis

The study survey was distributed by email, and data were collected and managed by using the Research Electronic Data Capture (REDCap) electronic data capture tools hosted at Mayo Clinic.^15,16 REDCap is a secure, web-based software platform designed to support data capture for research studies, providing multiple benefits for audit trails, data capture, manipulation, integration, export, and analysis.^15,16 Each survey was reviewed for completeness, which was 100%, as the software prompts the participant to complete all questions. Duplicate entries were avoided by preventing users' access to the survey through the settings within REDcap.^15,16

Survey data were summarized by using descriptive statistics for primary and exploratory outcome measures, using frequency and percentages for binary and categorical measures (demographic variables and binary measures of provider satisfaction), and using mean, median, standard deviation, and range for ordinal variables (Likert-scale measures of provider satisfaction). For primary outcome variables of provider satisfaction, we assessed whether levels of provider satisfaction were associated with age and prior experience with telehealth, using Spearman correlation for ordinal outcome variables and Wilcoxon rank-sum and Fisher's exact tests for binary outcome variables. Changes in number of video telehealth visits were tested by using paired Wilcoxon rank-sum tests. Analyses were performed by using SAS (version 9.4; Cary, NC).

Results

Demographics

The study survey was emailed to 193 mental health clinicians, with 112 returning the study survey (58.0%). The demographic information is summarized in Table 1. The sample included a variety of mental health professionals, including staff psychiatrists, psychologists, advanced practice providers, residents, fellows, and licensed therapists/counselors. There was a wide distribution of ages and years in practice (Table 1). Respondents indicated a large increase in the use of telehealth services related to the onset of the COVID-19 pandemic, with 90 (82.6%) having done 5 or fewer video telehealth visits before COVID-19, in contrast to 62 (55.9%) having completed more than 20 video telehealth visits in the 3 months before the survey (p < 0.0001).

Table 1.

Demographics

CHARACTERISTIC	CATEGORY	N	%
Discipline
	Faculty psychiatrist	37	33.0
	Faculty neuro/psychologist	18	16.1
	APP (APRN, PA)	9	8.0
	Resident/fellow (psychiatry)	15	13.4
	Resident/fellow (psychology)	8	7.1
	Clinical Social Worker	2	1.8
	LADC/LPCC	17	15.2
	Other	6	5.4
Age
	20–29	8	7.2
	30–30	37	33.3
	40–49	24	21.6
	50–59	26	23.4
	60+	16	14.4
	Missing	1
Gender
	Female	67	59.8
	Male	41	36.6
	Transgender	1	0.9
	Do not identify as F/M/T	1	0.9
Years in practice
	0–5	34	30.4
	6–10	19	17.0
	11–15	14	12.5
	16–20	10	8.9
	20–25	7	6.3
	25+	28	25.0
Visits pre COVID
	0–5	90	82.6
	5–10	7	6.4
	10–20	4	3.7
	20+	8	7.3
Visits post-COVID
	0–5	7	6.3
	5–10	19	17.1
	10–20	23	20.7
	20+	62	55.9

Implementation Measures

Respondents rated video telehealth highly with regard to acceptability, appropriateness, and feasibility (with 83–97% who agreed or completely agreed with these survey items) (Table 2). These items were analyzed for associations with specialty, age, years in practice, and previous experience with telehealth. There was a negative association between age and two of the items: (1) I welcome telehealth and (2) telehealth seems easy to use (Table 3). No other statistically significant associations were found.

Table 2.

Implementation Measures

	TELEHEALTH SEEMS	COMPLETELY AGREE/AGREE
	TELEHEALTH SEEMS	N	%
IAM
	Fitting	104	92.9
	Suitable	96	85.7
	Applicable	93	83.0
	Like a good match	103	92.0
FIM
	Implementable	108	96.4
	Possible	109	97.3
	Doable	109	97.3
	Easy to use
AIM
	Telehealth meets my approval	102	91.1
	Telehealth is appealing to me	97	88.2
	I like telehealth	105	93.8
	I welcome telehealth	89	80.9

AIM, Acceptability of Intervention Measure; FIM, Feasibility of Intervention Measure; IAM, Intervention Appropriateness Measure.

Table 3.

Association of Practice Years and Age

LABEL^a	QUESTION TEXT		PRACTICE YEARS
Pref post COVID	When COVID is over, I prefer connecting	Rho	−0.122
	w/patient via video for part of practice	p-value	0.2038
AIM 4	I welcome telehealth	Rho	−0.089
		p-value	0.3496
FIM 4	Telehealth seems easy to use	Rho	−0.122
		p-value	0.2038

Statistically significant p-value <0.05.

All other measures were checked, no significance.

Respondent Attitudes on TMH

Respondents indicated that their patients seemed highly satisfied with video visits [89 (79.5%)] (Table 4). Respondents agreed that they felt as comfortable with video visits as in-person visits and comfortable with video for both established and new patients. There was relatively less agreement regarding the efficiency of video visits, with 66 (58.9%) who agreed or completely agreed that video was more efficient than in-person visits (Table 4).

Table 4.

Additional Survey Questions

QUESTION	COMPLETELY AGREE/AGREE	COMPLETELY DISAGREE/AGREE
QUESTION	N	%	N	%
During the COVID pandemic, I prefer video as a safer and more effective option than in-person care with universal masking	82	73.2	11	9.9
When the COVID pandemic is over, I would prefer connecting with my patient by video for a significant part of my clinical practice	55	49.1	29	25.9
My patients seem highly satisfied with video visits	89	79.5	4	3.6
I feel as comfortable with video visits as in-person visits	71	64.5	23	20.5
I feel comfortable with video visits for both established and new patients	71	64.5	22	20
Video visits are more efficient than in-person visits	66	58.9	18	16.1
The technical support for telehealth was adequate	77	68.8	12	10.7
The training for telehealth was adequate	82	73.2	10	8.9
The video quality was not a barrier to communication	82	73.2	13	11.6
The audio quality was not a barrier to communication	78	69.6	19	17
Video visits feel appropriately secure	94	83.9	18	16.1
Video visits feel appropriately private	73	65.2	14	12.6
Video visits feel appropriately personal	77	68.8	20	17.9
Video visits allow me to provide a Mayo Clinic Standard of Care to the patient	85	76	9	8

Views of Technology, IT Support, and Training

Respondents gave high ratings of the quality of technical support and training for telehealth. Audio and visual quality also appeared to be mostly acceptable. Of the few respondents who cited technology concerns, more cited audio 19 (17%) than video 13 (11.6%) as a barrier. Conversely, more agreed that video was not a barrier 82 (73.2%) than audio 78 (69.9%) (Table 4).

Respondents' Attitudes about Future Use of Telehealth

Almost half, 55 (49.1%), of respondents agreed or completely agreed with the statement, “When the COVID-19 pandemic is over, I would prefer connecting with my patient by video telehealth for a significant part of my clinical practice” with only 29 (25.9%) who disagreed/completely disagreed (Table 4). When asked about the ideal future practice involving telehealth, the most frequent response was that an ideal practice would include 50% telehealth [47 (42.0%)], with only 5 (4.5%) respondents indicating a preference for no (0%) component of telehealth practice (Fig. 2). In total, 107 (95.5%) clinicians responded that they would like video telehealth visits to represent at least 25% of their practice in the future.

Figure 2.

Telehealth as part of future practice.

Discussion

This study survey examined mental health clinician perspectives on the acceptability, appropriateness/suitability, and feasibility of video telehealth as a way to deliver mental health services during the COVID-19 pandemic. Although the clinicians surveyed in this study had limited experience with telehealth before the onset of COVID-19, they were able to adapt to the rapid implementation of telehealth visits in the context of the COVID-19 pandemic. These clinician satisfaction data are unique in that they represent the perspective from mental health professionals who may have had little interest in, or may have been skeptical of, telehealth pre-COVID-19. Satisfaction results indicate that clinicians were accepting of telehealth, valued its role in responding to the COVID-19 pandemic, and rated telehealth as highly beneficial to their patients. This was facilitated by the shared incentive for patients and providers to maintain a therapeutic relationship during COVID-19, but also by strong institutional support in providing equipment, training, and IT assistance for the delivery of telehealth.

There is a growing body of literature on the experiences of institutions transitioning to telehealth during the COVID-19 pandemic. Some systems (e.g., UC Davis Health System) converted even more rapidly to an entirely virtual clinic over the course of just 3 days.¹⁷ Institutions transitioning to telehealth have shared similar challenges with training, technology, and administrative barriers (e.g., scheduling, licensing, credentialing).^12,17,18 However with institutional support, peer supervision, and adequate training, telehealth practices can be implemented quickly and effectively. In addition to provider acceptance of the implementation of video telehealth, tremendous institutional support was required to facilitate the process. The institution provided encrypted, telehealth ready laptops for all clinical staff along with increased IT support and expanded virtual networks. Administrative staff was required for the scheduling and check in process for video visits. These efforts were key contributors to high provider satisfaction with their adoption of a telehealth practice.

Patient demand for telehealth has been increasing over several decades, with clinicians, at times, representing a barrier to the broader implementation of telehealth.⁵ Many have suggested that clinician acceptance is a key element in expansion and sustainability of telehealth, more so than technology problems or lack of resources.¹⁹ The results of this survey suggest that clinicians may no longer represent a barrier to telehealth. Indeed, the findings from our study suggest that strong external factors (e.g., necessities for care, concerns for COVID-19 infection) may have altered attitudes and helped with motivation to adopt telehealth, and that institutional support, training, and hands-on experience will result in continued high clinician acceptance of telehealth as a significant component of practice.

In this study, a majority of mental health providers were interested in incorporating telehealth in their practice post-COVID-19. Despite Mayo Clinic shifting back to in-person health care shortly after the state emergency order, video visits have continued to represent a large percentage of the practice within the Department of Psychiatry and Psychology; representing ∼20% of outpatient visits in 2020. As patient and clinician demand for telehealth continues to increase, it is likely that telehealth will continue as a significant part of mental health care even after the end of the COVD-19 pandemic.

The conditions of the COVID-19 pandemic continue to be a significant driver of telehealth usage across the United States. As telehealth becomes increasingly commonplace, there is likely to be a growing expectation for telehealth to become a standard option in health care delivery. Unprecedented changes to regulations of telehealth, including relaxed requirements for state licensure, expansion of covered services under Medicare and Medicaid, and expanded options for controlled medication prescribing covered under the Ryan Haight Act, may or may not continue after the expiration of the COVID-19-related Federal State of Emergency, creating uncertainty about the future of telehealth.²⁰

This study has several important limitations. The study has a cross-sectional design, which yields impressions that are helpful, but not for determining cause-and-effect relationships, as causality could only be obtained by studies with pre- and post-assessments. The survey was founded on implementation-science research, but it was brief, partly to enhance completion by busy clinical mental health providers. Perhaps structured qualitative interviews may have yielded different findings. This institution may be different than other health systems, so there are limitations in the ability to generalize these results to different types of health care institutions. There are unique factors related to the COVID-19 pandemic, which are likely impacting these results, with no way to control for this variable. It is difficult to assess to what extent the exposure to telehealth resulted in the high levels of acceptance or whether providers had positive views of telehealth before the COVID-19 pandemic. In addition, this study aggregates perspectives from many different mental health clinicians performing various aspects of management through telehealth. Further investigation will add additional clarity regarding the different modes of treatment, interventions, and target populations that are best suited to TMH.

Conclusions

Despite the potential benefits of telehealth to provide high quality care while increasing access and reducing costs, the widespread implementation of telehealth has lagged expectations. The COVID-19 pandemic created a unique set of external forces that pushed telemedicine to the forefront of health care delivery. This has presented numerous risks and opportunities. The rapid implementation of telehealth in the COVID-era has changed the landscape of health care forever. This study demonstrates the willingness and ability of mental health clinicians, even those without prior telehealth experience, to embrace new technology to expand access to care during the COVID-19 pandemic. These results indicate that when supported by the institution, when licensing requirements have been addressed (waived during COVID-19), and with training and IT support to both patients and clinicians, TMH is viewed as an integral part of clinic practice now and into the future.

Footnotes

Disclosure Statement

No competing financial interests exist.

Funding Information

No funding was received for this article.

References

Hilty

, Ferrer

, Parish

, Johnston

, Callahan

, Yellowlees

. The effectiveness of telemental health: A 2013 review. Telemed J E Health, 2013; 19:444–454.

Bashshur

, Shannon

, Bashshur

, Yellowlees

. The empirical evidence for telemedicine interventions in mental disorders. Telemed J E Health, 2016; 22:87–113.

O'Reilly

, Bishop

, Maddox

, Hutchinson

, Fisman

, Takhar

. Is telepsychiatry equivalent to face-to-face psychiatry? Results from a randomized controlled equivalence trial. Psychiatr Serv, 2007; 58:836–843.

Ruskin

, Silver-Aylaian

, Kling

, et al. Treatment outcomes in depression: Comparison of remote treatment through telepsychiatry to in-person treatment. Am J Psychiatry, 2004; 161:1471–1476.

Cowan

, McKean

, Gentry

, Hilty

. Barriers to use of telepsychiatry: Clinicians as gatekeepers. Mayo Clin Proc, 2019; 94:2510–2523.

Gentry

, Lapid

, Rummans

. Geriatric telepsychiatry: Systematic review and policy considerations. Am J Geriatr Psychiatry, 2019; 27:109–127.

Berg

. Patient care information systems and health care work: A sociotechnical approach. Int J Med Inform, 1999; 55:87–101.

Scott Kruse

, Karem

, Shifflett

, Vegi

, Ravi

, Brooks

. Evaluating barriers to adopting telemedicine worldwide: A systematic review. J Telemed Telecare, 2018; 24:4–12.

Brooks

, Turvey

, Augusterfer

. Provider barriers to telemental health: Obstacles overcome, obstacles remaining. Telemed J E Health, 2013; 19:433–437.

10.

Richardson

, Frueh

, Grubaugh

, Egede

, Elhai

. Current directions in videoconferencing tele-mental health research. Clin Psychol (New York), 2009; 16:323–338.

11.

Connolly

, Stolzmann

, Heyworth

, Weaver

, Bauer

, Miller

. Rapid increase in telemental health within the Department of Veterans Affairs during the COVID-19 pandemic. Telemed J E Health, 2020; 27:454–458.

12.

Myers

, Coulon

, Knies

, et al. Lessons learned in implementing VA video connect for evidence-based psychotherapies for anxiety and depression in the Veterans Healthcare Administration. J Technol Behav Sci, 2020:1–7.

13.

Weiner

, Lewis

, Stanick

, et al. Psychometric assessment of three newly developed implementation outcome measures. Implement Sci, 2017; 12:108.

14.

Eysenbach

. Improving the quality of Web surveys: The Checklist for Reporting Results of Internet E-Surveys (CHERRIES). J Med Internet Res, 2004; 6:e34.

15.

Harris

, Taylor

, Minor

, et al. The REDCap consortium: Building an international community of software platform partners. J Biomed Inform, 2019; 95:103208.

16.

Harris

, Taylor

, Thielke

, Payne

, Gonzalez

, Conde

. Research electronic data capture (REDCap)—A metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform, 2009; 42:377–381.

17.

Yellowlees

, Nakagawa

, Pakyurek

, Hanson

, Elder

, Kales

. Rapid conversion of an outpatient psychiatric clinic to a 100% virtual telepsychiatry clinic in response to COVID-19. Psychiatr Serv, 2020; 71:749–752.

18.

Sharma

, Sasser

, Schoenfelder Gonzalez

, Vander Stoep

, Myers

. Implementation of home-based telemental health in a large child psychiatry department during the COVID-19 crisis. J Child Adolesc Psychopharmacol, 2020; 30:404–413.

19.

Wagnild

, Leenknecht

, Zauher

. Psychiatrists' satisfaction with telepsychiatry. Telemed J E Health, 2006; 12:546–551.

20.

Breitinger

, Gentry

, Hilty

. Key opportunities for the COVID-19 response to create a path to sustainable telemedicine services. Mayo Clinic Proc, 2020; 95:2602–2605.