Disagreement in Satisfaction Between Patients and Providers in the Use of Videoconferences by Depressed Adults

Abstract

Aim:

To evaluate whether there was a difference in satisfaction scores between providers and patients in the use of videoconferences (VCs) by depressed adults.

Method:

This study was a subanalysis of the joint European project, MasterMind, and participants were recruited from 15 pilot studies in 11 different countries. The Client Satisfaction Questionnaire (CSQ)-3 was used as assessment tool, and scores were summed to give total scores. The questionnaire consists of three items evaluating general satisfaction, fulfillment of needs in treatment, and usability.

Results:

A total of 362 respondents, 201 patients and 161 providers, completed the questionnaire. Providers had a mean total CSQ-3 score of 9.17 (95% confidence interval [CI] = 8.90–9.45), whereas patients had a mean of 9.70 (95% CI = 9.44–9.98). Mean scores for item 1 (the extent to which VCs had met the needs of the participants): patients 3.19, providers 2.93 (p = 0.00048); for item 2 (general satisfaction): patients 3.22, providers 3.08 (p = 0.083); and item 3 (whether participants wanted to use VCs again): patients 3.28 providers 3.16 (p = 0.045).

Conclusion:

The results showed that total satisfaction scores were higher in patients than in providers. The differences between patients and providers were significant for items 1 and 3 (p < 0.05), but we did not find a significant difference regarding item 2.

Introduction

Depression occurs in ∼14% of the population, but only ∼50% receive treatment.¹ Psychiatry faces several challenges including lack of resources, psychiatrists, and trained health care providers.^2,3 Immobility, negative attitudes, stigma, and great distances to clinics are often barriers that prevent patients receiving treatment.^2,4,5 Innovative approaches such as videoconferences (VCs) represent a useful addition to bring evidence-based care closer to patients' homes and to increase the range, access, and quality of available mental health services.^2,4

–7

VCs have been shown to be a viable option for delivering mental health care^8,9 to depressed patients. Several studies have shown that the clinical outcome of VCs is comparable with face-to-face (FTF) treatment.^1,8,10,11 VC treatment has actually turned out to be superior to FTF treatment with regard to depression outcomes.⁸ In addition, most studies demonstrated that use of VCs generally reduces direct and indirect costs compared with FTF treatment.^1,7,8,12 However, there are still challenges to its adoption.^13,14 An important aspect in the progression of telemedicine services into daily practice is whether the technology is acceptable to both patients and providers,^15,16 especially because providers are often the initial gatekeepers to use of VCs.^1,17

Despite positive results from studies, there seems to be barriers¹⁸ and a large gap between the research evidence and the use of telepsychiatry in routine health care.⁸ Satisfaction is an important indicator of the relative advantages, quality, and usability of new technology and influences its adoption.^16,19 Most studies of VC report high satisfaction and acceptance among patients.^8,9 Providers' satisfaction has been less evaluated and appears to be mixed but overall lower than patient satisfaction.^8,20 However, studies show that in certain applications, patients may view telemedicine more favorably than providers,²¹ whereas in other cases providers may view telemedicine more favorably than patients.²²

The existing studies are characterized by several limitations, including low sample sizes, results based on descriptive feasibility studies, or advice to other telemedicine providers and absence of validity testing of survey instruments.^8,16,20 However, there is a lack of studies comparing satisfaction between patients and providers in a defined disease group,^7,8 and no studies have compared satisfaction between patients and providers regarding the treatment of depression. The aim of this study was to evaluate whether there is a difference between measures of provider and patient satisfaction using VCs in an international joint European large-scale project in adults with unipolar depression.

Methods

Setting and Study Sample

The study was performed as a subanalysis of the MasterMind project. The project consisted of two different studies, partly an implementation study for scaling up computerized cognitive behavior therapy (cCBT) and partly collaborative care facilitated by videoconferencing (ccVC) in routine practice in 11 different European regions in 15 different implementation projects. Data from the two studies were recorded in the same database. Based on the aim of this subanalysis only data from participants in the ccVC part were included. Patients were adults 18+ years with mild to moderate unipolar depression. Depending on the diagnosis and care system, patients in MasterMind were treated in primary care or in specialized care settings. Health care providers were GPs, psychiatrists, psychologists, nurses, and other mental health care workers.

The effectiveness of these implementation projects was evaluated with the objective of identifying the factors that promote or inhibit implementation and upscaling of cCBT and ccVC for the treatment of unipolar depression in routine practice.^23,24 The evaluation was structured according to the Model for Assessment of Telemedicine (MAST),²⁵ in which seven highly interrelated domains are assessed, including patients' and providers' satisfaction with the use of cCBT and ccVC in routine practice.

In the MasterMind project, participants include patients, health care providers, and mental health care organizations.²⁴ The interactions between these three levels are of interest as they influence each other.²⁶ In the evaluation of satisfaction scores, only patients and providers were included. Depending on the health care system, patients in the MasterMind project were treated in primary care or in specialized care settings.

Interventions

The ccVC interventions in this subanalysis are described in the MasterMind protocol as given in Table 1.²³

Table 1.

Description of the Collaborative Care Facilitated by Videoconferencing Interventions in This Subanalysis^a

	DESCRIPTION OF INTERVENTION TYPES
A	Treatment for depression facilitated by a VC between specialist and GP with the patient present. The purpose of the meeting is to provide direct specialist support during treatment
B	Follow-up or outpatient care of the patient at home. In this case, the health care worker communicates with the patient, who is at home. The health care worker can be a GP, specialist, or other health care professional depending on the location. This can be as a follow-up to cCBT treatment, face-to-face treatment, and so on
C	Acute care. This is a VC between the acute ward and a specialist. Together with the patient, the specialist devises a plan for the patient, for example, a decision as to whether admission is necessary. By reducing unnecessary admissions, we save both resources in the health care system and avoid an extra burden on the patient

Reference.²³

cCBT, computerized cognitive behavior therapy; VC, videoconference.

Some pilot studies implemented several different types of interventions. The individual course of treatment and the number of VC contacts differed between pilot studies.

Satisfaction with the intervention as a whole was measured with the Client Satisfaction Questionnaire (CSQ). The CSQ is a portfolio of scales designed to be direct measures of an individual's personal experiences with a specific service rather than health care in general.²⁷ Internal consistency and construct validity have been established, and the instrument is widely used in research.²⁸ Patient satisfaction was assessed with CSQ-8, which contains eight items. The health care providers' satisfaction was assessed with CSQ-3, the core item set of the CSQ scales, and contains items 3, 7, and 8 from CSQ-8.^27,29 The comparison between the groups was based on the mentioned items and included the following questions: (1) To what extent has the treatment met your needs? Has this ccVC intervention met your needs in treating patients? (2) In an overall general sense, how satisfied are you with the treatment you have received? How satisfied are you with use of the ccVC intervention in treating your clients? (3) If you were to seek help again, would you make use of this treatment again? To provide treatment again, would you use this ccVC intervention again?

Participants were asked to rate satisfaction on a four-point scale, with a possible range of 3–12, with higher scores indicating greater satisfaction.²⁹ The CSQ-8 questionnaire was administered to patients at the end of treatment or on termination of the study and not later than 6 months after inclusion. The CSQ-3 questionnaire was administered to providers at the end of the study.²³ The questionnaires were written in the local language, and when no translation was available, local implementation teams translated the questionnaire by the forwards-backwards method. Demographic data, number of VC sessions, and symptom score were obtained in patients, and demographic data and professional experience in providers.²³

Data Analysis

Data from the questionnaires were analyzed using R-3.5.1. Patients and providers were included in the analysis if they had responded to all three CSQ-3 items. Because scores were not normally distributed, nonparametric tests were used and confidence intervals (CIs) were bootstrapped. Multiple linear regression was used to investigate the effect on the score of responder type (patient/provider) and region. Residuals were examined and found to be approximately normally distributed.

Ethics

Participants were informed about the aim of the study, that anonymity would be preserved, and that they could withdraw from the study at any time. Verbal and written consent were obtained from each participant. All procedures in each participating country were performed in conformity with existing local clinical guidelines and local legislation. Ethical standards were applied in line with the Declaration of Helsinki and the relevant EU legislation on data protection. Sharing data between participating countries followed the regulations of each host country. To protect all information, all partners followed all the relevant Advanced Encryption Standard procedures for personal password use and data encryption. Electronic data were password protected and accessed only by authorized personnel.²³

Results

A total of 362 respondents consisting of 201 patients and 161 providers filled out the questionnaire. The procedure used to enroll patients and providers from the MasterMind project is given in Figure 1.

Fig. 1.

Flow diagram of patients and providers from the MasterMind project included in the subanalysis.

Demographic information was available for patients (Table 2), but only information for service used and region was available for providers (Table 3). Service 3 corresponds to VC as a follow-up to cCBT, whereas services 1 and 2 are the other interventions.

Table 2.

Demographic Description of Patients and Service Used

VARIABLES	N = 201
Service = 3 (%)	125 (62.5)
Age, mean (SD)	51.59 (15.53)
Gender = male (%)	87 (43.3)
Education (%)
Higher and/or university	63 (31.5)
Other	27 (13.5)
Primary	32 (16.0)
Secondary	78 (39.0)
Employment (%)
None	80 (40.0)
Not applicable	16 (8.0)
Yes	104 (52.0)
Sessions, mean (SD)	9.71 (9.02)
Region (%)
Badalona	26 (12.9)
Denmark	31 (15.4)
Germany	81 (40.3)
Greenland	10 (5.0)
The Netherlands	2 (1.0)
Piemonte	48 (23.9)
Treviso	1 (0.5)
Wales	2 (1.0)

SD, standard deviation.

Table 3.

Distribution of Providers Between Regions and Service Used

N	161
Service = 3 (%)	111 (68.9)
Region (%)
Aragón	6 (3.7)
Badalona	66 (41.0)
Basque country	7 (4.3)
Denmark	11 (6.8)
Galicia	11 (6.8)
Germany	16 (9.9)
Greenland	21 (13.0)
The Netherlands	1 (0.6)
Piemonte	11 (6.8)
Treviso	8 (5.0)
Wales	3 (1.9)

Providers had a mean total CSQ-3 score of 9.17 (95% CI = 8.90–9.45), whereas patients had a mean of 9.70 (95% CI = 9.44–9.98) (Fig. 2). Mean score for region and responder type can be seen in Table 4. Mean scores of the individual items were distributed as follows: item 1: patients 3.19, providers 2.93 (p = 0.00048); item 2: patients 3.22, providers 3.08 (p = 0.083); and item 3: patients 3.28, providers 3.16 (p = 0.045).The difference is significant when unadjusted for regions (Wilcoxon rank sum test, W = 18,775, p = 0.007). To test whether the discrepancy persisted when region was accounted for, a linear regression model was constructed with region and responder type as covariates. Results are given in Table 5 and show that responder type did not significantly affect total score when region was considered (adjusted R ² = 0.242, F-statistic: 5.644 on 11 and 350 degrees of freedom, p < 0.001). The discrepancy in the original test was caused by strong variations in regional scores and imbalance between the number of patients and providers in some regions. Region and responder type were coded as dummy variables with type = patient and region = Aragón as base levels.

Fig. 2.

Histograms of CSQ-3 total score stratified by responder type

Table 4.

Mean Score for Region and Responder Type

REGION	RESPONDER	N	MEAN	95% CI
Aragón	Providers	6	11.3	10.1–12.6
Badalona	Patients	26	8.46	7.6–9.33
Badalona	Providers	66	9.58	9.25–9.9
Basque country	Providers	7	9.14	7.79–10.5
Denmark	Patients	31	9.48	8.87–10.1
Denmark	Providers	11	8.09	7.07–9.11
Galicia	Providers	11	8	6.3–9.7
Germany	Patients	81	10.7	10.3–11.1
Germany	Providers	16	8.38	7.28–9.47
Greenland	Patients	10	9.1	8.69–9.51
Greenland	Providers	21	10	9.57–10.4
The Netherlands	Patients	2	12	12–12
The Netherlands	Providers	1	9
Piemonte	Patients	48	8.81	8.26–9.37
Piemonte	Providers	11	8.27	6.6–9.95
Treviso	Patients	1	11
Treviso	Providers	8	7.75	5.87–9.63
Wales	Patients	2	10	10–10
Wales	Providers	3	10	7.52–12.5

CI, confidence interval.

Table 5.

Variations in Regional Scores

	COEFFICIENT	STANDARD ERROR	p
(Intercept)	11.764	0.762	<0.0001
Responder type = Professional	−0.431	0.2322	0.0642
Region = Badalona	−2.194	0.7518	0.0037
Region = Basque country	−2.190	0.9889	0.0274
Region = Denmark	−2.532	0.7945	0.0016
Region = Galicia	−3.333	0.9022	0.0003
Region = Germany	−1.374	0.7725	0.0762
Region = Greenland	−1.763	0.7963	0.0275
Region = The Netherlands	−0.621	1.2664	0.6244
Region = Piemonte	−2.972	0.785	0.0002
Region = Treviso	−3.270	0.9372	0.0005
Region = Wales	−1.506	1.0804	0.1643

Discussion

The aim of this study was to evaluate whether there was a difference between measures of provider and patient satisfaction regarding use of VCs in an international joint European large-scale project. Satisfaction was measured with the CSQ. The results in this analysis showed that the total satisfaction score in patients was significantly higher than in providers. The differences between patients' and providers' scores were significant for items 1 and 3 at p < 0.05, but there was no significant difference for item 2. Item 1 concerned the extent to which ccVC had met the needs by participants and item 3 whether they wanted to use ccVC again. After adjustment for regional differences, responder type did not significantly affect total score.

The results regarding patient and provider satisfaction in other telepsychiatry studies are generally positive; however, as in this study, results toward showing higher satisfaction in patients.^7,8,20 On the contrary, results from randomized clinical trials (RCTs) comparing satisfaction between FTF treatment and use of VCs by patients and providers³⁰ show no difference between the two groups. However, RCTs investigating provider satisfaction within the telepsychiatry area are sparse but report the lowest level of satisfaction in the VC group.³⁰ Overall, the results in the RCT studies and this study point toward patients being more satisfied with the use of VCs than providers.

The patients in this study tended to discriminate more in favor of using VCs than did providers, especially as regards the extent to which the use of VCs had met treatment needs and whether they would like to use VCs again. There may be several explanations for these differences. In other studies, both patients and providers report potential changes in therapeutic alliance.³¹ Providers, however, tend to express more concerns about the potentially adverse effects of telepsychiatry on therapeutic rapport.^7,31 Providers emphasized the importance of an ongoing therapeutic relationship as a mechanism for client recovery.^32,33 They report that telepsychiatry results in lower levels of care and may hinder doctor–patient interactions.^7,34,35 Patients are less likely to endorse concerns about impaired rapport with their provider,^7,31 which can explain the difference in satisfaction regarding the extent to which VCs meet treatment needs. In addition, patients seem in general to benefit from using VCs in several ways. Improved convenience, more comfortable, efficiency, decreased costs, possibility for social support, and privacy are mentioned as reasons for satisfaction with the use of VCs,^13,36 benefits that may explain why patients were more likely to choose VCs again.

Implementation of new technology will affect and have consequences for tasks, structure, and actors according to Leavitt's organization model.³⁷ Change can be a complicated process because the elements are closely linked to each other.³⁷ Resistance to change seems to be a more significant barrier for adopting telemedicine in providers than in patients.^18,38 Use of new technology causes significant changes in daily routines.³⁸ Investment of time in training new workflows and techniques, potential for extra work and high associated costs are mentioned as reasons for providers' resistance to change.^18,20,38 Patients seem to gain benefit from using VCs in several ways, as mentioned previously, whereas providers seem to encounter challenges in relation to the way their practices are organized. This might explain why use of VCs can have more far-reaching consequences for providers than for patients and why patients are more satisfied. The use of new technology may change according to the model on a macrolevel but also on a microlevel.³⁹ As a consequence, the changes can also include individual's choices or resistance to use of technology.^18,39 Providers feel that they are already providing the best care, and they also expressed concerns that use of this new technology might hamper their efforts to provide help to patients in a critical situation.³⁸ There can be other reasons why results in this study indicate that patients are more likely to choose the use of VCs again and why they feel that VCs met their needs.

The study was characterized by several strengths and limitations. A strength was that it provided knowledge regarding satisfaction in use of VCs in both patients and providers. In addition, the study assessed perceived satisfaction among a homogeneous patient group in a large-scale project, giving a general satisfaction score regarding use of VCs in different contexts. However, it is important to keep in mind that the study was performed in a variety of settings. Being a multisite study that included the use of VCs in different ways in the interventions (Table 1), this study is limited in a number of ways. Several regions used more than one of the VC interventions, and strong variations in regional scores and imbalance between the number of patients and providers in some regions characterized this study. Variations in satisfaction score between the regions could indicate that the type of intervention can play an important role. In this study, it is not possible to distinguish between the different ways of using VCs in the interventions and to identify whether the satisfaction score differs between the interventions.

The benefits and concerns relating to online mental health resources showed some patterns of variation across professional groups.⁷ Health care professionals in this study were GPs, psychiatrists, psychologists, nurses, and other mental health care workers. The broad definition was chosen because of the heterogeneity in the different settings, but it limits the possibility of distinguishing between the professional groups and determining whether there are differences in satisfaction score. In addition, the treatment course often included both VCs and FTF contacts, and this may differ from a situation in which VCs are used as a replacement for FTF contacts.¹⁶

Low response rate, especially by the patients, indicates that data could be biased owing to self-selection because those who are less satisfied or did not like the intervention are more likely to drop out.¹⁶ It cannot be ruled out that a higher response rate could have affected the satisfaction score.

Future Research

More mixed method studies and qualitative studies are needed to explain the differences in satisfaction scores between patients and providers and to identify which type of interventions are best suited for the use of VCs. In addition, more research is needed that focuses on the reasons why patients and providers refuse to use VCs to identify and explain opportunities and limitations in the future use of this technology.

Footnotes

Acknowledgments

The authors acknowledge the Psychiatric Research Fund in the Region of Southern Denmark, the MasterMind Project, Helsefonden, Jascha Fonden, and Beckett Fonden for funding the study and Edwin Stanton Spencer for English review.

Disclosure Statement

No competing financial interests exist.

Funding Information

The study was supported by Psychiatric Research Fund in the region of southern Denmark, the MasterMind Project, Helsefonden (Grant No. 14-B-0156), Jascha Fonden and Beckett Fonden.

References

Garcia-Lizana

, Munoz-Mayorga

. Telemedicine for depression: A systematic review. Perspect Psychiatr Care, 2010; 46:119–126.

Titov

Internet-delivered psychotherapy for depression in adults. Curr Opin Psychiatry, 2011; 24:18–23.

Bashshur

, Shannon

, Bashshur

, Yellowlees

. The empirical evidence for telemedicine interventions in mental disorders. Telemed J E Health, 2016; 22:87–113.

Barney

, Griffiths

, Jorm

, Christensen

. Stigma about depression and its impact on help-seeking intentions. Aust N Z J Psychiatry, 2006; 40:51–54.

Egede

, Acierno

, Knapp

, Walker

, Payne

, Frueh

. Psychotherapy for depression in older veterans via telemedicine: effect on quality of life, satisfaction, treatment credibility, and service delivery perception. J Clin Psychiatry, 2016; 8:8.

Shore

, Kaufmann

, Brooks

, Bair

, Dailey

, Richardson Jr

, et al. Review of American Indian veteran telemental health. Telemed J E Health, 2012; 18:87–94.

Hubley

, Lynch

, Schneck

, Thomas

, Shore

. Review of key telepsychiatry outcomes. World J Psychiatry, 2016; 6:269–282.

Chakrabarti

Usefulness of telepsychiatry: A critical evaluation of videoconferencing-based approaches. World J Psychiatry, 2015; 5:286–304.

Rowe

, Gibson

, Morley

, Krupinski

. Ten-year experience of a private nonprofit telepsychiatry service. Telemed J E Health, 2008; 14:1078–1086.

10.

Valdagno

, Goracci

, di Volo

, Fagiolini

. Telepsychiatry: New perspectives and open issues. CNS Spectrums, 2014; 19:479–481.

11.

Shore

JH.

Telepsychiatry: Videoconferencing in the delivery of psychiatric care. Am J Psychiatry, 2013; 170:256–262.

12.

Hyler

, Gangure

, Batchelder

. Can telepsychiatry replace in-person psychiatric assessments? A review and meta-analysis of comparison studies. CNS Spectrums, 2005; 10:403–413.

13.

Kruse

, Krowski

, Rodriguez

, Tran

, Vela

, Brooks

. Telehealth and patient satisfaction: A systematic review and narrative analysis. BMJ Open, 2017; 7:e016242.

14.

Saliba

, Legido-Quigley

, Hallik

, Aaviksoo

, Car

, McKee

. Telemedicine across borders: A systematic review of factors that hinder or support implementation. Int J Med Inform, 2012; 81:793–809.

15.

Simpson

Psychotherapy via videoconferencing: A review. Br J Guid Couns, 2009; 37:271–286.

16.

Williams

, May

, Esmail

. Limitations of patient satisfaction studies in telehealthcare: A systematic review of the literature. Telemed J E Health, 2001; 7:293–316.

17.

Conn

, Madan

, Lam

, Patterson

, Skirten

. Program evaluation of a telepsychiatry service for older adults connecting a university-affiliated geriatric center to a rural psychogeriatric outreach service in Northwest Ontario, Canada. Int Psychogeriatr, 2013; 25:1795–1800.

18.

Scott Kruse

, Karem

, Shifflett

, Vegi

, Ravi

, Brooks

. Evaluating barriers to adopting Telemedicine worldwide: A systematic review. J Telemed Telecare, 2018; 24:4–12.

19.

Rogers

EM.

Diffusion of innovations. New York: Simon and Schuster, 2010.

20.

Whitten

, Love

. Patient and provider satisfaction with the use of telemedicine: Overview and rationale for cautious enthusiasm. J Postgrad Med, 2005; 51:294.

21.

Mair

, Goldstein

, May

, Angus

, Shiels

, Hibbert

, et al. Patient and provider perspectives on home telecare: Preliminary results from a randomized controlled trial. J Telemed Telecare, 2005; 11(1_,suppl):95–97.

22.

Weinstock

, Nguyen

, Risica

. Patient and referring provider satisfaction with teledermatology. J Am Acad Dermatol, 2002; 47:68–72.

23.

Scientific Protocol MasterMind. Scientific trial protocol. 2015. Available at https://mastermind-project.eu/scientific-publications. (last accessed January 15, 2019 ).

24.

Vis

, Kleiboer

, Prior

, Bønes

, Cavallo

, Clark

, et al. Implementing and up-scaling evidence-based eMental health in Europe: The study protocol for the MasterMind project. Internet Interv, 2015; 2:399–409.

25.

Kidholm

, Ekeland

, Jensen

, Rasmussen

, Pedersen

, Bowes

, et al. A model for assessment of telemedicine applications: Mast. Int J Technol Assess Health Care, 2012; 28:44–51.

26.

Greenhalgh

, Robert

, Macfarlane

, Bate

, Kyriakidou

. Diffusion of innovations in service organizations: Systematic review and recommendations. Milbank Q, 2004; 82:581–629.

27.

Attkisson

, Greenfield

. The use of psychological testing for treatment planning and outcomes assessment: Instruments for adults. Mahwah, NJ: Lawrence Erlbaum Associates Publishers,, 2004; 799–811.

28.

Luxton

, Pruitt

, Wagner

, Smolenski

, Jenkins-Guarnieri

, Gahm

. Home-based telebehavioral health for

U.S

. military personnel and veterans with depression: A randomized controlled trial. J Consult Clin Psychol, 2016; 84:923–934.

29.

Attkisson

, Zwick

. The Client Satisfaction Questionnaire: Psychometric properties and correlations with service utilization and psychotherapy outcome. Eval Program Plann, 1982; 5:233–237.

30.

Garcia-Lizana

, Munoz-Mayorga

. What about telepsychiatry? A systematic review. Prim Care Companion J Clin Psychiatry, 2010; 12.

31.

Simpson

, Reid

. Therapeutic alliance in videoconferencing psychotherapy: A review. Aust J Rural Health, 2014; 22:280–299.

32.

Sinclair

, Holloway

, Geoffrey Riley

, Auret

. Online mental health resources in rural Australia: Clinician perceptions of acceptability. J Med Internet Res, 2013; 15:e193.

33.

Whitten

, Mackert

. Addressing telehealth's foremost barrier: Provider as initial gatekeeper. Int J Technol Assess Health Care, 2005; 21:517–521.

34.

Wynn

, Bergvik

, Pettersen

, Fossum

. Clinicians' experiences with videoconferencing in psychiatry. Stud Health Technol Inform, 2012; 180:1218–1220.

35.

Gibson

, O'Donnell

, Coulson

, Kakepetum-Schultz

. Mental health professionals' perspectives of telemental health with remote and rural First Nations communities. J Telemed Telecare, 2011; 17:263–267.

36.

Powell

, Henstenburg

, Cooper

, Hollander

, Rising

. Patient perceptions of telehealth primary care video visits. Ann Fam Med, 2017; 15:225–229.

37.

Leavitt

, Whisler

. Management in the 1980's: November. Brighton, MA: Harvard Business Review, 1958.

38.

Saeed

, Bloch

, Diamond

. Telepsychiatry: Overcoming barriers to implementation: Providing treatment via videoconferencing can improve access to care. Curr Psychiatry, 2012; 11:28–32.

39.

Markus

, Robey

. Information technology and organizational change: Causal structure in theory and research. Manage Sci, 1988; 34:583–598.