Differences in Attitudes Toward Online Interventions in Psychiatry and Psychotherapy Between Health Care Professionals and Nonprofessionals: A Survey

Abstract

Background:

Although the use of e-mental health interventions and their evaluation is already well advanced in countries such as the United States and Australia, research in this area is still in the early stages in Germany. Moreover, existing programs are used only to a small extent by patients, although physicians and therapists generally have a positive attitude toward their use. To help promote the use of online interventions in the future, an analysis of the differences in opinions and attitudes toward e-mental health interventions between health care professionals and nonprofessionals is necessary.

Objective:

This study aimed to examine the differences in attitudes toward online interventions between health care professionals and nonprofessionals.

Methods:

This study examined 92 physicians, 36 psychotherapists, and 1,353 randomly recruited nonprofessionals with the eight-item questionnaire entitled “Attitudes on telemedicine in psychiatry and psychotherapy (ATiPP).”

Results:

The questionnaires of n = 62 physicians, n = 37 psychotherapists, and n = 1,353 nonprofessionals were included in the analysis. Overall, nonprofessionals rate the use of telemedicine more critically than professionals. The itemwise t tests show significant differences between health care professionals and nonprofessionals on six out of eight items. The analyses of variance with post hoc tests for each single item also found differences between the groups (physicians vs. therapists vs. telephone participants vs. practice sample).

Conclusion:

There are significant differences in attitudes toward online interventions between professionals and nonprofessionals.

Introduction

The internet is used daily by only about 72% of all Germans, although a total of 95% have access to the internet.¹ This figure is similar in all Western countries, with an estimated 85% of people with internet access in Europe and 88% in Northern America.² In recent years, the increase has been exponential and it is to be expected that the distribution and access to modern technologies will continue even faster in the future. The internet allows easy access to a wide range of information, including health care information. Particularly in the case of disorders, which are unpleasant and shameful to talk about, patients actively seek information on the internet.³ In addition, utilization of the internet for seeking health-related information seems to increase as well. Although studies from 2000 showed that 53.5% of the participants reported using the internet for health information,⁴ more recent studies show utilization at 66.7%.⁵ However, using the internet as the only source for information without validation by a health care professional does not seem to be an attractive option for the general population. A Norwegian study examined user characteristics of health-related online forums. They found that the registered users sought information, social contacts, and support from peers. However, the users also expressed their need for a contact to professionals, who they wanted to also register in that forum and give support and advice.⁶

Using the internet for seeking health information can have advantages (e.g., decreasing health care disparities, informed health care decisions, and efficient use of clinical time) as well as disadvantages (mainly misinformation due to unreliable sources), but health care professionals cannot simply ignore their patients' information seeking behaviour.⁷ The discussion of health information, which was found on the internet, seems to become an important factor within the doctor–patient interaction in the sense of a triangulation (the internet as a third component between the physician and the patient).⁸ Therefore, it is only a logical consequence to integrate internet-based programs not only into the information but also in the treatment of patients.

Currently, there is no official definition of the term telemedicine. A literature review and analysis of >100 different definitions conclude that telemedicine refers to the delivery of health care services and medical education from one geographical location to another, using communication networks.⁹ This very broad definition already shows that a variety of different technologies are summarized in this study, for example, telephone, fax and e-mail, smartphone applications, or online (internet-based). The efficacy of internet-based therapeutic approaches in some medical disciplines has been evaluated in several other studies. Although a large part of these studies in the field of mental health tested internet-based treatment procedures against control or waiting groups, some studies also compared internet interventions to conventional face-to-face therapy.⁸ Studies on programs for treating anxiety or mood disorders have proven that therapist-guided internet-based cognitive behavioral therapy (CBT) can produce the same treatment results as to face-to-face therapy.^10

–13 Internet interventions in mental health are almost always based on CBT and a large number of programs for different mental illnesses have already been developed.¹⁴

The advantage of network-based methods is, above all, the removal of spatial and temporal limits (better accessibility for patients who cannot attend a regular face-to-face therapy due to local conditions or physical limitations), anonymity of participation (reduction of inhibition thresholds), and in the economy of application.¹⁵ Nevertheless, internet interventions are not in use around the world as often as one would expect given the good results of studies, especially in Germany. The approaches are not yet part of regular health care in Germany, and the existing programs are used only to a small extent by patients.¹⁶

One possible explanation for this discrepancy between the failure to use reliable telemedicine treatments and a generally high willingness to address health issues through the internet can be found in the German (and likely every other European) health care system. German health insurance covers a wide range of treatments, including psychotherapy, which means that every person with health insurance (∼99%) is entitled to access to psychotherapy. However, there is a supply gap because there are not enough psychotherapists available. The average wait time for a psychotherapy in Germany is about 3 months.¹⁷ Still, the acceptance of telemedicine in Germany, especially in the case of e-mental health, might be negatively influenced by the relatively easy access and the nearby no costs for the patient of a face-to-face treatment, especially psychotherapy.

Another explanation might be the attitudes and expectations toward telemedicine, which are examined in this study. The attitudes of health care professionals and nonprofessionals have been sufficiently investigated by now. Health care nonprofessionals show very mixed attitudes toward telemedicine. In general, the benefits of telemedicine (time and space benefits) are appreciated by users,¹⁸ but we also find concerns regarding data protection misunderstandings due to misinterpretations of what has been written online.¹⁹ Also, the interest in telemedicine seems to be higher as well as the acceptance of e-mental health applications if corresponding applications have already been used.²⁰ Regarding the acceptance of telemedicine after using said applications, we find no differences between health care professionals and nonprofessionals.²⁰ Although psychotherapists actually do integrate modern media into their daily clinical routine, especially for communicating with their patients, they seem to prefer the personal contact. However, they still claim a neutral view toward e-mental health.²¹

However, to our knowledge, there has been no comparison of the attitudes of laypeople and health care professionals toward telemedicine in general. Owing to the varying degrees of medical knowledge and treatment knowledge that professionals and laypeople possess, we expect differences in opinions and attitudes toward e-mental health interventions between health care professionals and nonprofessionals. These differences might have a strong influence on the doctor–patient communication. The influence of patient satisfaction and doctor–patient interaction on patient adherence to treatment has been thoroughly investigated for decades.^22

–25 Studies show that a positive doctor–patient relationship and effective communication have a positive effect on health outcomes.²⁵ Improving the communication between doctors and their patients can improve the treatment adherence as well as the recall of information and clinical outcomes.²⁶ Also, the doctor–patient relationship is influenced by modern technologies. Changes in this relationship are to be expected when in addition to personal contact modern communication technologies are used. Miller²⁷ introduced a model to show how health outcomes are affected by the medium doctors and patients use to communicate.

Introducing a telemedicine treatment to a patient might have an influence on the doctor–patient relationship as well, based on how similar the general expectations toward telemedicine are. Therefore, it is necessary to examine whether there are any differences in the general attitudes toward telemedicine between health care professionals and nonprofessionals. Knowing about these differences might help promoting the use of online interventions in the future. To our knowledge, there is no study yet investigating this topic.

Research Question

Do health care professionals and nonprofessionals differ in their attitudes on the use of e-mental health supplies?

Methods

We used the recently developed “Attitudes towards telemedicine in psychiatry and psychotherapy (ATiPP)” questionnaire, which consists of eight items that measure the attitudes of potential users toward online intervention. The questionnaire was originally developed in German, but an English version is currently in evaluation. There are three different versions of the questionnaire, each of which is adapted to a specific target group. These target groups are doctors, psychotherapists, and nonprofessionals. Table 1 shows the three different versions. Some of the items are identical, others differ minimally in the formulation. Item 6 measures the willingness to recommend a telemedical offer in the health care professional version and willingness to use a telemedical offer in the nonprofessional version. All items are rated on a five-step Likert scale, ranging from 1 (I completely agree) to 5 (I completely disagree). The higher values mean a more negative view of internet interventions. The questionnaire also gives a short introduction about telemedicine and collects demographic data (age, gender, and place of residence).²⁸

Table 1.

The Items of the Questionnaire ATiPP in the Three Versions for Laypeople, Physicians and Psychotherapists

NO.	LAYPEOPLE	PHYSICIANS	PSYCHOTHERAPISTS
1	Generally, telemedicine is a good addition to the medical services	Generally, telemedicine is a good addition to the medical services	Telemedicine in somatic medicine is a good addition to the medical services
2	For psychiatric or psychotherapeutic issues or mental illness, patient information through internet or telemedicine are very helpful	For psychiatric or psychotherapeutic issues or mental illness, patient information through internet or telemedicine is very helpful	For psychiatric or psychotherapeutic issues or mental illness, patient information through internet or telemedicine is very helpful
3	A successful treatment of patients with mental illness through internet or telemedicine is possible	An effective treatment of the patients with mental illness through internet or telemedicine is possible	An effective treatment of the patients with mental illness through internet or telemedicine is possible
4	The bridging of the waiting time for an appointment with a psychiatrist/psychotherapist by using the internet services or telemedicine is a sensible option	The bridging of the waiting time for an appointment in psychiatry/psychotherapy by using the internet services or telemedicine is a sensible option	The bridging of the waiting time for an appointment in psychiatry/psychotherapy by using the internet services or telemedicine is a sensible option
5	Aftercare and counseling after a presence therapy by a psychiatrist or psychotherapist through contacts via the internet, e-mail, or telephone are realizable	Aftercare and stabilization after a presence therapy by a psychiatrist or psychotherapist through contacts via the internet, e-mail, or telephone are realizable	Aftercare and stabilization after a presence therapy by a psychiatrist or psychotherapist through contacts via the internet, e-mail, or telephone are realizable
6	I would make use of web-based interventions or telemedicine without an accompanying face-to-face therapy in the case of a mental illness	I would absolutely recommend my patients with psychiatric or psychotherapeutic treatment needs an online intervention or telemedical support, if such were to be offered for the clinical picture	For my own patients, I would offer support and intervention through the internet or telephone
7	An online therapy through internet services/telemedicine is only sensible as an addition to face-to-face therapy	In addition to a face-to-face therapy, an accompanying psychoeducational or psychosocial or additional intervention through the internet is sensible	In addition to a face-to-face therapy, an accompanying psychoeducational or psychosocial or additional intervention through the internet is sensible
8	An online therapy through the internet services or telemedicine can only work effectively with live contacts with a therapist through video calling and e-mail or chat	An online therapy through the internet services or telemedicine for mental illness can only work effectively with live contacts with a therapist through video calling and e-mail or chat	An online therapy through the internet services or telemedicine for mental illness can only work effectively with live contacts to a therapist through video calling and e-mail or chat

Recruitment

The Ethics Committee of the Hamburg Medical Association approved of the study as harmless and noninfluential for the participants. Between March and June 2016, participants were recruited through telephone and were interviewed about their attitudes toward telemedicine. Furthermore, we approached patients in the waiting area of different medical centers and randomly recruited participants from a medical center for psychiatric and neurological disorders (PN), participants from a gynecological practice (GYN), and participants from a general practitioner practice (GP). We excluded participants, who answered two or less items, as well as participants, who did not answer any demographic questions. The final group of participants that we contacted was the group of health care professionals. We randomly contacted physicians and psychotherapists through letter, asking them to participate and fill out the ATiPP. The return postage was paid for in advance.

Statistical Analyses

We used the open source program R^®, version 3.2.4 (The R Foundation), to perform statistical analyses.

Results

Descriptive Statistics

Between March and June 2016, a total of N = 1,000 nonprofessionals (age 16 or older) were recruited through telephone and interviewed about their attitudes toward telemedicine. These participants were categorized by age, gender, and place of residence. The resulting sample is representative for the German general population. Table 2 gives an overview of the descriptive statistics. Furthermore, we personally approached patients in the waiting area of different medical centers and randomly recruited 198 participants from a medical center for PN, 42 participants from a GYN, and 113 participants from a GP. In total, 92 physicians and 36 psychotherapists took part.

Table 2.

Epiemiological Data of the Survey

	GENDER	AGE (Years)	PLACE OF LIVING/PRACTICE	WORKING AS A PSYCHIATRIST
Professionals	F: 52.6% M: 47.4%	Under 36: 2.4% 36–45: 9.5% 46–55: 41.3% Over 55: 46.8%	Metropolitan: 44.6% Urban: 29.5% Rural: 25.9%	Yes: 72.6% No: 14.2% Missing: 13.3%
				Among physicians:Yes: 50%No: 33.7%Missing: 16.3%
Nonprofessionals	F: 54.9%M: 45.1%	Under 36: 30.4%36–45: 15.6%46–55: 17.6%Over 55: 46.8%	Metropolitan: 37.1%Urban: 14.6%Rural: 48.3%	—

Statistical Analyses

Overall, nonprofessionals rate the use of telemedicine more critically than professionals. The itemwise t tests show significant differences between health care professionals and nonprofessionals on the items 1, 2, 5, 6, 7, and 8. Table 3 lists the results of the performed t tests.

Table 3.

Results of the Study in a Group Comparison (Professionals vs. Laypeople)

ITEM	IV	T-VALUE	SIGNIFICANCE	EFFECT SIZE
Item 1: Telemedicine is a good addition to medical services	Professionals: M = 2.26 (SD = 1.09)Laypeople: M = 2.97 (SD = 1.25)	t = 6.872	p = 1.392e-10^***	d = 0.64
Item 2: Patient information through internet/telemedicine is possible	Professionals: M = 2.54 (SD = 1.14)Laypeople: M = 2.99 (SD = 1.22)	t = 4.040	p = 5.633e-05^***	d = 0.374
Item 3: An effective treatment of patients with mental illness through internet/telemedicine is possible	Professionals: M = 3.45 (SD = 1.20)Laypeople: M = 3.57 (SD = 1.21)	t = 1.058	p = 0.2902	d = 0.098
Item 4: The bridging of the waiting time by using the internet services or telemedicine is a sensible option	Professionals: M = 2.66 (SD = 1.26)Laypeople: M = 2.85 (SD = 1.27)	t = 1.6553	p = 0.09808	d = 0.153
Item 5: Aftercare and stabilization through contacts via the internet, e-mail, or telephone are realizable	Professionals: M = 2.45 (SD = 1.14)Laypeople: M = 2.86 (SD = 1.25)	t = 3.805	p = 0.0002029^***	d = 0.345
Item 6: use/recommendation of web-based interventions or telemedicine	Professionals: M = 2.89 (SD = 1.33)Laypeople: M = 3.92 (SD = 1.27)	t = 8.731	p = < 2.2e-16^***	d = 0.807
Item 7: An online therapy through internet services/telemedicine is only sensible as an addition to face-to-face therapy	Professionals: M = 2.39 (SD = 1.13)Laypeople: M = 2.73 (SD = 1.34)	t = 3.2252	p = 0.001524^**	d = 0.299
Item 8: An online therapy through the internet services or telemedicine can only work effectively with live contact with a therapist through video calling and e-mail or chat	Professionals: M = 2.66 (SD = 1.24)Laypeople: M = 2.91 (SD = 1.22)	t = 2.1314	p = 0.03322^*	d = 0.198

^* p < 0.05, ^** p < 0.01, ^*** p < 0.001.

We also used analyses of variance with post hoc tests for each single item to find differences between the groups (physicians vs. therapists vs. telephone participants vs. practice sample).

Item 1 shows significant differences between the groups (F = 23.07, p = 5.634e-12). The telephone sample (M = 3.06, SD = 1.27) is more critical than the practice sample (M = 2.70, SD = 1.15) (p = 5.7e-06, d = 0.294), more critical than the physicians (M = 2.23, SD = 1.10) (p = 2.2e-09, d = −0.664), and more critical than the therapists (M = 2.35, SD = 1.07) (p = 0.0034, d = −0.561). The practice sample is more critical than the physicians (p = 0.0026, d = −0.41).

Item 2 also shows significant differences (F = 7.717, p = 4.08e-05). The telephone sample (M = 3.04) (p = 3.6e-o5) and the practice sample (M = 2.86, SD = 1.19) are more critical than the physicians (M = 2.47, SD = 1.07; p = 0.023).

Item 3 would show significant differences (F = 3.199, p = 0.02563) between the groups if we used the common level of significance (0.05). However, since we use multiple tests, we have to adjust the level of significance (Bonferroni adjustment) to avoid type I error. We can only identify a nearly significant difference between the therapists (M = 3.89, SD = 1.21) and the physicians (M = 3.89, SD = 1.21) (therapists being more critical than physicians).

Item 4 shows significant differences (F = 10.977, p = 1.798e-06); the telephone sample (M = 2.93, SD = 1.29) is more critical than the practice sample (M = 2.60, SD = 1.17; p = 4.9e-05, d = 0.265) and more critical than the physicians (M = 2.43, SD = 1.18; p = 0.001, d = 0.392). Psychotherapists (M = 3.22, SD = 1.32) are also more critical than the physicians (p = 0.025, d = 0.633).

Item 5 also shows (F = 11.862, p = 6.362e-07) that the telephone sample (M = 2.95, SD = 1.28) is more critical than the practice sample (M = 2.60, SD = 1.14) (p = 1.4e-05, d = 0.279) and more critical than the physicians (M = 2.47, SD = 1.12) (p = 0.001, d = 0.379).

Item 6 shows (F = 27.24, p ≤ 2e-16) significant differences between the groups; the telephone sample (M = 3.97, SD = 1.28) is more critical than the physicians (M = 2.95, SD = 1.23) (p = 5.9e-11, d = 0.791) and more critical than the therapists (M = 2.75, SD = 1.57) (p = 0.00031, d = 0.931). The practice sample (M = 3.97, SD = 1.28) is also more critical than the physicians (p = 1.2e-07, d = 0.0708) and more critical than the therapists (p = 0.00240, d = 0.848).

Item 7 also shows (F = 16.496, p = 3.665e-09) that the telephone sample (M = 2.84, SD = 1.37) is more critical than the practice sample (M = 2.40, SD = 1.19) (p = 8.1e-08, d = 0.344) and more critical than the physicians (M = 2.23, SD = 1.05) (p = 5.8e-06, d = −0.457).

Item 8 also shows the same significant differences between the groups (F = 6.094, p = 0.0006603). Again, the telephone sample (M = 2.96, SD = 1.26) is more critical than the practice sample (M = 2.77, SD = 1.10) (p = 0.0331, d = 0.162) and more critical than the physicians (M = 2.51, SD = 1.14) (p = 0.0029, d = 0.361).

Discussion

The study reveals significant differences in attitudes toward online interventions between professionals and nonprofessionals. Overall, nonprofessionals rate the use of telemedicine more critically than professionals. This might be because nonprofessionals generally have less knowledge about their therapeutical options, differences between treatments, and benefits and efficacy. Psychotherapists in Germany are also critical toward the use of internet interventions. This might be the result of the fear of being replaced on the one side and scepticism about the feasibility of e-mental health on the other side.

Since internet interventions have many advantages over traditional face-to-face therapy, as described in the Introduction section, and due to the current development of technology, in general, it is to be expected that the number of internet interventions in psychiatry and psychotherapy will increase drastically over the next decade. Therefore, it is necessary to find ways to improve patient's attitudes toward telemedicine to promote the usage of internet interventions and also improve the treatment compliance. A positive doctor–patient relationship and a positive doctor–patient communication might provide a good basis for lively discussions and information.

There are several studies that research the changes in patient's satisfaction with telemedicine offerings²⁹ and attitudes toward such offers before and after use.³⁰ It turns out that patients develop a more positive attitude after use. By contrast, this does not seem to be the case with health care professionals.³¹ In terms of our study, this could mean that nonprofessionals may have a more negative attitude toward using telemedicine initially, but after use, this difference in groups might be reversed. Possible explanations could be that patients make positive usage experiences, whereas professionals are more disappointed with the results.

Furthermore, it would be interesting to analyze why there are differences in attitudes toward telemedicine between health care professionals and nonprofessionals. In this study, there are several possibilities conceivable. For one, nonprofessionals may not have the same amount of information about telemedicine services as professionals do. This problem could be solved by specific advice from the doctor on the basis of a good doctor–patient relationship. Another reason could be that the demands and expectations toward telemedicine services may be fundamentally different in both groups. Although physicians pay attention primarily to the effectiveness of an intervention, other factors such as accessibility, costs, and manageability might be more important to nonprofessionals. Knowledge about these factors is also important to make telemedicine offers more interesting to the intended users. All in all, to promote the use of internet interventions, health care professionals need to give more information about internet interventions, efficacy studies, and overall benefits and limitations to their patients.

On the other side, the study remembers that attitudes and expectations are relevant factors in the effects and the efficacy of every treatment. It makes sense that future studies on e-mental health will also measure participants' expectations of this new therapeutic options. In comparison with the basic data presented in this study and elsewhere, it can be determined whether the study population and thus possibly also good study results express positive expectations.

The results of this study show some implications for programmers and clinicians for the design of telemedicine services that should help increase the adoption of telemedicine.

To engage patients:

Keeping in mind that patients show a higher acceptance of telemedicine after using telemedicine services, you might consider providing a short demo version for patients to try.

Since the trust between doctor and patient might be a meditating factor, it might be useful if the practitioner has tried a demo version of the telemedicine service himself and can report his own experiences.

Provide information about the experiences of other users of your product.

Provide information about telemedicine in general, its efficacy, and different uses in the medical field.

Try to include the patient's physicians into the intended treatment, for example, through video conferences.

For engaging practitioners:

Provide clear information on the development and scientific basis of your telemedicine offer.

Evaluate your product and publish the results.

Again, offer a free demo version of your product.

Limitations and Future Research

Unfortunately, there are a few methodological limitations to our study. The ATiPP questionnaire, which was used to measure the attitudes toward telemedicine, still lacks a thorough evaluation with cross-correlation and retests. Although the results of the first evaluation²⁸ are rather promising (Cronbach's α = 0.85 and higher). In addition, not all of the items are identical in the three versions of the test. Identical items are 2, 4, and 8. Items that are very similar but not identical in the three versions are items 1, 3, 5, and 7. Item 6 is very different for the three versions of the ATiPP.

We are currently developing an e-learning program consisting of three lessons, which gives a brief overview of different kinds of e-mental health interventions and their efficacy. The attitudes toward telemedicine will be measured before and after doing the lessons. We want to examine whether attitudes toward online intervention can be influenced.

Footnotes

Disclosure Statement

No competing financial interests exist.

References

Koch

, B. ARD/ZDF-Onlinestudie 2017: Neun von zehn Deutschen online. Fachzeitschrift Media Perspektiven, 2017; 2017:434–446.

Internet World Stats Usage and Population Statistics. Internet users and. 2017. population in North America. 2018. Available at https://www.internetworldstats.com/stats14.htm (last accessed November 1, 2018 ).

Leung

. Internet embeddedness: Links with online health information seeking, expectancy value/quality of health information websites, and Internet usage patterns. Cyberpsychol Behav, 2008; 11:565–569.

Diaz

, Griffith

, Ng

, Reinert

, Friedmann

, Moulton

. Patients' use of the Internet for medical information. J Gen Intern Med, 2002; 17:180–185.

Bujnowska-Fedak

. Trends in the use of the Internet for health purposes in Poland. BMC Public Health, 2015; 15:194.

Kummervold

, Gammon

, Bergvik

, Johnsen

, Hasvold

, Rosenvinge

. Social support in a wired world: Use of online mental health forums in Norway. Nord J Psychiatry, 2002; 56:59–65.

Wald

, Dube

, Anthony

. Untangling the Web—The impact of Internet use on health care and the physician-patient relationship. Patient Educ Couns, 2007; 68:218–224.

Barak

, Hen

, Boniel-Nissim

, Shapira

. A comprehensive review and a meta-analysis of the effectiveness of internet-based psychotherapeutic interventions. J Technol Human Serv, 2008; 26:109–160.

Sood

, Mbarika

, Jugoo

, Dookhy

, Doarn

, Prakash

, et al. What is telemedicine? A collection of 104 peer-reviewed perspectives and theoretical underpinnings. Telemed J E Health, 2007; 13:573–590.

10.

Meyer

, Berger

, Caspar

, Beevers

, Andersson

, Weiss

. Effectiveness of a novel integrative online treatment for depression (Deprexis): Randomized controlled trial. J Med Internet Res, 2009; 11:e15.

11.

Moritz

, Schilling

, Hauschildt

, Schroder

, Treszl

. A randomized controlled trial of internet-based therapy in depression. Behav Res Ther, 2012; 50:513–521.

12.

Ruwaard

, Broeksteeg

, Schrieken

, Emmelkamp

, Lange

. Web-based therapist-assisted cognitive behavioral treatment of panic symptoms: A randomized controlled trial with a three-year follow-up. J Anxiety Disord, 2010; 24:387–396.

13.

Ruwaard

, Lange

, Schrieken

, Dolan

, Emmelkamp

. The effectiveness of online cognitive behavioral treatment in routine clinical practice. PLoS One, 2012; 7:e40089.

14.

Cuijpers

, van Straten

, Andersson

. Internet-administered cognitive behavior therapy for health problems: A systematic review. J Behav Med, 2008; 31:169–177.

15.

Berger

, Andersson

. [Internet-based psychotherapies: Characteristics and empirical evidence]. Psychother Psychosom Med Psychol, 2009; 59:159–166; quiz 67–70.

16.

Eichenberg

, Wolters

, Brahler

. The internet as a mental health advisor in Germany—Results of a national survey. PLoS One, 2013; 8:e79206.

17.

BPtK-Studie zu Wartezeiten in der ambulanten psychotherapeutischen Versorgung. Umfrage der Landespsychotherapeutenkammern und der BPtK. 2011.

18.

Ferwerda

, van Beugen

, van Burik

, van Middendorp

, de Jong

, van de Kerkhof

, et al. What patients think about E-health: Patients' perspective on internet-based cognitive behavioral treatment for patients with rheumatoid arthritis and psoriasis. Clin Rheumatol, 2013; 32:869–873.

19.

Gieselmann

, Böckermann

, Pietrowsky

. Internetbasierte Gesundheitsinterventionen. Evaluation aus der Perspektive von Patienten vor und während ambulanter Psychotherapie. Der Psychotherapeut, 2015; 60:433.

20.

Gun

, Titov

, Andrews

. Acceptability of Internet treatment of anxiety and depression. Australas Psychiatry, 2011; 19:259–264.

21.

Eichenberg

, Kienzle

. Psychotherapeuten und Internet Einstellung zu und nUtzung von therapeutischen Onlineangeboten im Behandlungsalltag. Der Psychotherapeut, 2013; 58:485–489.

22.

Vermeire

, Hearnshaw

, Van Royen

, Denekens

. Patient adherence to treatment: Three decades of research. A comprehensive review. J Clin Pharm Ther, 2001; 26:331–342.

23.

Ong

, de Haes

, Hoos

, Lammes

. Doctor-patient communication: A review of the literature. Soc Sci Med, 1995; 40:903–918.

24.

Stewart

. Effective physician-patient communication and health outcomes: A review. CMAJ, 1995; 152:1423–1433.

25.

Stewart

, Brown

, Boon

, Galajda

, Meredith

, Sangster

. Evidence on patient-doctor communication. Cancer Prev Control, 1999; 3:25–30.

26.

Harrington

, Noble

, Newman

. Improving patients' communication with doctors: A systematic review of intervention studies. Patient Educ Couns, 2004; 52:7–16.

27.

Miller

. Telemedicine and doctor-patient communication: A theoretical framework for evaluation. J Telemed Telecare, 2002; 8:311–318.

28.

Tonn

, Reuter

, Kuchler

, Reinke

, Hinkelmann

, Stöckigt

, et al. Development of a questionnaire to measure the attitudes of laypeople, physicians, and psychotherapists toward telemedicine in mental health. JMIR Ment Health, 2017; 4:39.

29.

Chae

, Hein Lee

, Hee Ho

, Ja Kim

, Hong Jun

, Uk Won

. Patient satisfaction with telemedicine in home health services for the elderly. Int J Medical Informatics, 2001; 61:167–173.

30.

Cranen

, Veld

, Ijzerman

, Vollenbroek-Hutten

. Change of patients' perceptions of telemedicine after brief use. Telemed J E Health, 2011; 17:530–535.

31.

Hanson

, Calhoun

, Smith

. Changes in provider attitudes toward telemedicine. Telemed J E Health, 2009; 1539–1543.