Psychotherapists’ perspective of the use of eHealth services to enhance positive mental health promotion

Abstract

Objective

Keyes’s two-continua model of mental health proposes that mental illness and positive mental health are two separate, correlated, unipolar dimensions. eHealth services have been used to deliver mental health care, though the focus remained largely on symptom reduction and management of negative aspects of mental health. The potential of eHealth services to promote positive mental well-being, however, has not been tapped sufficiently. The present study aims to explore psychotherapists’ perspective on the feasibility of eHealth services to enhance positive mental health promotion.

Methods

Seven focus group discussions were conducted among professionals (n = 38) who delivered psychotherapy to examine positive mental health in their practice. Responses related to the use of e-psychotherapy to promote mental well-being were extracted for use in a secondary analysis of data in this study. Thematic analysis of data via inductive approach was conducted to allow emergence of common themes.

Results

Three main themes related to psychotherapists’ perspective on the feasibility of eHealth intervention in enhancing positive mental health were identified: (1) use of eHealth to educate and improve positive mental health; (2) concerns on incorporating psychotherapy into online services; (3) other factors that affect uptake or effectiveness of eHealth intervention for positive mental health.

Conclusions

The study generally found support among psychotherapists for the feasibility of eHealth intervention in promoting positive mental health among clients. Potential difficulties in implementation and practicality concerns were discussed.

Keywords

Positive mental health mental well-being psychotherapy online intervention qualitative study

Background

Positive mental health is the foundation for a good well-being and the effective functioning of the self and community. It is more than the absence of mental illness. Keyes’s two-continua model of mental health proposes that mental illness and positive mental health reflect two distinct dimensions, rather than the extreme ends of a single spectrum.¹ Those living with a mental disorder can therefore experience positive mental health, and it is also possible for individuals to experience poor mental health without a mental health condition. Multiple aspects of positive mental health or well-being have been identified and they include perceived self-efficacy, autonomy, competence, recognition of the ability to realize one's intellectual and emotional potential, positive affect, satisfaction with life, self-esteem, purpose in life and sense of hope, as well as energy and vitality.^2–5

Psychiatric care has traditionally focused on elucidating the causes of mental illness, providing safe and effective treatments and reducing the associated suffering and disability but these do not adequately address the global crisis of mental health care, particularly in promoting human welfare.⁶ In line with Keyes’s dual-continua model of mental health, there has been an increased emphasis on the need to promote well-being and positive mental health as a further measure beyond symptom reduction to prevent the development of mental disorders.^7,8 Positive mental well-being has been found to improve individual's functioning such as increased productivity at work, better physical health and having more meaningful relationships.^9,10 Various psychological, social and behavioural factors have been known to protect health and support positive mental health. For example, psychological attributes such as optimism, self-esteem and self-efficacy exert healthy influence in early development years and foster resilient climate for future growth.^11–13

eHealth involves the use of modern electronic information and communication technologies in the delivery of health-related activity and eHealth interventions include services delivered online, by phone and through mobile applications (‘apps’).¹⁴ Currently, eHealth services have been widely used for mental health care due to distinctive features such as privacy, accessibility, affordability, acceptability and cost-efficiency.¹⁵ Unlike traditional mental health care, eHealth applications reduce patients’ threshold to seek both information and help, as they can be utilised privately from home environment¹⁶ and have the potential to overcome the fear of shame, guilt and stigma, which are common concerns of mental health patients by providing anonymity.¹⁷ Furthermore, the need of remote psychotherapy to support vulnerable populations has become more evident during the outbreak of the novel coronavirus disease-2019 (COVID-19).^18,19 A variety of eHealth services and applications have been developed for the treatment of common mental health problems such as depression, anxiety, sleep and eating disorders and substance use disorders, as well as the management of stress or psychological distress.^20–23 Such interventions include components such as cognitive training, behavioural activation, mindfulness approaches, psychoeducation, or monitoring of symptoms. To date, the delivery of existing eHealth interventions appears to emphasize on reducing or managing the negative aspects of mental health (i.e. targeting mental disorders and symptoms) but less on promoting the positive aspects of mental health.

When health providers endorse and ‘prescribe’ these web or mobile-based tools, consumer performance expectancy is heightened, and adoption rates by patients tend to greatly improve.^24,25 Mental health professionals’ perspective has been shown to be vital in the successful implementation, uptake and dissemination of e-mental health resources.^26,27 The present qualitative study therefore aimed to evaluate the use of eHealth interventions to improve positive mental health from the perspectives of psychotherapists as they are identified as gatekeepers to clinical knowledge whom patients rely on for opinions about clinical resources.²⁴ Specifically, we intend to explore whether and how positive mental health components can be further integrated with e-psychotherapy practice to support clients better.

Methods

Participants

This study recruited professionals who provided psychotherapy to people with mental health problems at either public or private institutions in Singapore. Participants were recruited using a mix of personal network and by word of mouth, while adopting purposive sampling approach to achieve diversity in the sample in terms of work experience (e.g. length of work, employment sector). Data collection took place from April to November 2019. Individuals who expressed interest to participate in the study were contacted via phone calls and emails, and provided with more details of the study. Inclusion criteria included participants aged 21 years and above and those who were able to provide consent. All participants provided written informed consent and received an inconvenience fee upon completion of the study. Approval of the study was obtained from the institutional ethics committee, the Domain Specific Review Board of National Healthcare Group, Singapore.

Study procedures

The current analysis is part of a bigger study, which aimed to identify psychotherapy-based interventions and strategies to improve positive mental health among clients. For the purpose of this study, we included only discussion relevant to the use of eHealth interventions to improve positive mental health.

Focus group discussions (FGDs) were facilitated by a female senior researcher (JAV) who has a background in epidemiology and had extensive prior experience in qualitative research and domain knowledge on mental well-being. Each FGD session was conducted with about 4 to 6 participants and lasted between 1.5 and 2 hours. A study team member (ES, SC, RS) was also present during the session as a note taker. Before the FGD session, all participants completed a short questionnaire that collected data on their sociodemographic information and clinical experience. The FGD session then began with an icebreaker activity where all individuals present introduced themselves about their work and personal interests. An interview guide that was developed with inputs from clinicians and psychologists from the study team was used to facilitate the discussion. During the interview, participants were first asked about the interventions they have used in their clinical practice to promote positive mental health or psychological well-being, or how they would design interventions to address positive mental health. Questions in the guide were also included to encourage in-depth discussion and brainstorming on key elements for positive mental health-based interventions such as treatment phase, targeted client traits, factors affecting therapy effectiveness and the specific goal of such treatment(s). Although these questions were meant to address the primary aim of the original study, they provided a context for subsequent discussion and aided in eliciting relevant responses for the topic of interest for this study, which was to explore the feasibility of e-psychotherapy to promote positive mental health. After the initial discussion on designing positive mental health-based interventions, participants were asked the following question as a starting point for the next round of discussion: ‘Which of the interventions mentioned earlier could be applicable to be administered online or using e-modules?’ Additionally, participants were asked to describe components of the online intervention that they would like to include and were also prompted to discuss any factors that they would consider when delivering these interventions online or via e-modules. Participants were given time to pen down their thoughts to what they would like to include in positive mental health e-modules for psychotherapy before allowing free exchange in discussion, with the facilitator requesting for clarification guided by the responses of the participants.

Analysis

All FGDs were audiotaped and transcribed verbatim. For the purpose of data coding and processing, NVivo Version11 (QSR International; Computer Software) was used. Data was analysed using thematic analysis via inductive approach to allow emergence of common themes.²⁸ Three study team members (JAV, ES and SC) independently read a transcript each based on random assignment and employed descriptive code to index meaningful segments or contents. These codes and themes were then discussed and combined to generate a list of preliminary themes, which was used to code the remaining transcripts. Additional codes were created and reviewed to capture unexpected themes, which emerged during the course of reading the remaining transcripts. Finally, emergent themes were progressively integrated to produce higher-order themes based on their contexts. Any disagreements among team members were resolved through discussions.

Results

A total of 38 participants were recruited, with 7 FGDs conducted when data reached saturation (see Table 1). Participants ranged between 27 and 63 years old and were mostly females (84.2%) and of Chinese ethnicity (81.6%). The duration of their employment ranged between 2 months and 17 years. Majority of the participants worked with adult clients with depression and anxiety, as well as other mental disorders such as personality disorder, post-traumatic stress disorder, obsessive compulsive disorder and schizophrenia. Some had also worked with children and adolescents with childhood disorders such as intellectual disability, autism and attention deficit hyperactivity disorder, as well as older adults with dementia or individuals who needed life coaching.

Table 1.

Sociodemographic of participants (n = 38).

		n
Gender	Male	6
Gender	Female	32
Age group	21–39	30
Age group	40–65	8
Ethnicity	Chinese	31
	Indian	4
	Others	3
Marital status	Single/never married	13
	Married	24
	Divorced/separated	1
Highest education	University degree	2
Highest education	Post-graduate degree (e.g. Masters/PhD)	36
Employment sector	Public institution	34
	Private practice/institution	3
	Both	1

Three main themes with respect to psychotherapists’ perspective on the feasibility of eHealth intervention to promote positive mental health were identified: (1) use of eHealth to educate and improve positive mental health; (2) concerns on incorporating psychotherapy into online services; and (3) other factors that affect the uptake or effectiveness of eHealth intervention for positive mental health. The main findings are summarised in Figure 1.

Figure 1.

Summary of main findings.

Use of eHealth to educate and improve positive mental health

Understanding positive mental health

A handful of participants mentioned the need to include online content and materials to educate the public or patients with mental illnesses on the concept of positive mental health to differentiate it from the absence of mental illness, as well as questionnaires for users to determine their individual level of positive metal health.

‘We can have things like patients’ testimonials of what positive mental health means to them so they (other patients) get a different perspective…getting [inputs] from people who have experienced something like that so that others can incorporate some of the positive mental health in their life…this is something we cannot get from our therapy itself’ – FGD2

‘I think it is to introduce what is positive mental health (PMH). Considering not everybody knows what it means. So I think that part will be quite important…then also for them to create their own plan as to how to promote their own PMH’ – FGD4

Aspects of positive mental health that could be targeted through e-intervention

Participants quoted various attributes or traits that could be targeted to improve positive mental health and would be appropriate for online intervention. These included the use of psychoeducation or resources on eHealth services to instil sense of gratitude, self-compassion, self-efficacy, self-esteem, confidence, growth mindset, empowerment, hope, resilience and mental alertness.

Participants also suggested using the eHealth services to impart life skills to achieve various aspects of positive mental health. For example, coaching techniques that could help to develop balanced perspectives, lifestyle changes, life goals, coping strategies, managing stress and conflicts, and building on social support and personal character strength. Having a feature to introduce relaxation was the most cited form of intervention recommended by the participants. Table 2 shows some additional suggestions regarding the specific aspects of positive mental health targeted by the participants, along with their recommended features.

Table 2.

Target aspects of positive mental health and the corresponding feature suggestions by participants.

Aspect of positive mental health	Suggested features by participants
Self-reflection	‘create some space for self-reflection, where there's a private function for the patient…only they can view it’ – FGD5
Self-awareness	‘a check list for people to assess themselves so they have an understanding of their [own] emotions, values, life goals, and behaviours’ – FGD3
Relaxation	‘if we could have a virtual reality kind of portal where they have multisensorial kind of experiences and we could augment and enhance all the different relaxation exercises that we can teach them’ – FGD7
Relationship building	‘incorporate MMORPG (massively multiplayer online roleplaying game) where all you adolescents are playing… in terms of addressing relationship and peer issues as they play out within the world’ – FGD7
Cognitive reframing	‘encourage them to think about more perspectives and then with sensorial and biofeedback, it is more about letting them see how their body reacts after they do, so that they see the change and are more likely to join in the future’ – FGD7

Concerns on incorporating psychotherapy into online services

Face-to-face versus online psychotherapy

Most participants believed that the online intervention could serve as an ‘extension’ or a ‘supplement’ to the face-to-face psychotherapy by imparting some of the self-help skills and techniques or providing homework that could actually help users to continue their practice beyond the therapy session.

‘Maybe [having] resources like sometimes we give out worksheets or templates, monitoring diaries’ – FGD3

Few participants had, however, perceived that online psychotherapy would not be as effective as face-to-face session with the therapist – ‘because electronically we cannot tailor much, we cannot give much feedback’ (FGD5). Others provided recommendations on implementing e-psychotherapy with face-to-face sessions for better treatment efficacy:

‘I would limit the number of online sessions and include components of face-to-face contact’ – FGD4

‘I probably prefer meeting them before they start e-module…[it] depends on what the patients want to get out of the e-module’ – FGD3

Several participants proposed e-modules to be implemented only after end of face-to-face therapy.

Suitability of therapy modalities

In terms of therapy modalities, participants generally agreed that not all forms of therapies could be administered online but many of these therapies do have components that can be administered in an e-module. Cognitive behavioural therapy (CBT), followed by mindfulness, were cited as the most suitable to be delivered as an e-module. Reasons for CBT were that it is typically ‘more structured’ and will be ‘more beneficial for a wider population’, without the need for personalization.

‘I could use it for what we call in CBT as SMART (specific, measurable, achievable, relevant and time-bound) goals that could be tracked online over the time, session by session’ – FGD6

Acceptance and commitment therapy (ACT) was also discussed as an viable option but opinions were mixed:

‘ACT is a bit too flexible. It is very tailored to individual needs’ – FGD5

‘I think partly because it (ACT) has very good manual. They have printed all the worksheets and it is very easy to pick up even for someone without any psychological knowledge’ – FGD6

Factors that affect the uptake or effectiveness of eHealth intervention for positive mental health

Target audience

Most participants felt that it is important to consider the nature of the users (age group, technology savviness, reading level, etc.) to increase the uptake of eHealth services for positive mental health, and to note the levels of motivation and commitment to use eHealth interventions within the client group in comparison with the general public.

‘I think this suited those who are psychological in mind and motivated, can walk through this on their own.’ – FGD6

‘If I am a member of the general public, I think usually when things are well, the desire to look for these kind of resources is not there….even if content is great.’ – FGD4

Screening for personalization

Several participants felt that for the eHealth intervention to be effective, there is a need to include baseline screening at the start (similar to case formulation in psychotherapy) to understand clients' concerns, clarify individual's behavioural goals or needs, and customise interventions that target their personal problem areas. Those who expressed severe underlying mental health conditions should be directed or encouraged to seek face-to-face professional help.

‘First part of the module would probably be trying to get the person to understand the contributions to their struggles and how they piece together, in essence like a formulation when doing therapy. And then the second part would be kind of practical, concrete strategies that they could implement to address their identifying needs from the first part.’ – FGD6

Recommendations to improve users’ motivation

Various strategies were proposed among the participants to increase users’ motivation to access eHealth interventions and enhance the effectiveness of the eHealth services in promoting positive mental health.

Giving users an incentive or using extrinsic reward to create the drive to encourage usage

Normalizing the e-services so that people with or without mental health will not feel stigmatized to access it

Sharing successful stories on how users have participated in certain techniques and treatment modalities and have achieved a positive outcome

Making e-services interactive or introduce novel content, particularly the psychoeducation components

Validate and recognize that users are going through some form of problems and difficulties, and ensure them that there is a way to deal with these issues

Have therapists acting as facilitators to guide and check in on users periodically

Assessment at the start to determine suitability, that is, whether users will benefit from use

Allow users to create their own well-being questionnaires and track their progress over time

Provide practical knowledge and tips that could be applied to one's daily life

Free access

Discussion

In light of the numerous technology-delivered interventions for symptom management among patients with mental health and/or substance use disorders, the current qualitative study aimed to evaluate the use of eHealth interventions for enhancing positive mental health among individuals with or without underlying mental health issues, from the perspective of psychotherapists. Overall, the interventions were found to be feasible, although some aspects of feasibility remain questionable and would need to be considered further.

Our study is a secondary analysis of data from a qualitative study to investigate how proposed face-to-face psychotherapy interventions for positive mental health could be applicable to be administered online or using e – modules based on professional input. While we did not adopt any established framework for conducting a feasibility study or specify any primary outcomes to determine feasibility, Bowen et al. has generally proposed that there are eight general areas of focus addressed by feasibility studies: acceptability, demand, implementation, practicality, adaptation, integration, expansion and limited efficacy testing.²⁹ Our FGDs attempted to discuss some of these areas of focus described by Bowen et al.

Firstly, areas that supported feasibility as identified in our study were as follows: findings related to aspects of implementation, practicality and adaptation generally indicated that positive mental health interventions, particularly building awareness, trait development and self-help strategies, can be potentially and successfully delivered to intended users. Factors affecting implementation difficulty were discussed with various recommendations proposed to improve feature/design and effectiveness. Practicality concerns such as ability of users to carry out intervention activities, best practices or evidence-based science and components of psychotherapy that could be delivered as e-modules were considered. Adaptation of positive mental health intervention via eHealth platform or with non-patient group was also collectively determined to be feasible in practice. Although eHealth in mental health care has typically been developed for treating psychiatric conditions among those suffering from them, other basic approaches to prevent specific mental illnesses or symptoms (depression, anxiety, burnout, or stress) in general populations were in line with strategies for positive mental health promotion. These approaches include self-management, improvement of cognition, skills training, supportive care and passive data collection, which were common among existing mental health apps and computer-delivered interventions.³⁰ There is consensus among our participants that the use of certain psychotherapy strategies such as psychoeducation, coaching, relaxation, CBT and mindfulness are suitable in promoting positive mental health via eHealth services owing to inherent features of these strategies. Such psychotherapy-based programs were similarly used in internet-based interventions to target or prevent mental disorders.³¹

There were also concerns on the feasibility of such e-PMH interventions. One major challenge found regarding acceptability and demand would be the type of individuals who would use the eHealth services. While these eHealth interventions are meant to benefit the community at large, their delivery and uptake may be problematic. Those who were suffering from mental disorders and had been seeking professional help may prefer their routine, face-to-face intervention given that their immediate concern would be to manage their symptoms rather than to improve their positive mental health. Psychotherapists were also sceptical about the effectiveness of eHealth services for delivering interventions compared to face-to-face mode. Those without mental health concerns may not be sufficiently distressed enough to want to access these e-modules or may feel stigmatized to use such mental health e-services. As a result, we postulated that those with mild-to-moderate psychological distress, and who prefer self-help techniques may be more motivated to utilise and more likely to benefit from such e-services. Further studies looking at acceptability of such interventions by various groups are warranted.

Despite the encouraging evidences regarding efficacy, eHealth interventions have consistently showed low rates of engagement.^32,33 For those who do start on eHealth interventions, majority failed to engage in even minimal use or complete the intervention (i.e. high drop-out rates).^34,35 A review looking at real-world uptake and engagement with digital self-help interventions for depression and anxiety cited completion and sustained use rates to range from 0.5% to 28.6%.³⁶ Various strategies around the features or designs of the eHealth interventions and other recommendations to increase uptake and effectiveness of such services were proposed in the current study. In particular, our therapists have highlighted the importance of tailoring an intervention to patients’ needs. eHealth interventions can be easily designed and applied to meet the demands of different users and customized content is more likely to be read, remembered and viewed as personally relevant, thus increasing the efficacy of eHealth services.^37,38

Several limitations of the study should be noted. Firstly, our feasibility assessment clearly lacks opinions and inputs of potential users. It is pertinent to hold future focus group discussions with the target population to understand how these interventions could fit with their daily-life activities or guide their behavioural choices before designing specific intervention for future testing. Secondly, majority of our participants were psychotherapists from clinical settings and some discussions on eHealth interventions tend to centre around symptom reduction in patients, which did not coincide with the topic of interest – positive mental health promotion. Such content were therefore removed from analysis. Furthermore, our participants may have limited familiarity and experience using eHealth interventions as their main mode of therapy, which could result in preference biases against eHealth approaches. Nonetheless, it has been found that during the COVID-19 pandemic, psychotherapists coped generally well with the rapid change from provision of psychotherapy through personal contact to psychotherapy via the internet.³⁹ Next, topics around integration, expansion and limited efficacy were not covered in our FGDs, and hence it is unknown to what extent would such interventions be feasible in these areas. Last but not least, we did not discuss on the potential ethical issues raised by the use of digital technologies in the mental health settings. Concerns around confidentiality, protection and governance of personal data, equity in access to care and safety were expressed by professionals in light of the shift of classic consultation model towards teleconsultation during the pandemic.^40,41

Overall, this study provided further insights into how eHealth interventions could be used to improve positive mental health, a relatively novel concept in e-mental health, and how e-psychotherapy could be incorporated into such interventions from professional viewpoint. More research to examine the various area of focus proposed by Bowel et al needs to be conducted to provide a more accurate perception of the feasibility of eHealth interventions for positive mental health promotion. Importantly, implementation data should be routinely gathered and reported to facilitate improved uptake and engagement.

Footnotes

Acknowledgements

The authors are grateful to all the study participants for sharing their valuable insights and making contributions to the research study.

Contributorship

All authors contributed to the study conception and design. ES, SC, RS, SHL, HA, GT were involved in recruiting participants and data collection, and interviews were conducted by senior author JAV. Supervision of the project was provided by MS. JAV, ES and SC conducted the analysis of data, and the first draft of the manuscript was written by ES and SC. All authors provided intellectual inputs and critically revised the manuscript, and have read and approved the final draft.

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Ethical approval

Ethical approval was obtained from the relevant institutional review board (Ref No: 2018/00870). All the participants provided written informed consent for the study and for audio-recording of the interviews. Verbatim transcripts were anonymized to safeguard participants’ identities.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Singapore Ministry of Health's National Medical Research Council under the Centre Grant Programme (NMRC/CG/M002/2017).

Guarantor

ORCID iD

Sherilyn Chang

References

Keyes

CLM

. Mental illness and/or mental health? Investigating axioms of the complete state model of health. J Consult Clin Psychol 2005; 73: 539–548.

Keyes

CLM

Dhingra

Simoes

. Change in level of positive mental health as a predictor of future risk of mental illness. Am J Public Health 2010; 100: 2366–2371.

Lehtinen

Sohlman

Kovess-Masfety

. Level of positive mental health in the European Union: results from the eurobarometer 2002 survey. Clin Pract Epidemiol Ment Heal 2005; 1: 9.

Shek

. Adolescent positive mental health and psychological symptoms: a longitudinal study in a Chinese context. Psychologia 1998; 41: 217–225.

Srivastava

. Positive mental health and its relationship with resilience. Ind Psychiatry J 2011; 20: 75–76.

Jeste

, et al. Positive psychiatry: its time has come. J Clin Psychiatry 2015; 76: 675–683.

Iasiello

Agteren

Muir-Cochrane

, Mental health and/or mental illness: a scoping review of the evidence and implications of the dual-continua model of mental health. Evidence Base 2020. 2020: 1–45.

Singh

Kumar

Gupta

. Mental health prevention and promotion – a narrative review. Front Psychiatry 2022; 13: 898009.

Howell

Kern

Lyubomirsky

. Health benefits: meta-analytically determining the impact of well-being on objective health outcomes. Health Psychol Rev 2007; 1: 83–136.

10.

Keyes

CLM

Grzywacz

. Health as a complete state: the added value in work performance and healthcare costs. J Occup Environ Med 2005; 47: 523–532.

11.

Barry

Jenkins

(2007) Implementing Mental Health Promotion.

12.

Bartley

(2006) Capability and resilience: beating the odds. University College London Department of Epidemiology and Public Health, London.

13.

Werner

Smith

(1982) Vulnerable, but invincible: a longitudinal study of resilient children and youth. New York: McGraw-Hill.

14.

Kluge

E-H.W

. Chapter 5 – Framework considerations. The Electronic Health Record: Ethical Considerations. Academic Press. 2020, 105–133.

15.

Basnet

. The feasibility of eHealth in mental health care. J Addict Res Ther 2014; 05: 205. https://doi.org/10.4172/2155-6105.1000205

16.

Ball

Lillis

. E-health: transforming the physician/patient relationship. Int J Med Inform 2001; 61: 1–10.

17.

Davison

Pennebaker

Dickerson

. Who talks? The social psychology of illness support groups. Am Psychol 2000; 55: 205–217.

18.

Humer

, et al. Provision of psychotherapy during the COVID-19 pandemic among Czech, German and Slovak psychotherapists. Int J Environ Res Public Health 2020. 17(13): 4811. https://dx-doi-org.web.bisu.edu.cn/10.3390/ijerph17134811

19.

Humer

, et al. Experiences of psychotherapists with remote psychotherapy during the COVID-19 pandemic: cross-sectional web-based survey study. J Med Internet Res 2020; 22: p. e20246.

20.

Boumparis

, et al. Short- and long-term effects of digital prevention and treatment interventions for cannabis use reduction: a systematic review and meta-analysis. Drug Alcohol Depend 2019; 200: 82–94.

21.

Ben-Zeev

, et al. Transdiagnostic mobile health: smartphone intervention reduces depressive symptoms in people with mood and psychotic disorders. JMIR Ment Health 2019; 6: e13202.

22.

Moghimi

Davis

Rotondi

. The efficacy of eHealth interventions for the treatment of adults diagnosed with full or subthreshold binge eating disorder: systematic review and meta-analysis. J Med Internet Res 2021; 23: e17874.

23.

Currie

, et al. The impact of eHealth group interventions on the mental, behavioral, and physical health of adults: a systematic review protocol. Syst Rev 2020; 9: 217.

24.

East

Havard

. Mental health mobile apps: from infusion to diffusion in the mental health social system. JMIR Ment Heal 2015; 2: e10.

25.

Schueller

Washburn

Price

. Exploring mental health providers’ interest in using web and mobile-based tools in their practices. Internet Interv 2016; 4: 145–151.

26.

Hollis

Sampson

Simons

, et al. Identifying research priorities for digital technology in mental health care: results of the James Lind alliance priority setting partnership. Lancet Psychiatry 2018; 5: 845–854.

27.

Martinez-Martin

Kreitmair

. Ethical issues for direct-to-consumer digital psychotherapy apps: addressing accountability, data protection, and consent. JMIR Ment Heal 2018; 5: e32.

28.

Braun

Clarke

. Using thematic analysis in psychology. Qual Res Psychol 2006; 3: 77–101.

29.

Bowen

Kreuter

Spring

, et al. How we design feasibility studies. Am J Prev Med 2009; 36: 452–457.

30.

National Institute of Mental Health. (2019) Technology and the future of mental health treatment. https://www.nimh.nih.gov/health/topics/technology-and-the-future-of-mental-health-treatment/index.shtml

31.

Sander

Rausch

Baumeister

. Effectiveness of internet-based interventions for the prevention of mental disorders: a systematic review and meta-analysis. JMIR Ment Heal 2016; 3: e38.

32.

Christensen

Proudfoot

Woodward

, et al. e-Mental health services in Australia 2014: current and future, 2014.

33.

McClure

Greene

Wiese

, et al. Interest in an online smoking cessation program and effective recruitment strategies: results from project quit. J Med Internet Res 2006; 8: e14.

34.

Moreira

TdC

Signor

Figueiró

, et al. Non-adherence to telemedicine interventions for drug users: systematic review. Rev Saude Publica 2014; 48: 521–531.

35.

Stockings

. Commentary on boumparis, Et al. (2017): it works, but only if they use it—addressing treatment adherence and attrition in internet interventions for illicit substance use. Addiction 2017; 112: 1533–1534.

36.

Fleming

Bavin

Lucassen

, et al. Beyond the trial: systematic review of real-world uptake and engagement with digital self-help interventions for depression, low mood, or anxiety. J Med Internet Res 2018; 20: e199.

37.

Brug

Campbell

van Assema

. The application and impact of computer-generated personalized nutrition education: a review of the literature. Patient Educ Couns 1999; 36: 145–156.

38.

Dijkstra

De Vries

. The development of computer-generated tailored interventions. Patient Educ Couns 1999; 36: 193–203.

39.

Humer

, et al. Psychotherapy via the internet: what programs do psychotherapists use, how well-informed do they feel, and what are their wishes for continuous education? Int J Environ Res Public Health 2020. 17(21): 8182. https://doi.org/10.3390/ijerph17218182

40.

Kane

, et al. Opportunities and challenges for professionals in psychiatry and mental health care using digital technologies during the COVID-19 pandemic: systematic review. JMIR Hum Factors 2022; 9: e30359.

41.

Smith

, et al. COVID-19 and telepsychiatry: development of evidence-based guidance for clinicians. JMIR Ment Health 2020; 7: e21108.