Effects of a mobile health intervention on activities of stress self-management for workers

Abstract

BACKGROUND:

Mobile health interventions are being widely tried because of their attractive advantages. However, there is not enough evidence for its effectiveness.

OBJECTIVE:

This study aimed to evaluate the effects of mobile app-based stress management intervention (mSMI) on highly stressed workers.

METHODS:

A sample of 82 white-collar workers with elevated symptoms of perceived stress (Perceived Stress Scale-10≥22) were randomly assigned to the intervention or control group. The mSMI consisted of three modules: self-management including a work diary, counseling based on cognitive behavioral therapy and interventions focused on music, meditation, relaxation and image healings. Self-report data were collected at the baseline and post-intervention. Study outcomes analyzed perceived stress, anxiety, depression, and work engagement. Data were assessed using analysis of covariance with covariates.

RESULTS:

There was significantly reduced perceived stress from baseline to 6 weeks in mSMI (t = 5.788, p < 0.001) and control group (t = 3.184, p = 0.003). After adjusting for covariates, the between-group difference in the perceived stress was significantly different (F = 4.051, p = 0.048); however, the effect size was small. There was no significant intervention effect on anxiety, depression, and work engagement. The process evaluation indicated that most participants (85.3%) were satisfied with the intervention and their mental health benefited.

CONCLUSION:

This study found that mobile health intervention facilitated perceived stress management for highly stressed workers. Further studies should address job-related outcomes and mental health symptoms in workers by applying the latest information technology and addressing the limitations of mobile interventions.

Keywords

Mental health mobile health application occupational health services office workers

1 Introduction

All workers are at risk of various mental health problems, including stress, depression, anxiety, posttraumatic stress disorder (PTSD), self-stigma [1], and insomnia [2]. These mental health problems can affect workers who have experienced workplace bullying [3], workplace violence [4], or disaster [5]. Anxiety and depression are reported to increase by 2.15 and 1.87 times, respectively, among workers in a manufacturing company working > 60 hours a week [6]. When a worker experiences symptom of deteriorating mental health, it can be difficult to focus on work [7]. A study of workers in the metallurgical industry showed a significant association between occupational stress and work engagement [8]. Employee health problems reduce work performance and increase sick leave [9]. Therefore, workers should engage in mental health-promoting activities that facilitate inspection and management of their mental health.

To keep up with the advances in information technology, mental health interventions are being offered using internet-based health (eHealth) programs, mobile communication, and social media [10]. Mobile health (mHealth) interventions that utilize smartphone apps help overcome geographical distance and allow limited face-to-face access [11]. Further, mHealth self-care interventions can be simultaneously administered to many people, which makes them an inexpensive strategy [12].

According to previous research, the eHealth stress management for highly stressed employees has improved perceived stress [13, 14], anxiety [13], burnout [13, 14], work engagement [13], and work experience [14]. However, the effects of mHealth interventions on workers have not been clearly identified. There have been inconsistent reports from studies to reduce stress of workers. One study [15] found that an in-person intervention for workers with work-related stress was more useful than a smartphone-delivered intervention. The smartphone-delivered intervention only promoted well-being; in contrast, the in-person intervention reduced stress and emotional burnout as well as promoted well-being. A mindfulness-based mHealth intervention for healthcare workers during the coronavirus pandemic (COVID-19) significantly improved depression, anxiety, stress, and insomnia in the group receiving psychotropic medications or psychotherapy as compared with the control group [16]. This shows that mHealth interventions are effective when coupled with other treatment modalities. Therefore, there remains limited evidence supporting the effectiveness of mHealth interventions in workers.

We aimed to develop a mobile app-based stress management intervention (mSMI) for highly stressed workers familiar with mobile devices who require mental health management, and evaluate the effects of the intervention.

2 Methods

2.1 Study design

We used an equivalent comparison group pretest-posttest design to evaluate the effectiveness of mSMI in workers with high perceived stress.

2.2 Participants

We enrolled workers aged≥20 years with high perceived stress (Perceived Stress Scale-10 [PSS-10]) score≥22 at the screening test who owned an Android-based mobile device and could use mobile apps. The selection criteria of participants were prepared based on meta-analyses findings that psychological wellbeing and work effectiveness outcomes were more significant through eHealth intervention in the study targeting participants with high levels of stress or depression [1, 9]. The exclusion criteria were a PSS-10 score <22, a self-reported diagnosis of mental illnesses, and current use of psychotherapy or psychotropic medications. This study was approved by the Institutional Review Board before data collection (CUIRB-2020-0041).

The sample size was calculated using the G* Power 3.1.7 software [17]. As a reference, we used the effect sizes reported by a study on an internet-based stress intervention for workers (Cohen d = 0.75) [14]. For an effect size of 0.70, significance level of 0.05, and power of 0.80 for two-tailed tests, the minimum sample size was calculated to be 68. Considering a 20% drop-out rate, 82 participants were required.

We performed convenience sampling of office workers in South Korea at public administration, information technology (IT), education, and health industries. We recruited participants through health managers and a recruitment announcement. Participation was voluntary, and participants provided informed consent. Among 634 individuals who participated in the screening test, we enrolled 82 eligible participants (selection rate 12.9%). The enrolled participants were randomly allocated to the experimental or control group. The participants were blinded to their group allocation. At the 6-week posttest survey, there were 64 participants in all groups, indicating a dropout rate of 22.0% (Fig. 1).

Fig. 1

Flow of participants’ recruitment, allocation and outcome assessment.

2.3 Intervention

To determine the contents of the mSMI, we searched for studies in databases using keywords, including “mental health,” “stress,” “workers,” “application,” “intervention,” and “self-management.” The contents and features for the mobile app were identified through collaboration with a mental health nurse. First, the “self-management” feature instructed participants to log in the times they start and finish work to monitor daily work hours, as well as check daily stress levels and work intensity. Additionally, the participants could record self-stress management efforts and methods, including taking walks and exercising. Second, a “counseling” feature based on cognitive behavioral therapy was added to help workers requiring counseling and assistance. Participants were asked to describe their feelings and thoughts during a specific problem situation, to describe their current thoughts and feelings, and to anticipate how they would deal with similar situations in the future. Two assigned mental health care professionals checked these notes and replied within 24 hours. Third, to facilitate stress relief, techniques such as music [18], meditation [19], relaxation [20], and image healing [21] were utilized. We included a poem recital feature with music and a breathing exercise [21] (Fig. 2).

Fig. 2

Screenshot of mobile application called MindCafe.

The contents of the developed app were evaluated by an expert panel comprising of college professors, nurses, and clinical psychologists. Furthermore, all items were confirmed to have an item content validity index (I-CVI)≥0.80. Additionally, the quality of the developed app was rated using the Mobile Application Rating Scale (out of 5) [22]. The app was rated 4.16 and 3.72 by five experts and five non-experts, respectively. After correcting errors and modifying the app based on the feedback, the mSMI (called MindCafe) for workers was finalized.

The mSMI was provided as a self-guided intervention. The certification process, how-to guides for each feature, and information regarding privacy policies were explained on the Frequently Asked Questions (FAQ) page. Participants were asked to use the app for 10 minutes twice a week for 6 weeks. During the intervention period, we sent out a short message service (SMS) three times a week to encourage participation. The reason for using SMS was that the meta-analysis results showed its usefulness in delivering psychoeducation and mobile app-based interventions, thus demonstrating that mobile apps are highly effective (effect size 0.05– 3.37) in reducing stress, anxiety, and depression immediately after the intervention [23]. The SMS contained messages for attempting to motivate the participants to utilize the counseling, music, meditation, and relaxation features. Further, we opened up a chat channel (Kakao channel) for participants to ask questions during the intervention period, including app downloading problems, instructions for each feature, and data entry. We responded to questions regarding the app around the clock. Using self-report questionnaires, data were collected at the baseline (before intervention) and 6 weeks after completing the intervention. The control group did not undergo anyintervention.

2.4 Measures

2.4.1 Perceived stress

Perceived stress was measured using the PSS-10, developed by Cohen [24] and translated into Korean and validated by Lee and colleagues [25]. This scale comprises 10 items, a higher score indicates higher perceived stress.

2.4.2 Anxiety and depression

Anxiety and depression were measured using the Hospital Anxiety and Depression Scale. This scale was developed for patients with physical illness; however, it has been validated for use in the general population and has been used for a large worker population in Korea [6]. This scale comprises 14 items, with seven items each for depression and anxiety. The total score for each subscale is 21, a higher score indicates more severe anxiety or depression symptoms.

2.4.3 Work engagement

Work engagement was measured using the Utrecht Work Engagement Scale (UWES)– Korean Version [7], which is the Korean version of the 9-item short form of the UWES developed by Schaufeli and Bakker [26]. This scale comprises three subscales, vigor (three items), dedication (three items), and absorption (three items). A higher score on the UWES indicates greater work engagement.

2.5 Statistical analyses

Statistical analyses were performed following the intention-to-treat principle (ITT) using the SPSS/WIN 24.0 software (IBM, Armonk, NY, USA). We used the multiple imputation procedure to impute missing sum scores for participants who didn’t complete the post intervention evaluation. Between-group differences in general characteristics were analyzed using the chi-square test, independent t-test, and analysis of variance. Within- and between-group differences in post-intervention changes were analyzed using a paired t-test and analysis of covariance (ANCOVA) with the baseline scores and age as covariates.

3 Results

Table 1 shows the results of the between-group homogeneity test. The mean age was 38.0±8.96 and 32.7±7.23 years in the experimental and control groups, respectively (t = 3.000, p = 0.004). There were no significant between-group differences in gender, marital status, education level, income, and workplace size.

Table 1
General characteristics of participants and differences between-group

Variables Categories Total (N = 82) mSMI (N = 39) Control (n = 43) χ²/t/F p

N (%) or N (%) or N (%) or

Mean±SD (range) Mean±SD (range) Mean±SD (range)

Gender Male 18 (22.0) 9 (23.1) 9 (20.9) 0.055 0.815

Female 64 (78.0) 30 (76.9) 34 (79.1)

Age (years) 35.2±8.49 38.0±8.96 32.7±7.23 3.000 0.004

(23.0∼66.0) (25.0∼66.0) (23.0∼50.0)

<30 28 (34.2) 8 (20.5) 20 (46.5) 7.876 0.019

30∼39 32 (39.0) 16 (41.0) 16 (37.2)

≥40 22 (26.8) 15 (38.5) 7 (16.3)

Marital status Unmarried 44 (53.7) 17 (43.6) 27 (62.8) 3.032 0.082

Married 38 (46.3) 22 (56.4) 16 (37.2)

Education ≤University 65 (79.3) 30 (76.9) 35 (81.4) 0.249 0.618

≥Master 17 (20.7) 9 (23.1) 8 (18.6)

Income (million KRW) <4 49 (59.8) 19 (48.7) 30 (69.8) 5.169 0.075

4∼4.9 16 (19.5) 8 (20.5) 8 (18.6)

≥5 17 (20.7) 12 (30.8) 5 (11.6)

Workplace size (number <50 20 (24.4) 8 (20.5) 12 (27.9) 0.702 0.873

of employees) 50∼299 17 (20.7) 9 (23.1) 8 (18.6)

300∼1,999 18 (22.0) 9 (23.1) 9 (20.9)

≥2,000 27 (32.9) 13 (33.3) 14 (32.6)

Variables	Categories	Total (N = 82)	mSMI (N = 39)	Control (n = 43)	χ²/t/F	p
Gender	Male	18 (22.0)	9 (23.1)	9 (20.9)	0.055	0.815
	Female	64 (78.0)	30 (76.9)	34 (79.1)
Age (years)		35.2±8.49	38.0±8.96	32.7±7.23	3.000	0.004
		(23.0∼66.0)	(25.0∼66.0)	(23.0∼50.0)
	<30	28 (34.2)	8 (20.5)	20 (46.5)	7.876	0.019
	30∼39	32 (39.0)	16 (41.0)	16 (37.2)
	≥40	22 (26.8)	15 (38.5)	7 (16.3)
Marital status	Unmarried	44 (53.7)	17 (43.6)	27 (62.8)	3.032	0.082
	Married	38 (46.3)	22 (56.4)	16 (37.2)
Education	≤University	65 (79.3)	30 (76.9)	35 (81.4)	0.249	0.618
	≥Master	17 (20.7)	9 (23.1)	8 (18.6)
Income (million KRW)	<4	49 (59.8)	19 (48.7)	30 (69.8)	5.169	0.075
	4∼4.9	16 (19.5)	8 (20.5)	8 (18.6)
	≥5	17 (20.7)	12 (30.8)	5 (11.6)
Workplace size (number	<50	20 (24.4)	8 (20.5)	12 (27.9)	0.702	0.873
of employees)	50∼299	17 (20.7)	9 (23.1)	8 (18.6)
		300∼1,999	18 (22.0)	9 (23.1)	9 (20.9)
		≥2,000	27 (32.9)	13 (33.3)	14 (32.6)

Note. mSMI = mobile app-based stress management intervention; SD = standard deviation; KRW = South Korean Won.

Table 2 shows post-intervention changes within each group. There was a significant post-intervention reduction in perceived stress in the experimental group (t = 5.788, p < 0.001) and control group (t = 3.184, p = 0.003). However, there were no significant post-intervention changes in anxiety, depression, and work engagement.

Table 2

Changes of post-intervention within-group (mSMI = 39, control = 43)

Variables	Categories	Baseline	6 weeks	Difference^*	t-test
		Mean±SD (range)	Mean±SD (range)	Mean±SD (range)	(p)
PSS-10	mSMI	24.79±2.34 (22.0∼32.0)	21.20±4.15 (12.0∼32.0)	3.60±3.88 (– 5.0∼14.0)	5.788 (<0.001)
	Control	24.44±2.56 (22.0∼31.0)	22.23±4.87 (9.0∼33.0)	2.21±4.56 (– 11.0∼13.0)	3.184 (0.003)
Anxiety	mSMI	9.44±3.60 (2.0∼17.0)	8.59±2.73 (0.0∼13.0)	0.85±3.40 (– 5.0∼11.0)	1.554 (0.129)
	Control	8.67±3.29 (4.0∼17.0)	8.89±3.06 (3.0∼19.0)	– 0.21±2.77 (– 9.0∼5.4)	– 0.506 (0.616)
Depression	mSMI	10.79±3.15 (3.0∼17.0)	10.70±3.37 (0.0∼17.0)	0.09±3.46 (– 7.0∼11.0)	0.165 (0.870)
	Control	9.81±2.66 (3.0∼17.0)	9.93±2.70 (4.0∼18.0)	– 0.12±2.35 (– 5.8∼4.0)	– 0.330 (0.743)
UWES	mSMI	25.15±8.63 (9.0∼49.0)	26.54±10.14 (7.0∼54.0)	– 1.38±5.83 (– 15.0∼13.0)	– 1.479 (0.147)
	Control	24.26±8.23 (10.0∼47.0)	25.33±8.24 (8.0∼40.0)	– 1.07±5.85 (– 15.0∼13.0)	– 1.203 (0.236)

Note. ^*Scores at baseline– Scores at 6 weeks; mSMI = mobile app-based stress management intervention; SD = standard deviation; PSS = Perceived Stress Scales; UWES = Utrecht Work Engagement Scale.

Table 3 shows the between-group differences in the post-intervention scores. There were between-group differences in the age distribution. Moreover, since the baseline score affects the post-intervention score, we adjusted for age and baseline score as covariates. There was a significant between-group difference in the post-intervention perceived stress score (F = 4.051, p = 0.048); however, the effect size was small (0.23). There were no significant between-group differences in anxiety, depression, and work engagement scores.

Table 3

Differences in post-intervention between-group (mSMI = 39, control = 43)

Variables	Difference in means^*	ANCOVA^†	Adjusted R²
	(95% CI)	F(p)
PSS-10	– 1.03 (– 3.03 to 0.97)	4.051 (0.048)	0.176
Anxiety	– 0.30 (– 1.58 to 0.98)	1.960 (0.165)	0.272
Depression	0.77 (– 0.57 to 2.11)	0.064 (0.801)	0.251
UWES	1.21 (– 2.84 to 5.25)	0.082 (0.776)	0.601

Note. ^*Post-intervention scores in the experimental group– Post-intervention scores in the control group; ^†Controlling for baseline scores and age; mSMI = mobile app-based stress management intervention; 95% CI = 95% Confidence Interval; ANCOVA = Analysis of covariance; PSS = Perceived Stress Scales; UWES = Utrecht Work Engagement Scale.

Table 4 shows the users’ comments regarding their experiences and satisfaction with the mobile app. Most users (85.3%) were satisfied with the app. Two-thirds of the users commented that the mobile app increased stress management awareness. Furthermore, more than half of the users stated that the mobile app motivated them to change their attitude and behavior towards stress management. Other comments included requests for alert settings, stress measurement using a sensor, music repeat, and periodic content updates.

Table 4

The users’ comments on the mobile application (N = 34)

Items	Categories	N (%)	Other comments
Overall satisfaction with mobile application	Satisfied	29 (85.3)	•Alarm settings
	Dissatisfied	5 (14.7)	•Stress measurement using sensor
			•Music repeat
			•Periodic contents updates
Does this mobile application	Agree	23 (67.6)
increase your stress	No change	9 (26.5)
management awareness?	Disagree	2 (5.9)
Does this mobile application	Agree	18 (52.9)
improve your attitude toward	No change	11 (32.4)
stress management?	Disagree	5 (14.7)
Does this mobile application bring	Agree	20 (58.8)
behavioral change in	No change	9 (26.5)
stress management?	Disagree	5 (14.7)

4 Discussion

This study developed an mSMI for highly stressed workers and evaluated its effects. We found that mSMI effectively reduced perceived stress. This is consistent with previous findings that a mobile app-based stress management program (called Mind Healer) for nurses [18] and a mindfulness-based mHealth intervention for health care workers during the COVID-19 pandemic reduced perceived stress [16].

However, we found a small effect size of mSMI on perceived stress. A recent meta-review of seven meta-analyses on mobile app mental health interventions reported a small to medium effect size of mobile apps for stress management [27]. In contrast, a mobile application administered for 4 weeks to nurses had a high effect size on perceived stress [18]. Our observed small effect size could be attributed to an inadequate treatment dose. We instructed participants to use the mSMI for at least 10 minutes per session twice a week for 6 weeks. However, our logs showed that the participants accessed the app for an average of 10.4 minutes per week. Since we used a self-guided intervention, we delivered information regarding how-to instructions for each feature and personal information processing through the FAQ page and SMS. Additionally, we answered participants’ questions through a chat channel. Nevertheless, the participants did not adequately use the interventions. In the post-intervention satisfaction survey, participants mentioned that the periodic reminder text messages increased stress and they did not reinforce program engagement. Facilitating program engagement requires an approach that includes enhancing participants’ initial motivation, frequent and effective reminders, education on the importance of active participation, and financial and verbal incentives [28].

The method of this study which was the limitation of non-face-to-face interventions could have contributed to the failure to induce marked stress relief. A study [15] that administered a 6-week resilience training to health care workers with work-related stress reported that a smartphone-delivered intervention only improved well-being in contrast to an in-person intervention which reduced stress and emotional burnout, as well as promoted well-being. Therefore, we believe one or two face-to-face coaching sessions should be included to boost the effectiveness of mHealth interventions. If in-person interventions are not possible due to factors such as the recent COVID-19 pandemic, then video conferences (e.g., Zoom, Meet, Teams, and Webex) should be conducted at the participants’ convenience to emphasize the importance of self-management and to encourage steady participation.

In our study, mSMI was not effective for anxiety, depression, and work engagement. This is consistent with a previous report showing that a 4-week stress-management intervention for nurses using a mobile app did not significantly mitigate depression, anxiety, and emotional labor [18]. However, meta-analyses indicated [27, 29] app-supported smartphone interventions for mental health problems significantly improved depressive or anxiety symptoms with medium effect size. According to Wang et al. [11], a systematic review of study on mHealth interventions targeting stress, anxiety, depression, alcohol disorder, sleep disorder, suicidal behaviors, and PTSD in university, hospital, and community-based recruitment, 14 out of 16 apps significantly reduced mental health symptoms. However, the effects of mobile app interventions on workers, including those in this study, have not been clearly identified. Future studies on mHealth intervention among workers should examine changes in mental health symptoms or job-related outcomes. Anxiety and depression were about twice as high for manufacturing workers who work more than 60 hours a week [6]. Approximately two-thirds of workers who had to work during the COVID-19 pandemic showed more than moderate tendency to experience anxiety [30]. Therefore, further studies are needed to examine mental health symptoms with respect to work conditions, and to verify the efficacy of mobile health intervention.

In our study, participants were generally satisfied with the app; however, they suggested several improvements, including incorporating the latest IT such as automatic stress measurement using a sensor or voice control features. User acceptability and adherence rates are especially critical for mobile interventions [31]. It is likely that emotionally relaxed, warm, and appealing designs incorporating the latest technology to apps promoting mental health management would boost users’ adherence rates to maximize effective prevention and treatment of mental health conditions. There is a need for customized mobile apps tailored to specific target populations. In addition, if the effectiveness of digital health intervention using these customized mobile apps are verified in South Korea, the apps might be generalized to international users with different cultural backgrounds after the accessibility and satisfaction of purchases in the App store have improved [32].

Nevertheless, this study has several contributions to the current scenario. The intervention method was changed from the web-based approach used in previous studies to a mobile application for workers accustomed to using a smartphone. Moreover, we targeted a specific group of highly stressed workers who needed the close monitoring of their mental health in the workplace.

This study has some limitations. Future studies should develop mHealth interventions addressing the shortcomings of mobile-based interventions. First, the treatment dose was lower than planned. This could have compromised the internal validity. Workers who are interested in or are more familiar with using the apps for their mental health are more likely to have higher program engagement. The authors believe self-guided stress management might be beneficial to anyone, regardless of the participant’s stress severity. Second, we did not control for participants engaging in other health care promotions during this study or those who are engaged in other mHealth interventions. This might affect the findings of this study because the other programs might be more important than mSMI interventions, or the mSMI intervention might have been due to an effect of “another new one for mental health” rather than the specific mHealth aspect. Third, the study was conducted during the COVID-19 pandemic. External factors, including COVID-19-related depression, may have influenced the intervention outcomes. Finally, although we ensured allocation concealment, we could not ensure blinding of participants between users of mobile-app intervention and those without any intervention. It might have influenced the results to a certain extent.

5 Conclusions

This study is significant because we administered an individualized mHealth intervention for preventing mental health problems on highly stressed workers. We developed an mHealth intervention and evaluated its stress management effects on workers highly vulnerable to mental health problems. We found that a self-guided, non-face-to-face mSMI for 6 weeks reduced perceived stress in office workers; however, it did not improve anxiety, depression, and work engagement. We suggest that further studies are needed on mobile interventions applied to workers in other occupational groups and workers of different social status. After sufficient evidence has been accumulated, occupational health managers and professionals will be able to contribute to the prevention and management of perceived stress by implementing tailored mobile interventions for stressed workers.

Ethical approval

This study was approved by the Institutional Review Board of Daegu Catholic University (CUIRB-2020-0041).

Informed consent

All participants provided informed consent. No identifying information of participants was used in this study.

Conflict of interest

The author declares that there is no conflict of interest.

Footnotes

Acknowledgments

The author has no acknowledgments.

Funding

This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korean government (MSIT) (No. NRF-2018R1C1B5042936).

References

Lee

, Kim

. A Systematic Review of Workplace Interventions forthe Mental Health of Workers. J Korean Acad Psychiatr Ment Health Nurs. 2020;29(2):155–67.

Omeogu

, Shofer

, Gehrman

, Green-McKenzie

. Efficacy of a Mobile Behavioral Intervention for Workers with Insomnia. J Occup Environ Med. 2020;62(3):246–50.

Verkuil

, Atasayi

, Molendijk

. Workplace Bullying and Mental Health: A Meta-Analysis on Cross-Sectional and Longitudinal Data. PLoS One. 2015;10(8):e0135225.

Tarquinio

, Rotonda

, Houllé

, Montel

, Rydberg

, Minary

, Dellucci

, Tarquinio

, Fayard

, Alla

. Early psychological preventive intervention for workplace violence: A randomized controlled explorative and comparative study between EMDR-recentevent and critical incident stress debriefing. Issues Ment Health Nurs. 2016;37(11):787–99.

Difede

, Malta

, Best

, Henn-Haase

, Metzler

, Bryant

, Marmar

. A randomized controlled clinical treatment trial for World Trade Center attack-related PTSD in disaster workers. J Nerv Ment Dis. 2007;195(10):861–5.

Kang

, Park

, Jang

, Lim

, Ann

, Cho

, Moon

. Comparison of anxiety and depression status between office and manufacturing job employees in a large manufacturing company: across sectional study. Ann Occup Environ Med. 2016;28, 47.

Kim

, Park

, Kwon

. Work Engagement in South Korea: Validationof the Korean Version 9-Item Utrecht Work Engagement Scale. Psychol Re. 2017;120(3):561–78.

Padula

, Chiavegato

, Cabral

CMN

, Almeid

, Ortiz

, Carregaro

. Is occupational stress associated with work engagement? Work 2012;41:2963–5.

Carolan

, Harris

, Cavanagh

. Improving Employee Well-Being and Effectiveness: Systematic Review and Meta-Analysis of Web-Based Psychological Interventions Delivered in the Workplace. J Med Internet Res. 2017;19(7):e271.

10.

Boulos

, Brewer

, Karimkhani

, Buller

, Dellavalle

. Mobile medical and health apps: state of the art, concerns, regulatory control and certification. Online J Public Health Inform. 2014;5(3):229.

11.

Wang

, Varma

, Prosperi

. A systematic review of the effectiveness of mobile apps for monitoring and management of mental health symptoms or disorders. J Psychiatr Res. 2018;107:73–8.

12.

Matcham

, Rayner

, Hutton

, Monk

, Steel

, Hotopf

. Self-help interventions for symptoms of depression, anxiety and psychological distress in patients with physical illnesses: a systematic review and meta-analysis. Clin Psychol Rev. 2014;34(2):141–57.

13.

Ebert

, Heber

, Berking

, Riper

, Cuijpers

, Funk

, Lehr

. Self-guided internet-based and mobile-based stress management for employees: Results of a randomised controlled trial. Occup EnvironMed. 2016;73(5):315–23.

14.

Persson Asplund

, Dagöö

, Fjellström

, Niemi

, Hansson

, Zeraati

, Ziuzina

, Geraedts

, Ljótsson

, Carlbring

et al. Internet-based stress management for distressed managers: Results from a randomised controlled trial. Occup Enviro Med. 2018;75(2):105–13.

15.

Mistretta

, Davis

, Temkit

, Lorenz

, Darby

, Stonnington

. Resilience Training for Work-Related Stress Among Health Care Workers: Results of a Randomized Clinical Trial Comparing In-Personand Smartphone-Delivered Interventions. J Occup Environ Med. 2018;60(6):559–68.

16.

Fiol-DeRoque

, Serrano-Ripoll

, Jiménez

, Zamanillo-Campos

, Yáñez-Juan

, Bennasar-Veny

, Leiva

, Gervilla

, García-Buades

, García-Toro

et al. A Mobile Phone-Based Intervention to Reduce Mental Health Problems in Health Care Workers During the COVID-19 Pandemic (PsyCovidApp): Randomized Controlled Trial. JMIR Mhealth Uhealth. 2021;9(5):e27039.

17.

Faul

, Erdfelder

, Buchner

, Lang

A-G

. Statistical power analysesusing G*Power 3.1: Tests for correlation and regression analyses. Behav Res Methods. 2009;41:1149–60.

18.

Hwang

, Jo

. Evaluation of the Effectiveness of Mobile App-Based Stress-Management Program: A Randomized Controlled Trial. Int J Environ Res Public Health. 2019;16(21).

19.

Huberty

, Green

, Glissmann

, Larkey

, Puzia

, Lee

. Efficacy of the Mindfulness Meditation Mobile App “Calm” to Reduce Stress Among College Students: Randomized Controlled Trial. JMIR MhealthUhealth. 2019;7(6):e14273.

20.

Scott

, Webb

, Rowse

. Self-help interventions for psychosis: a meta-analysis. Clin Psychol Rev. 2015;39:96–112.

21.

Mayo-Wilson

, Montgomery

. Media-delivered cognitive behavioural therapy and behavioural therapy (self-help) for anxiety disorders in adults. Cochrane Database Syst Rev. 2013(9):Cd005330.

22.

Stoyanov

, Hides

, Kavanagh

, Zelenko

, Tjondronegoro

, Mani

. Mobile app rating scale: a new tool for assessing the quality of health mobile apps. JMIR Mhealth Uhealth. 2015;3(1):e27.

23.

Rathbone

, Prescott

. The Use of Mobile Apps and SMS Messaging as Physical and Mental Health Interventions: Systematic Review. J Med Internet Res. 2017;19(8):e295.

24.

Cohen

. Perceived stress in a probability sample of the United States. In: The social psychology of health. Thousand Oaks, CA, USA:Sage Publications, Inc. 1988. pp. 31–67.

25.

Lee

, Shin

, Ko

Y-H

, Lim

, Joe

S-H

, Kim

, Jung

I-K

, Han

. The Reliability and Validity Studies of the Korean Version of the Perceived Stress Scale. Korean J Psychosom Med. 2012;20(2):127–34.

26.

Schaufeli

, Bakker

, Salanova

. The Measurement of Work Engagement With a Short Questionnaire:A Cross-National Study. Educ Psychol Meas. 2006;66(4):701–16.

27.

Lecomte

, Potvin

, Corbière

, Guay

, Samson

, Cloutier

, Francoeur

, Pennou

, Khazaal

. Mobile Apps for Mental Health Issues: Meta-Review of Meta-Analyses. JMIR Mhealth Uhealth. 2020;8(5):e17458.

28.

Zeng

, Guo

, Li

, Hong

, Li

, Zhu

, Zeng

, Zhang

, Cai

, Liu

, et al. Relationship Between Patient Engagement and Depressive Symptoms Among People Living with HIV in a Mobile Health Intervention: Secondary Analysis of a Randomized Controlled Trial. JMIR Mhealth Uhealth. 2020;8(10):e20847.

29.

Linardon

, Cuijpers

, Carlbring

, Messer

, Fuller-Tyszkiewicz

. The efficacy of app-supported smartphone interventions for mental health problems: a meta-analysis of randomized controlled trials. World Psychiatry. 2019;18(3):325–36.

30.

Çelikkalp

, Irmak

, Ekuklu

. Working conditions and anxiety levels of employees who have to work during the COVID-19 pandemic. Work. 2021;70, 1047–55.

31.

Rismawan

, Marchira

, Rahmat

. Usability, Acceptability, and Adherence Rates of Mobile Application Interventions for Preventionor Treatment of Depression: A Systematic Review. J Psychosoc Nurs Ment Health Serv. 2021;59(2):41–7.

32.

Prata

, de Moraes

, Quaresma

. User’s demography and expectation regarding search, purchase and evaluation in mobile application store. Work. 2012;41, 1124–31.