A Smartphone App for Promoting Mental Well-being and Awareness of Anxious Symptoms in Adolescents: A Pilot Cluster Randomized Controlled Trial

Abstract

Objective:

To design a smartphone app and evaluate its effect on promoting mental well-being and awareness of anxious symptoms in adolescents.

Materials and Methods:

A pilot cluster randomized controlled trial was conducted with two secondary schools (390 students, mean age 13.1 years) randomized to the intervention and control groups. An app was designed to encourage “Sharing, Mind, and Enjoyment (SME)” between students and their parents (e.g., express gratitude), including interaction and game elements. The app was used daily over 1 month. The waitlist control group was offered the app after completing all assessments. The primary outcomes were changes in SME behaviors measured at 1 month and 3 months. Secondary outcomes included subjective happiness, well-being, personal health and happiness, family health, happiness and harmony, self-perceived knowledge, and understanding of anxious symptoms. Two focus groups of students and three individual in-depth interviews of community partners were conducted.

Results:

Seventeen of 152 students (11.2%) in the intervention group used the app together with parents (adherent subgroup) and 69 (45.4%) used it without involving parents. The intervention group did not show significant difference in the change of SME behaviors at 1- or 3-month follow-up compared with the control group. However, the intervention group reported greater increase in the awareness of anxious symptoms at follow-ups than the control group (d = 0.52 at 1 month and d = 0.43 at 3 month, both P < 0.001). Post hoc analysis showed a significantly greater increase in SME-related behaviors in the adherent subgroup than the control group at 3 months (d = 0.46, P = 0.04). The interviews found favorable changes in app users, but motivation to use the app was low in general. Both students and community partners suggested primary school students would be more receptive users.

Conclusions:

The app did not show effectiveness in increasing SME behaviors of students, but increased awareness of anxious symptoms. Further improvements and tests are warranted.

Trial Registration: ClinicalTrials.gov NCT03361475

Introduction

Mental disorders affect 10%–20% of children and adolescents worldwide and constitute a substantial health burden.¹ Conventional problem and treatment-oriented programs have little impact at the population level,² while brief preventive interventions could be adopted widely and sustainably, despite smaller effect sizes.³ The public mental health approach is increasingly accepted for improving population mental well-being.^4,5 Prevention and early intervention programs may prevent later adult mental health problems and improve personal well-being and productivity.¹

Digital technology devices can be adopted as attractive and convenient tools for health promotion in children and adolescents as they use them frequently.⁶ Increasing number of studies have evaluated the effectiveness of apps for children and adolescents' mental health,^7–9 focusing on management of mental health symptoms or disorders. However, only a few app-based interventional studies among children were found on the clinical trial registry, using keywords of “mental well-being/mental wellness and app.” These studies either focused on monitoring mood, or the treatment of psychosis, depression, anxiety, or other types of mental disorders.

It is estimated that around 7% of Hong Kong children and adolescents have some form of anxiety disorders.^10,11 In January 2016, the Centre for Health Protection of the Hong Kong Government launched the territory-wide mental health promotion campaign, “Joyful@HK,” to promote mental well-being under three main themes: Sharing, Mind, and Enjoyment (SME).¹² Sharing connects family and friends, and supports those in need. Mind entails keeping an open mind and being positive and optimistic. Enjoyment is about engaging in enjoyable activities to maximize one's potential and achieve satisfaction. These themes are underpinned by the theories of positive psychology,^13,14 of which interventions are effective in enhancing well-being and reducing symptoms of mental disorders.^15–18

The SME App Pilot Project was one of the community-based engagement projects under Joyful@HK to promote mental well-being of adolescents. The SME app was designed as a family-based program, as family plays an important role in adolescent mental well-being and family harmony is highly valued in Chinese culture.¹⁹ App-based interventions among Hong Kong adolescents are feasible, given 94.0% of Hong Kong adolescents 10–24 years of age have smartphones.²⁰

The objective of this pilot trial was to design a smartphone app and use a cluster randomized controlled trial to evaluate its effectiveness in promoting SME-related behaviors and mental well-being of adolescents. We adopted a public health approach to deliver a brief preventive intervention rather than an intensive remedial intervention. We hypothesized more favorable changes in SME behaviors, happiness, well-being, family well-being, and awareness of anxious symptoms in the intervention group than the control group.

Materials and Methods

Participants

The trial was conducted in 2017–2018 by the School of Public Health, University of Hong Kong, in collaboration with (1) the School Social Work Service and Student Guidance Service, Tung Wah Group of Hospitals, and (2) the Technology-Enriched Learning Initiative, University of Hong Kong. Secondary 1 and 3 (Grade 7 and 9) students (mean age 13.1 years, SD 1.2) were recruited from two co-educational schools with medium academic performance. The presence of pre-existing anxiety disorders in students was not assessed as our objectives were to improve mental well-being in general and the awareness of anxious symptoms with a simple public health intervention that can benefit many.

The Institutional Review Board of the University of Hong Kong/Hospital Authority Hong Kong West Cluster granted ethical approval (reference no.: UW 17-293). An invitation letter was sent to parents through the schools to obtain consent from parents. Written informed consent was also sought from students.

Sample size

For pilot trials, a formal calculation of sample size might not be appropriate. A minimum of 30–35 participants were recommended in each arm^21,22 and 75 participants were needed for an extra small standardized effect size (≤0.1).²³ To generate preliminary estimates of the intervention effectiveness, we expected a total sample size of 300 (150 for each arm).

Randomization

Two schools were randomized into the intervention or waitlist control group using two sealed opaque envelopes, each containing a group allocation card. The process was conducted by a person who was not involved in the study. The workshop and app were also offered to the waitlist control students after they had completed all assessments. Because the intervention was obvious to both students and researchers, blinding was not feasible.

Interventions: SME Smartphone App

The SME App (Chinese name: e家·樂分享) was designed to be stylish, attractive, and practical for family of junior secondary school students with features that promote SME and raise awareness of adolescent mental health problems such as anxious symptoms (Supplementary Appendix SA1). The app was tested and fine-tuned after conducting three focus group discussions with 25 students who were from two other schools (8–9 students per group). They were not involved in the trial. The core features were the SME “tasks” in which adolescents and their parents interact to complete and earn digital tokens daily.

The SME tasks included the following: spend time with family, express gratitude to family in words or actions, share personal preferences with family, share happy experiences with family, communicate with family, express care and support to family, do outdoor or physical activities with family, enjoy doing tasks together with family, and learn new things together with family. The SME tasks were designed based on the SME concept, and further improved by the school social workers to ensure the tasks were feasible for students and parents.

The students could choose a location on a virtual community map for their “dream house” and use the earned digital tokens to buy furniture and decorations for the house (Figs. 1 and 2). The top 10 token owners were shown on a ranking list visible to all users. A feature to attract use of the app was a mini digital soccer game with the top 10 users also displayed. The game can be played by each family user for 10 rounds per day at most if the users did not complete any task. One completed task will unlock five more rounds. For each round of the game, the users tried to catch the footballs coming from different directions (Supplementary Appendix SA1). The users could also earn tokens through reading materials and watching videos on anxious symptoms.

FIG. 1.

Virtual community maps.

FIG. 2.

Dream house and decorations.

An introductory workshop was conducted for the students by an experienced social worker covering concepts of SME, SME in daily life, features of the SME App, anxious symptoms, and how to seek help for emotional disturbances. The students were guided to download and explore features of the app immediately. Each student was assigned one login ID and password. Each day over 1 month, the students could do up to 10 routine tasks (10 digital tokens each, e.g., “show gratitude to your family,” “prepare a beverage for your family”) and 1 daily task (30 tokens, e.g., “send mom a message,” “tell a joke to family”).

A total of 10 special tasks (150 tokens each, e.g., “go to the supermarket with your family and buy their favorite food”) were also provided randomly over 1 month. The parents also received prompts to do 10 routine tasks (20 tokens each, e.g., “share a childhood story with your children”) per day. After the students or parents had finished their tasks, an automatic reminder was sent to their counterparts for verification, only then would the tokens be granted.

To check whether the introductory workshop had followed the protocol, six people (four research officers from the Centre for Health Protection and two research assistants from our research team) completed a fidelity checklist to evaluate the adherence independently.

Outcome measurements

We measured all the outcomes at baseline (T1), 1 month (T3, after using the app), and 3 months (T4). Immediately after the workshop (T2), we measured the self-perceived knowledge and understanding of anxious symptoms. The students completed the anonymous questionnaires in paper format.

Primary outcomes

The primary outcomes were nine SME behaviors, each measuring the number of days in the past 7 days (range 0–7) when the behavior was practiced. These behaviors were divided into three subsets of SMEa, SMEb, and SMEc, each with three behaviors. SMEa (“spend time with family,” “express gratitude to family in words or actions,” and “do outdoor or physical activities with family,” range 0–21, congeneric reliability 0.65) was repeatedly measured at T1, T3, and T4. To avoid practice/learning effect through repeated answering, SMEb and SMEc (range 0–21, congeneric reliability 0.82 and 0.78, respectively) were only measured at T3 and T4, respectively. SMEb included “share personal preferences with family,” “share happy experiences with family,” and “enjoy doing tasks (e.g., cooking or cleaning) together with family.” SMEc included “communicate with family,” “express care and support to family,” and “learn new things together with family.”

Secondary outcomes

Subjective happiness

The 4-item Subjective Happiness Scale was used to assess an individual's overall happiness using a 7-point Likert scale.²⁴ Higher scores indicate higher levels of happiness. The reliability and validity of the Chinese version were established in the general population.²⁵

Well-being

The 7-item Short Warwick-Edinburgh Mental Well-being Scale used a 5-point Likert scale. The raw score was calculated by summing all seven items.²⁶ Higher scores indicate a higher level of well-being. The Chinese version indicated good validity and reliability.²⁷

Personal health and happiness

Personal health and happiness were measured by asking the respondents “how healthy/happy do you think you are.” Respondents rated each item from 0 (not at all healthy/happy) to 10 (very healthy/happy), with higher scores indicating more healthy or happy.^28,29

Family health, happiness, and harmony

Family health, happiness, and harmony were measured by asking the respondents “how harmonious/happy/healthy do you think your family is.”^29,30 Respondents rated each item from 0 (not at all harmonious/happy/healthy) to 10 (very harmonious/happy/healthy), with higher scores indicating better family well-being.

Self-perceived knowledge and understanding of anxious symptoms

This self-developed measurement was to assess the awareness of anxious symptoms (“I know the anxious symptoms,” range 0–10), intention to seek help (“I will talk to family or friends when I have emotional disturbance,” range 0–10), self-efficacy of help seeking (“I know how to seek help when I have emotional disturbance,” range 0–10), and understanding of anxious symptoms (“Not being able to stop worrying” is one of the anxious symptoms, right/wrong) (T1, T2, T3, and T4). Another three items of understanding of anxious symptoms (“hard to focus on study or other activities/headache/hand shaking is one of the anxious symptoms,” right/wrong) were measured once to avoid learning effect, respectively, at T2, T3, and T4.

Intention to change and subjective changes

Intention to change (“spend time with family,” “express appreciation to family in words or action,” and “have outdoor activities or do exercise with family”) was measured at T2. Subjective changes of the above three aspects were measured at T3 and T4. The questions were rated on a 5-point Likert scale, with higher scores indicating stronger intention or greater increases.

Focus groups and in-depth interviews

The students who joined the workshop or used the app were invited to join focus group discussions, and individual community partners were invited to join in-depth interviews. These were to explore their perspectives on satisfaction with the contents, subjective changes, difficulties in adherence, and suggestions for improvement.

Statistical analysis

Quantitative data were analyzed using Stata 13.0. Chi-square tests and independent t-tests were conducted to compare the demographic characteristics. We compared primary and secondary outcomes between groups using multilevel models adjusted for baseline values of the outcomes, and demographic variables, with cluster included as a random effect. Maximum likelihood multilevel modeling was built by xtmixed command in Stata.³¹ Intention-to-treat (ITT) analysis, including all randomized subjects, was conducted with missing observations from loss to follow-up imputed. Multiple imputation was used to replace each missing value with five complete datasets.³² The effect estimates were combined from the imputed datasets using Rubin's rule.³³

A post hoc subgroup analysis (unspecified in the protocol) was conducted to compare the adherent subgroup (students who used the app with their parents) with the whole control group. The characteristics of the subgroup and control group were compared, and the imbalanced factors were adjusted for in the analyses of primary and secondary outcomes. An effect size (Cohen's d) of 0.2 is considered a small effect, 0.5 a medium effect, and 0.8 a large effect. All tests were two sided with P < 0.05 indicating statistical significance. Qualitative data were analyzed using thematic content analysis.³⁴ All the interviews were transcribed verbatim and coded by a researcher (Sasha Deng) thoroughly and double checked by another one (Yuying Sun). Then the data were collated into potential themes. The themes were checked and refined until clear definitions and names were created for each theme. Some compelling contents were extracted and reported as examples.³⁴

Results

Two schools were included with one randomized into the intervention group (152 students) and the other into the control group (238 students) from September to November 2017. Figure 3 shows the CONSORT flow chart. The retention rates of all subjects were 94.6% at 1 month and 96.9% at 3 months. The adherence rate of the introductory workshop was 98.3% (SD = 4.1). The adherence to the core messages was 97.5% (SD = 4.2). Table 1 shows 53.8% of students were boys. The intervention students were significantly younger (P < 0.001). Age, grade, whether the students had ever joined other SME-related programs, and baseline of the outcomes were included as covariates in subsequent analyses. Seventeen (11.2%) intervention students used the app with parents (adherent subgroup), 69 (45.4%) used the app without involving parents, and 66 (43.4%) attended the workshop, but did not use the app.

FIG. 3.

CONSORT flow chart.

Table 1.

Baseline Demographic Characteristics in Intervention and Control Arms

Demographics	Categories	Intervention (n = 152) n (%)	Control (n = 238) n (%)	P
Sex	Male	85 (55.9)	125 (52.5)	0.51
Sex	Female	67 (44.1)	113 (47.5)
Age, mean (SD)		12.8 (1.09)	13.3 (1.24)	<0.001
Grade	Form 1	119 (78.3)	124 (52.1)	<0.001
Grade	Form 3	33 (21.7)	114 (47.9)
Birthplace	Hong Kong	124 (81.6)	204 (85.7)	0.53
	Guangdong Province	18 (11.8)	23 (9.7)
	Other places	10 (6.6)	11 (4.6)
Duration of stay in Hong Kong	<7 years	18 (11.8)	18 (7.7)	0.17
Duration of stay in Hong Kong	≥7 years	134 (88.2)	216 (92.3)
Family financial status	Poor	8 (5.3)	7 (3.0)	0.15
	Below average	16 (10.7)	42 (17.9)
	Average	104 (69.3)	157 (67.1)
	Above average	19 (12.7)	27 (11.5)
	Rich	3 (2.0)	1 (0.4)
Parents' marital status	Married	119 (79.3)	198 (83.9)	0.37
	Separated or divorced	25 (16.7)	30 (12.7)
	Any of the parents died	6 (4.0)	8 (3.3)
Living with parents	Both parents	118 (77.6)	191 (80.3)	0.23
	Either parent	29 (19.1)	33 (13.9)
	Neither parents	5 (3.3)	14 (5.9)
Has ever joined SME-related programs	Yes	3 (2.0)	0 (0.0)	0.04

SME, sharing, mind, and enjoyment.

Primary outcomes

Table 2 shows no difference in changes in SME behaviors between the intervention and control groups at 1 month or 3 months. However, the increase in SMEa was significantly greater in the adherent subgroup than the control group at 3 months (BMD = 2.30, 95% CI: 0.16–4.44, d = 0.46, P = 0.04) (Supplementary Appendix SA2). As for the one-time measurements, SMEb and SMEc, no difference was observed between the adherent and control groups (Supplementary Appendix SA2). No harmful effect was observed in students after the intervention. We did not receive any report of unintended effects or privacy breaches.

Table 2.

Effects of Intervention on Sharing, Mind, and Enjoyment Behaviors, Well-being, and Awareness of Anxious Symptoms

Items	Time point	Mean/%		BMD/OR^a (95% CI)	Effect size	P
Items	Time point	Intervention (n = 152)	Control (n = 238)	Intervention versus control	Intervention versus control	P
SMEa	T1	9.52	9.03
	T3	8.79	9.13	−0.53 (−1.33 to 0.26)	−0.10	0.19
	T4	10.03	9.00	0.77 (−0.16 to 1.69)	0.15	0.10
SMEb	T3	8.45	8.18	−0.08 (−0.99 to 1.16)	−0.01	0.88
SMEc	T4	9.74	9.27	0.16 (−0.85 to 1.17)	0.03	0.76
Subjective happiness	T1	4.69	4.57
	T3	4.56	4.49	−0.05 (−0.21 to 0.10)	−0.05	0.50
	T4	4.38	4.45	−0.17 (−0.35 to 0.02)	−0.17	0.07
Well-being	T1	23.34	23.14
	T3	22.85	23.35	−0.55 (−1.25 to 0.15)	−0.12	0.12
	T4	23.20	22.72	0.58 (−0.19 to 1.36)	0.13	0.14
Personal health	T1	7.25	6.29
	T3	6.80	6.24	−0.03 (−0.44 to 0.38)	−0.01	0.90
	T4	6.67	6.37	−0.16 (−0.58 to 0.25)	−0.07	0.45
Personal happiness	T1	6.98	6.55
	T3	6.45	6.28	−0.15 (−0.49 to 0.19)	−0.06	0.38
	T4	6.56	6.39	−0.14 (−0.52 to 0.24)	−0.07	0.47
Family health	T1	7.38	6.71
	T3	6.78	6.69	−0.30 (−0.70 to 0.09)	−0.13	0.13
	T4	6.59	6.81	−0.54 (−0.95 to −0.14)	−0.24	0.01
Family happiness	T1	7.08	6.81
	T3	6.69	6.59	−0.09 (−0.48 to 0.31)	−0.04	0.67
	T4	6.54	6.61	−0.22 (−0.61 to 0.17)	−0.10	0.28
Family harmony	T1	7.05	6.95
	T3	6.70	6.71	−0.08 (−0.49 to 0.33)	−0.03	0.70
	T4	6.46	6.73	−0.34 (−0.73 to 0.06)	−0.15	0.10
The awareness of anxious symptoms	T1	4.26	4.16
	T2	6.26	—	2.01 (1.75 to 2.26)	0.63	<0.001
	T3	5.65	4.29	1.50 (0.91 to 2.09)	0.52	<0.001
	T4	5.55	4.43	1.25 (0.67 to 1.84)	0.43	<0.001
Intention to seek help	T1	5.92	5.39
	T2	6.41	—	0.49 (0.29 to 0.68)	0.17	<0.001
	T3	6.26	5.47	0.58 (0.10 to 1.07)	0.21	0.02
	T4	6.06	5.60	0.42 (−0.13 to 0.97)	0.16	0.13
Self-efficacy of help seeking	T1	6.31	6.12
	T2	6.80	—	0.49 (0.29 to 0.65)	0.18	<0.001
	T3	6.48	6.10	0.31 (−0.19 to 0.81)	0.12	0.22
	T4	6.36	6.19	0.09 (−0.39 to 0.57)	0.03	0.71
Understanding of anxious symptoms (worry)^a	T1	46.0%	50.0%
	T2	66.2%	—	4.77 (2.07 to 11.48)	—	<0.001
	T3	67.1%	65.6%	1.18 (0.72 to 1.95)	—	0.51
	T4	59.2%	63.3%	1.01 (0.62 to 1.65)	—	0.96
Understanding of anxious symptoms (attention)^a	T2	71.5%	—	2.70 (1.18 to 6.41)	—	0.01
Understanding of anxious symptoms (headache)^a	T3	34.9%	21.3%	2.52 (1.51 to 4.20)	—	<0.001
Understanding of anxious symptoms (hand shaking)^a	T4	34.5%	24.9%	1.87 (1.13 to 3.08)	—	0.01

SMEa: “spend time with family,” “express gratitude to family in words or actions,” and “do outdoor or physical activities with family”; SMEb: “share personal preferences with family,” “share happy experiences with family,” and “enjoy doing tasks together with family”; SMEc: “communicate with family,” “express care and support to family,” and “learn new things together with family,” all ranged from 0 to 21.

The other unspecified variables were compared using between-group mean difference.

BMD, between-group mean difference; OR, odds ratio; T1, baseline; T2, after the workshop; T3, 1-month follow-up; T4, 3-month follow-up.

Secondary outcomes

Table 2 shows that, in the intervention group, the increases in the awareness of anxious symptoms at 1 month (d = 0.52, P < 0.001) and 3 months (d = 0.43, P < 0.001), and the intention to seek help at 1 month (d = 0.21, P = 0.02) were significantly greater than the control group. An unexpected greater decrease in family health in the intervention group than the control group was observed at 3 months (d = −0.24, P = 0.01). The intervention group had greater improvement in self-perceived understanding of anxious symptoms (headache) than the control group at 1 month (odds ratio [OR] = 2.52, 95% CI 1.51–4.20, P < 0.001) and 3 months (hand shaking, OR = 1.87, 95% CI 1.13–3.08, P = 0.01).

The other outcomes showed no significant difference between groups at 1 month or 3 months. Supplementary Appendix SA2 shows significantly greater positive changes in the adherent subgroup than the control group in the awareness of anxious symptoms (1-month d = 1.30, P < 0.001; 3-month d = 0.77, P = 0.002), intention to seek help (1-month d = 0.81, P < 0.001; 3-month d = 0.89, P = 0.001), self-efficacy of help seeking (1-month d = 0.70, P = 0.003), and family health (1-month d = 0.43, P = 0.03).

Intention to change and subjective changes

After the workshop, the proportion of students with an intention (including strong intention) to “spend time with family,” “express appreciation to family in words or action,” and “do outdoor or physical activities with family” was 44.7%, 42.4%, and 46.4%, respectively. At 1 month, 22.7%, 20.6%, and 17.1% students reported “a little” or “much more” positive change of the above three aspects, respectively. Subjective improvement in health and happiness was reported by 29.3% and 26.0% of intervention students at 1 month. At 3 months, the self-reported changes in SME-related behaviors, health and happiness were similar to those at 1 month.

Focus groups and in-depth interviews

Two focus group discussions were conducted, involving 10 boys and 8 girls. Some students tried to do tasks but reported difficulties in doing so every day or asking parents to verify immediately. Some students were most impressed about the house decoration part. Some liked the mini game as a good leisure time activity. Some found the learning page useful and educational.

“I like this app as it is novel and can link to our daily life.” (Girl, 15 years old)

“I tried, but sometimes I can't do it.” (Boy, 12 years old)

“I'm impressed by the house decorations, which allow me to create things.” (Boy, 12 years old)

Some parents used the app together with their children, but not very frequently as they were busy. A few students reported more communication with their family after using the app. Some students reported no change in family relationships that were already good.

“My Mom only used it several times. She was very busy.” (Boy, 12 years old)

“We used to watch our own smartphone during dinner and rarely talked. Now my parents will share stories during dinner.” (Girl, 12 years old)

Some students indicated that they felt awkward to do the tasks with parents, for example, hugging family members without any specific reason. They thought those tasks would be more suitable for primary school students. Some students suggested we include functions such as sending reminders to do the tasks and creating a more sophisticated map. Some students wished the app could be more innovative and include more mini games or personalized characters.

“I think the app fits better for primary school students. And they will have more time to play.” (Girl, 14 years old)

“The map can be larger, three-dimensional and more vivid.” (Boy, 12 years old)

Three in-depth interviews were conducted, involving one social worker, one school principal, and one app developer. All interviewees found the project objective to promote positive parent-child relationship important. Although the app involved multiple components and functions, the game part was simple and less attractive compared with popular games. A major difficulty was the lack of parents' participation as many found the app too complicated and hence not willing to spend time on it. The interviewees preferred a simpler app with the task function only. Older students were less active in using the app as they had less motivation to communicate with their parents. A workshop for parents was suggested to encourage their participation.

“It is very meaningful to use a smartphone app to promote family relationships, which is one of our school's focus.” (Male, school principal)

“The app is too complicated, which might have affected the retention rate. It is difficult to incorporate games and education together in one app. We can adopt a simpler design with only one function.” (Male, app developer)

“The secondary students are in the period of rebellion and becoming more independent. They are reluctant to communicate with parents.” (Female, social worker)

Discussion

The quantitative results did not show the effectiveness of the intervention in improving students' SME behaviors but showed moderate effectiveness in improving the awareness of anxious symptoms (d: 0.43–0.63) and other mental health outcomes. However, the increase in SME-related behaviors was significantly greater in the adherent subgroup than the control group at 3 months (d: 0.46). Our effect sizes are comparable with those of previous public health interventions (d: 0.25, 27 studies) for children's social, emotional, and behavioral problems.³⁵ To our knowledge, this was the first study to comprehensively evaluate the effectiveness of a smartphone app involving both adolescents and their parents. The strengths include using both quantitative and qualitative evaluations, the use of both ITT and per-protocol analyses, and high fidelity of the introductory workshop. However, full adherence of both students and parents was only 11%.

Significant increases in SME behaviors were observed only in the adherent subgroup at 3 months. This was consistent with the self-reported changes, which showed that 20%–30% students perceived “a little” or “much more” SME behaviors. The nonsignificant results might have been due to the low rate of app usage. Adherence is essential for any efficacious intervention to demonstrate an effect. More motivated participants are also more likely to be adherent. In a trial setting, it would be difficult to separate the effects of motivation adherence from the intervention itself. Some factors might have affected the participants' adherence, such as motivation and interests.

The unavailability of parents was suggested as one of the most important factors. Many parents work full time and cannot take part in parent-child activities. Both the students in the focus groups and the community partners suggested we use this app among younger children. Parents may spend more time on younger children. Some even quit their jobs when the children are young, but have to return to work full time when they get older. A meta-analysis on 177 primary prevention programs found that affective education and problem-solving programs were most successful with the youngest children (d: 0.70 and 0.93, respectively), but showed smaller effect on older students 11 years of age or older (d: 0.36).³⁶

The first stage of adolescent development and establishment of autonomy begins during early adolescence (12–14 years of age), which is characterized by forming same-sex peer groups, with decreasing interest in family activities and parental advice.³⁷ Secondary school students are in the transient period from children to adults. They tend to be more independent and care more about their peers than their parents. To promote the students' engagement, their motivation and collaboration with others are important, such as creating challenging educational experiences for students, enriching and extending their academic abilities, and fostering collaborative learning.³⁸

Practice effects are common for tests or tasks involving repeated measures as learning and memory may lead to enhanced performance.^39–41 In our trial, some questions were only measured once (without baseline data) to avoid the practice/learning effect. For the repeatedly measured outcome at all time points, both intervention and control group had improvements at 1 month and 3 months. While improvements in the intervention group may be largely due to the intervention, any improvement in the control group on account of learning from previous identical assessment would attenuate the effect size. Indeed, for the one-time measurement, the intervention group showed better understanding of anxious symptoms than the control group. Using different questions in each assessment may reduce the impact of learning effect. This method provided an innovative approach to assess knowledge-related outcomes that can be affected by learning and memory.

Limitations

Our study had several limitations. First, this pilot trial only involved two schools. Due to the small sample size, the statistical comparison between arms might lack power. The maximum likelihood multilevel modeling has been used, while restricted maximum likelihood estimation is recommended when the cluster size is small.³¹ However, we had only one cluster in each group and the cluster might not affect the effect estimates. Second, adherence was low, but we found more satisfactory results in the adherent subgroup. The statistical difference needs cautious interpretation, given the selection bias and low statistical power.⁴² Third, blinding was not feasible, which was common for psychosocial or behavioral interventions. Finally, because validated questionnaires were not available and many existing questionnaires were too long, we developed our own outcome and impact-oriented questionnaires to assess the changes in the participants. Nevertheless, we found moderate to good congeneric reliability for those items.

Conclusion

Our pilot trial is the first study to explore a family-based smartphone app for promoting mental well-being and raising awareness of anxious symptoms among a general sample of adolescents. The app did not show effectiveness in increasing SME behaviors of students but was effective in increasing awareness of anxious symptoms. Given the parents' hesitation toward complicated games, a more concise design may enhance adherence. Future studies may evaluate a simplified version of the smartphone app, preferably among younger children and their parents, who are suggested to be more receptive users by community partners.

Footnotes

Acknowledgments

We sincerely thank the participants, the social workers in the Tung Wah Group of Hospitals for conducting the interventions, and research assistants for data collection. We also thank Ms. Sasha Deng for her help with the qualitative data coding.

Author Disclosure Statement

No competing financial interests exist.

Funding Information

This study was funded by Health and Medical Research Fund Health Care and Promotion Scheme (Project No. CPP-HKU).

Supplementary Material

References

Kieling

, Baker-Henningham

, Belfer

, et al. Child and adolescent mental health worldwide: Evidence for action. Lancet, 2011; 378:1515–1525.

Prinz

, Sanders

. Adopting a population-level approach to parenting and family support interventions. Clin Psychol Rev, 2007; 27:739–749.

Fabrizio

, Lam

, Hirschmann

, et al. A brief parenting intervention to enhance the parent-child relationship in Hong Kong: Harmony@Home. J Child Fam Stud, 2013; 22:603–613.

Wahlbeck

. Public mental health: The time is ripe for translation of evidence into practice. World Psychiatry, 2015; 14:36–42.

Forsman

, Wahlbeck

, Aaro

, et al. Research priorities for public mental health in Europe: Recommendations of the ROAMER project. Eur J Public Health, 2015; 25:249–254.

CST

, Fowler

, Lam

WYY

, et al. Parenting approaches and digital technology use of preschool age children in a Chinese community. Ital J Pediatr, 2014; 40:44.

Wang

, Varma

, Prosperi

. A systematic review of the effectiveness of mobile apps for monitoring and management of mental health symptoms or disorders. J Psychiatr Res, 2018; 107:73–78.

Grist

, Porter

, Stallard

. Mental health mobile apps for preadolescents and adolescents: A systematic review. J Med Internet Res, 2017; 19:e176.

Rathbone

, Prescott

. The use of Mobile apps and SMS messaging as physical and mental health interventions: Systematic review. J Med Internet Res, 2017; 19:e295.

10.

Leung

, Hung

, Ho

, et al. Prevalence of DSM-IV disorders in Chinese adolescents and the effects of an impairment criterion: A pilot community study in Hong Kong. Eur Child Adolesc Psychiatry, 2008; 17:452–461.

11.

Yuen

WWY

, Liu

, Tse

. Adolescent mental health problems in Hong Kong: A critical review on prevalence, psychosocial correlates, and prevention. J Adolesc Health, 2019; 64(6S):S73–S85.

12.

Joyful@HK. Tips on being joyful. www.joyfulathk.hk/en/contents.asp?id=40 (accessed July 18, 2016).

13.

Seligman

MEP

. Flourish: A Visionary New Understanding of Happiness and Well-Being. New York, NY: Free Press; 2011.

14.

Seligman

MEP

. Authentic Happiness: Using the New Positive Psychology to Realize Your Potential for Lasting Fulfillment. New York, NY: Free Press; 2002.

15.

Bolier

, Haverman

, Westerhof

, et al. Positive psychology interventions: A meta-analysis of randomized controlled studies. BMC Public Health, 2013; 13:119.

16.

Gander

, Proyer

, Ruch

, et al. Strength-based positive interventions: Further evidence for their potential in enhancing well-being and alleviating depression. J Happiness Stud, 2012; 14:1241–1259.

17.

Meyers

, van Woerkom

, Bakker

. The added value of the positive: A literature review of positive psychology interventions in organizations. Eur J Work Organ Psychol, 2013; 22:618–632.

18.

Sin

, Lyubomirsky

. Enhancing well-being and alleviating depressive symptoms with positive psychology interventions: A practice-friendly meta-analysis. J Clin Psychol, 2009; 65:467–487.

19.

Wei

, Li

. The Confucian value of harmony and its influence on Chinese social interaction. Cross-Cultural Commun, 2013; 9:60–66.

20.

Hong Kong Census and Statistics Department. Thematic Household Survey Report—Report No. 67, 2019:53.

21.

Billingham

, Whitehead

, Julious

. An audit of sample sizes for pilot and feasibility trials being undertaken in the United Kingdom registered in the United Kingdom Clinical Research Network database. BMC Med Res Methodol, 2013; 13:104.

22.

Teare

, Dimairo

, Shephard

, et al. Sample size requirements to estimate key design parameters from external pilot randomised controlled trials: A simulation study. Trials, 2014; 15:264.

23.

Whitehead

, Julious

, Cooper

, et al. Estimating the sample size for a pilot randomised trial to minimise the overall trial sample size for the external pilot and main trial for a continuous outcome variable. Stat Methods Med Res, 2016; 25:1057–1073.

24.

Lyubomirsky

, Lepper

. A measure of subjective happiness: Preliminary reliability and construct validation. Social Indic Res, 1999; 46:137–155.

25.

Nan

, Ni

, Lee

, et al. Psychometric evaluation of the Chinese version of the subjective happiness scale: Evidence from the Hong Kong FAMILY cohort. Int J Behav Med, 2014; 21:646–652.

26.

Stewart-Brown

, Tennant

, et al. Internal construct validity of the Warwick-Edinburgh Mental Well-being Scale (WEMWBS): A Rasch analysis using data from the Scottish Health Education Population Survey. Health Qual Life Outcomes, 2009; 7:15.

27.

Sun

, Luk

, Wang

, et al. The reliability and validity of the Chinese Short Warwick-Edinburgh Mental Well-being Scale in the general population of Hong Kong. Qual Life Res, 2019:1–8.

28.

Shen

, Wang

, Ho

, et al. Test–retest reliability and validity of a single-item Self-reported Family Happiness Scale in Hong Kong Chinese: Findings from Hong Kong Jockey Club FAMILY Project. Qual Life Res, 2019; 28:535–543.

29.

Wang

, Wang

, Viswanath

, et al. Digital inequalities of family life information seeking and family well-being among Chinese adults in Hong Kong: A population survey. J Med Internet Res, 2014; 16.

30.

Chan

, Viswanath

, Au

, et al. Hong Kong Chinese community leaders' perspectives on family health, happiness and harmony: A qualitative study. Health Educ Res, 2011; 26:664–674.

31.

Marston

, Peacock

, Yu

, et al. Comparing methods of analysing datasets with small clusters: Case studies using four paediatric datasets. Paediatr Perinat Epidemiol, 2009; 23:380–392.

32.

Armijo-Olivo

, Warren

, Magee

. Intention to treat analysis, compliance, drop-outs and how to deal with missing data in clinical research: A review. Phys Ther Rev, 2009; 14:36–49.

33.

Royston

. Multiple imputation of missing values: Update. Stata J, 2005; 5:188–201.

34.

Vaismoradi

, Turunen

, Bondas

. Content analysis and thematic analysis: Implications for conducting a qualitative descriptive study. Nurs Health Sci, 2013; 15:398–405.

35.

Sanders

, Kirby

, Tellegen

, et al. The triple P-positive parenting program: A systematic review and meta-analysis of a multi-level system of parenting support. Clin Psychol Rev, 2014; 34:337–357.

36.

Durlak

, Wells

. Primary prevention mental health programs for children and adolescents: A meta-analytic review. Am J Community Psychol, 1997; 25:115–152.

37.

Sanders

. Adolescent psychosocial, social, and cognitive development. Pediatr Rev, 2013; 34:354–358; quiz 58–59.

38.

Zepke

, Leach

. Improving student engagement: Ten proposals for action. Act Learn Higher Educ, 2010; 11:167–177.

39.

McCaffrey

, Westervelt

. Issues associated with repeated neuropsychological assessments. Neuropsychol Rev, 1995; 5:203–221.

40.

Bartels

, Wegrzyn

, Wiedl

, et al. Practice effects in healthy adults: A longitudinal study on frequent repetitive cognitive testing. BMC Neurosci, 2010; 11:118.

41.

Bell

, Lamport

, Field

, et al. Practice effects in nutrition intervention studies with repeated cognitive testing. Nutr Healthy Aging, 2018; 4:309–322.

42.

Cui

, Hung

, Wang

, et al. Issues related to subgroup analysis in clinical trials. J Biopharm Stat, 2002; 12:347–358.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.86 MB

0.07 MB