A Preliminary Randomized Controlled Evaluation of a Universal Healthy Relationships Promotion Program for Youth

Abstract

Bullying and mental health problems are pressing concerns for adolescents. Given their burden, we need to find efficacious ways to prevent these experiences. However, existing prevention programs tend to be single-issue and may not focus on the universal capacities required to reduce these problems among youth. To this end, we evaluated the universal, small groups Healthy Relationships Plus (HRP) program, which focuses on the promotion of positive mental health and the reduction of bullying and substance misuse. A sample of 212 youth from Southwestern Ontario were randomly assigned to the HRP or an attention-control condition over an 8-day period during summer 2014. Primary outcome measures (i.e., most important outcomes for this evaluation) were positive mental health, bullying victimization/perpetration, and substance misuse (alcohol, marijuana). We also examined a mediator (help-seeking) and two moderators (sex, adverse childhood experiences) of main effects. Participation in HRP was associated with reduced odds of physical bullying victimization at 1-year follow-up, compared with adolescents in the attention-control condition. This finding was mediated by increased intention to seek help from a mental health professional following HRP participation. We did not find main effects for positive mental health or substance misuse in this sample; however, there was an interaction effect whereby youth with significant trauma experiences reported less marijuana use at 1-year follow-up compared with control students. Findings reiterate the importance of help-seeking for bullying prevention and demonstrate the preliminary efficacy of the HRP as a universal strategy for preventing bullying victimization among mid-adolescents within school and community settings.

Keywords

adolescence bullying mental health prevention substance misuse

Introduction

Universal approaches that promote adolescent well-being are a critical part of tiered school mental health strategies (Short, 2016; Stephan, Sugai, Lever, & Connors, 2015). For middle school students, important targets of universal school mental health programs include social-emotional learning competencies and healthy relationships skills (Freeman, King, & Pickett, 2016; Greenberg, Domitrovich, Weissberg, & Durlak, 2017). Furthermore, as universal programs are designed to be offered to all students (i.e., are not based on the presence of any particular risk factor or diagnoses), current evidence suggests that integrated, prevention science–based approaches may be especially promising for improving population health. Through integrated approaches (i.e., approaches that target common risk and protective factors underlying multiple domains), universal programs can address multiple outcomes and also build the competencies needed to engage in healthy, respectful relationships with peers and dating partners (Wolfe, Jaffe, & Crooks, 2006). In this regard, we evaluated an integrated, universal healthy relationships approach to promoting positive mental health and preventing substance misuse and bullying among Canadian adolescents.

Domain-Integrated Prevention Programs

Adolescent mental health problems and substance misuse are prevalent public health concerns in Canada and elsewhere. Poor mental health is prevalent among adolescents (Freeman et al., 2011; Perou et al., 2013) and many mental health issues have their onset in the adolescent period (Auerbach et al., 2018). Furthermore, national Canadian data indicate that 19% to 30% of adolescents report past month binge drinking and that 21% to 37% report lifetime marijuana use (Young et al., 2011). Bullying is also a common experience in adolescence. Data from the Health Behavior in School-Aged Children survey indicate that one in four youth in Canada experienced bullying behavior at least twice a week in the past 2 months, and that the number of students experiencing bullying significantly increased between 2006 and 2014 (Freeman et al., 2016). A growing body of literature also demonstrates the association of bullying experiences with a number of adverse mental health outcomes, including substance use, depression/anxiety, and suicidal ideation (e.g., Klomek, Marrocco, Kleinman, Schonfeld, & Gould, 2007; Ttofi, Farrington, & Losel, 2011).

However, despite a noted overlap in outcomes, the majority of universal prevention programs for adolescents focus on single problem issues (e.g., bullying, substance misuse). Yet, many adolescent risk behaviors arise from common root causes and as such can be considered linked. For example, considering the individual level, Jessor’s (2016) problem behavior theory suggests that these multiple risk behaviors are linked by underlying risk and protective factors, including biology/genetics, the social environment, the perceived environment, personality, and behavior. At the interpersonal level, Wolfe et al. (2006) argue that adolescent risk behaviors are linked through how they are embedded in relationships with peers, parents, and social others, and thus by focusing on the importance of healthy interpersonal relationships, multiple risk behaviors can be targeted at the same time. Finally, incorporating macro-level factors, an intersectionality approach demonstrates that these behaviors are linked through the individual’s experience of multiple and overlapping systems of oppression (Cole, 2009). As such, creating integrated prevention programs focused on multiple health risk behaviors with common root causes (i.e., as opposed to single-issue programs) is increasingly recognized as an efficacious approach to health promotion (Catalano, Hawkins, Berglund, Pollard, & Arthur, 2002; Hale, Fitzgerald-Yau, & Viner, 2014). When considering common risk and protective factors as part of an integrated approach, emerging evidence documents that healthy relationships skills and capacities are foundational for promoting well-being among middle school students (Pepler & Craig, 2011; Wolfe et al., 2006). Given the importance of social-emotional learning competencies to the ability to engage in safe, supportive, and violence-free relationships (Exner-Cortens, Wells, Lee, & Spiric, 2018a), many healthy relationships programs focus on the promotion of social-emotional competencies as core program components (e.g., Crooks, Zwarych, Hughes, & Burns, 2015).

The presence of healthy relationships is related to fewer mental health problems (Freeman et al., 2016), including substance use (Ttofi, Farrington, Losel, & Loeber, 2011), and healthy relationships strategies also directly target bullying prevention through their focus on power and aggression (Pepler & Craig, 2011). Healthy relationships approaches also recognize how peers are implicated in adolescent well-being, both through influence on risk-taking behaviors (Gardner & Steinberg, 2005) and through promotion of social support and health promotive behaviors (Gulliver, Griffiths, & Christensen, 2010). Overall, a relationship promotion strategy can be successful at reducing adolescent risk behaviors because it engages youth in making healthy choices, and anchors these choices in the context of their relationships with peers, romantic partners, and adults (Wolfe et al., 2006). Finally, as part of a tiered mental health strategy, universal healthy relationships approaches can help meet Tier 1 goals for improving school climate and culture by promoting the skills and capacities required for inclusive, respectful environments (Exner-Cortens et al., 2018a).

In sum, although adolescence is a period with increased vulnerability for engagement in risk behaviors, it is also a time for building positive assets and skills, and both risk and asset building occur within the context of relationships. Thus, prevention programs can simultaneously encourage healthy adolescent development and the reduction of negative health risk behaviors by focusing on the importance of relationships (Wolfe et al., 2006, 2009).

Healthy Relationships Plus (HRP) Program: Theory and Overview

Given the theoretical promise of such integrated healthy relationships approaches, this article presents a randomized controlled evaluation of the HRP program. The HRP focuses on healthy relationships as a health promotion and harm reduction strategy for multiple adverse health outcomes, including mental health problems, substance misuse, and bullying, in adolescence (Figure 1). To simultaneously promote assets and reduce risk factors, the intervention is grounded in both competence enhancement (e.g., through promoting communication skills) and social resistance skills training (e.g., through analyzing media portrayals of substance use; Botvin & Griffin, 2004). The program also draws on social learning theory (Bandura, 1977), recognizing the importance of building self-efficacy around the social skills needed to reduce harm, as well as the theory of planned behavior (Ajzen, 1991).

Figure 1.

Overview of the HRP program.

The HRP is a small groups program designed to be implemented with adolescents aged 14 to 18 years in school or community settings. The HRP utilizes core components from the evidence-based, Canadian Fourth R program, a classroom-based healthy relationships promotion and dating violence prevention curriculum (Crooks et al., 2015; Crooks, Scott, Ellis, & Wolfe, 2011; Wolfe, Crooks, Chiodo, Hughes, & Ellis, 2012; Wolfe et al., 2009). However, the HRP adds more extensive skills practice and small-group interactions over 15, 1-hr lessons to help youth develop effective and healthy responses to risk-taking situations and also has greater emphasis on mental health (Figure 1). The HRP was designed to be flexible in delivery (per requests from community partners and implementers of the Fourth R) to reduce barriers to implementation: HRP can be delivered during the school day as a class-based universal intervention or delivered outside of class time as a universal, small groups program. Lessons within the program are also designed to be offered in a flexible format to best meet the needs of program implementers (e.g., lessons can be offered over the course of 15 weeks or in a condensed 1-week camp format). For more information on the program, see www.youthrelationships.org.

Current Study

Based on prior evaluations of the Fourth R program, we anticipated that participants in the HRP would demonstrate significant and lasting changes in health risk behaviors and positive social skills. To address these questions, we examined 12-month follow-up outcome data from a randomized controlled trial of the HRP. Our primary hypotheses were that HRP participants (compared with youth in an attention-control condition) would show

Hypothesis 1: Positive change in past-month positive mental health from pretest to 12-month follow-up.

Hypothesis 2: Reduced odds of past-month binge drinking and marijuana use from pretest to 12-month follow-up.

Hypothesis 3: Reduced odds of past-month physical bullying victimization and perpetration from pretest to 12-month follow-up.

As well, we sought to examine for whom the program was effective, which involved assessing intraindividual differences that might moderate program effectiveness, an increasingly important approach in prevention science (Guerra, Boxer, & Cook, 2006). Given past work demonstrating sex differences in Fourth R program outcomes (Wolfe et al., 2009) and differential program impacts for youth with a trauma history (Crooks et al., 2011), we chose to specifically examine sex and adverse childhood experiences as potential moderators of HRP outcomes. In addition, to understand the mechanism of program effects, we examined help-seeking (a key target of program activities; Figure 1) as a potential mediator of treatment–outcome relationships (Rickwood, Deane, Wilson, & Ciarrochi, 2005).

Method

Participants

Participants were recruited from five high schools in a medium-sized city in Southwestern Ontario, Canada. Administrators at participating schools had an existing relationship with the researchers and expressed willingness to participate in the study. Students were invited to participate by their school guidance counselor through regular announcements and posters in the guidance office. To be eligible, students needed to be in the ninth or 10th grade, available for the entire duration of the program offering, and not identified by their school guidance counselor as posing a serious safety risk to themselves or others (this criterion was added at the request of the participating schools). Of the 222 participants randomized to condition, approximately two thirds were female (Figure 2). The retention rate across the study was 90.6% (Figure 2).

Figure 2.

CONSORT flow diagram.

Study Design

Participants were randomized to treatment and control groups, using school, grade, and sex as stratification variables. Specifically, prior to randomization, participants were stratified by school (four levels¹), grade (two levels), and sex (two levels). Randomization was then conducted within each of the defined strata (e.g., Grade-9 girls from School 1). Stratification reflects the school-based recruitment strategy and the desire to balance groups by grade and sex. Following randomization, participants were allocated to the HRP or to an attention-control condition designed for this study (Figure 2; Supplemental Material). Across all involved schools, 14 groups (seven treatment, seven control) were offered simultaneously to 212 students. Additional details on the two conditions are available as Supplemental Material.

Procedures

At each school, information about the project was presented to students in an assembly and in daily school announcements. Guidance counselors gave a package containing study information, parental consent and youth assent documents, and a parent questionnaire to all students who expressed interest in the project. Given the significant time commitment for participation, students received compensation for completing the research (Can$225 total over the course of the five outcome and eight process evaluation surveys), as well as bus tickets to get to and from the program each day (if needed). We administered the baseline assessment (T0) on the day prior to the start of both conditions.

Posttesting (T1) immediately followed the conclusion of the program. Pre- and posttests were administered by computer at all but one school where surveys were administered on paper. Follow-up surveys were administered at 4 months (T2), 8 months (T3), and 12 months (T4) after the program. Follow-up surveys were sent to students by email and completed online at the student’s home or school. Because this study is focused on the primary outcome measures, only data collected at T0, T1, and T4 are reported.

On each day of the program, we also collected process evaluation information from both youth and facilitators. Specifically, youth completed daily feedback surveys asking for their thoughts on the day’s activities and, on the final day of the program, were asked to answer additional questions on things they learned during the program and for their overall thoughts. Facilitators completed daily youth attendance/engagement checklists and activity fidelity checklists, as well as an end-of-program implementation survey. Overall, the activity fidelity checklists in this project indicated that fidelity was high (i.e., on average, facilitators implemented 90% of HRP activities). For more detail on process evaluation tools and findings, see Supplemental Material.

Measures

Outcomes

We assessed positive mental health at T0 and T4 using the Mental Health Continuum–Short Form (MHC-SF), which uses 14 items to capture emotional (e.g., “How often did you feel happy?”), social (e.g., “How often did you feel that you had something important to contribute to society?”), and psychological (e.g., “How often did you feel good at managing the responsibilities of your daily life?”) well-being during the past month (1 = never to 6 = every day; alpha, T0 = .94). This scale has evidence of reliability and validity in an adolescent sample (Keyes, 2005). Responses across all items were averaged to create the total score, with higher scores indicating more positive mental health.

We assessed bullying at T0 and T4 using the Bullying Evaluation and Strategies Tool (BEST; PREVNet, 2014). Respondents were asked to report any physical, verbal, social, or electronic bullying that they experienced or perpetrated in the past month (0 = never to 3 = at least once a week). Bullying was indicated by the experience of any physical bullying because physical bullying in this sample was an indicator of more serious involvement with bullying in the past month. For example, over 80% of those reporting physical bullying reported at least two other forms of bullying, as well. Because of the sample distribution, physical bullying was dichotomized for analyses (1 = bullying experienced, 0 = no bullying).

Substance misuse was indicated at T0 and T4 by two variables: any binge drinking in the past month (1 = yes, 0 = no) and any marijuana use in the past month (1 = yes, 0 = no). These variables were taken from the Youth Risk Behavioral Surveillance Survey (Centers for Disease Control and Prevention, 2011), a survey with demonstrated reliability in adolescent samples (Brener, Collins, Kann, Warren, & Williams, 1995).

Mediator

We assessed intentions to seek help from a variety of individuals (e.g., friend, mental health professional, doctor) at T1 using the General Help-Seeking Questionnaire (GHSQ; Wilson, Deane, Ciarrochi, & Rickwood, 2005). Based on the content of the HRP (Figure 1), intention to seek help from a mental health professional was chosen as the mediator variable for this study. Responses to this question were measured on a 4-point scale (1 = very unlikely to 4 = very likely). Higher scores indicate greater intentions to seek help. This scale was developed to assess help-seeing intentions in adolescent samples (Wilson et al., 2005).

Moderators

Adverse childhood experiences (ACEs), including physical, emotional, and sexual maltreatment; neglect; and witnessing domestic violence during the participant’s lifetime, were assessed at T1² using 10 items from the Adverse Childhood Experiences study. A dichotomous score was created, indicating whether the participant had experienced four or more ACEs. The cut point of four was chosen based on prior research demonstrating that the experience of four or more ACEs is associated with greatly increased health risk (Felitti et al., 1998). Eighteen percent of the sample (n = 38) had experienced four or more ACEs during their lifetime.

Sex at birth was reported by the participant’s parent on study recruitment material (0 = male, 1 = female).³

Demographics

We included grade (nine or 10), age on the baseline assessment day, population group (White vs. non-White), pubertal development, family structure (two-parent home vs. other structure), and socioeconomic status, as indicated by the highest level of parent education (four levels). Grade, age, population group, family structure, and socioeconomic status were assessed by parent report on the study recruitment material. Pubertal status was assessed via student report using four sex-specific items (Foster, Hagan, & Brooks-Gunn, 2004), based on items in the Pubertal Development Scale. Each item was standardized to M = 0 and SD = 1, and then averaged. Higher scores indicate more advanced development compared with same-sex peers in the sample (Table 1).

Table 1.

Baseline Characteristics of the Full Sample at T0 (n = 212).

Variable	Treatment group (n = 108)	Control group (n = 104)	p value
Demographics
Sex: % (n), female	68.5 (74)	65.4 (68)	.628
Grade: % (n), Grade 9	50.0 (54)	50.0 (52)	1.00
Age: M (SD)	15.53 (0.61)	15.49 (0.56)	.662
Population group: % (n), White	71.0 (76)	82.5 (85)	.049
Pubertal development: M (SD)	0.098 (0.95)	−0.099 (1.04)	.152
Family structure: % (n), two-parent home	40.7 (44)	41.3 (43)	.929
Socioeconomic status: % (n), parent education high school or less	17.9 (19)	14.0 (14)	.786
Outcomes
Positive mental health: M (SD)	3.98 (1.14)	3.92 (1.14)	.732
Physical bullying victimization: % (n), yes	29.6 (32)	22.5 (23)	.243
Physical bullying perpetration: % (n), yes	12.0 (13)	5.8 (6)	.115
Any binge drinking in past month: % (n), yes	9.3 (10)	20.2 (21)	.026
Any marijuana use in past month: % (n), yes	13.1 (14)	12.5 (13)	.899
Mediator
GHSQ, Mental health professional: M (SD)	2.24 (1.03)	2.08 (0.87)	.215
Moderators
Adverse childhood experiences: % (n), > = 4	17.0 (18)	19.6 (20)	.624

Note. T0 = baseline assessment; GHSQ = General Help-Seeking Questionnaire.

Analysis

We explored T0 data through bivariate analyses to investigate baseline equivalency, using χ² or t tests as appropriate. Associations with outcome variables at T4 were explored using structural equation models in MPlus v.7 (Muthén & Muthén, 2010). Intent-to-treat analyses were used, and missing data were handled using full information maximum likelihood (Muthén & Muthén, 2010). The main predictor for all models was group status (treatment vs. control). All models controlled for school, grade, and sex (i.e., the three stratification variables); population group; and the outcome variable at T0. Mediation was explored using bias-corrected bootstrap confidence intervals (CIs) in MPlus v7 (MacKinnon, Lockwood, & Williams, 2004). Moderation was explored in MPlus v7. If a significant interaction was found, probing of simple slopes for dichotomous outcomes was conducted using the PROCESS macro (v.16) for SPSS (Hayes, 2013). Results for this study were evaluated at the p < .05 level.

Results

Baseline Equivalence

As shown in Table 1, the treatment and control groups were well-balanced at baseline. Equivalence was found on all demographics except population group, with a higher percentage of the control group reporting White population group (Table 1). To address this imbalance, all multivariate models included population group as a covariate. All outcome variables were balanced at baseline except for binge drinking, which was more likely to be reported by the control group (Table 1). All putative mediators and moderators were balanced at T0.

Main Effects

Main effect models are shown in Table 2. The only significant effect of the treatment condition was on bullying victimization at T4: compared with control group participants, treatment group participants had 0.34 times the odds of bullying victimization 1 year following the program (Table 2). There were no main effects of the program on positive mental health, bullying perpetration, or substance misuse.

Table 2.

Regression Results for Program Outcomes (n = 187).

	Treatment, % (n) or M (SD)		Control, % (n) or M (SD)		aOR (95% CI) or b (95% CI)
	T0	T4	T0	T4	aOR (95% CI) or b (95% CI)
Past month positive mental health	3.98 (1.14)	4.16 (1.14)	3.92 (1.14)	4.15 (1.22)	0.003 [−0.28, 0.28]
Any past month physical bullying victimization	27.1 (26)	10.5 (10)	21.3 (20)	22.9 (22)	0.34 [0.14, 0.84]
Any past month physical bullying perpetration	11.5 (11)	8.4 (8)	5.3 (5)	8.4 (8)	0.91 [0.29, 2.88]
Any past month binge drinking	10.4 (10)	26.6 (25)	20.8 (20)	24.2 (23)	1.91 [0.84, 4.33]
Any past month marijuana use	14.6 (14)	13.5 (13)	17.9 (17)	18.9 (18)	0.93 [0.40, 2.20]

Note. All models controlled for sex, grade, school, population group, and the outcome at T0. Bold text represents a significant finding at the p < .05 level. T0 = baseline assessment; T4 = follow-up survey administered at 12 months after the program; aOR = adjusted odds ratio; CI = confidence interval.

Mediation Effects

Mediation analyses were conducted to understand the main effect of the HRP on bullying victimization. In bivariate analyses (t tests), physical bullying victimization at T4 was related to lower intention to seek help from a mental health professional at T1 (p = .036), while being in the treatment group was associated with greater intentions to seek help from a mental health professional at T1 (p = .007). In multivariate models, intention to seek help was part of an indirect effect, IE = −.13, bias-corrected bootstrap 95% CI = [–0.47, –0.001]; Figure 3. This suggests that the reduced likelihood of physical bullying victimization for treatment group participants was explained by their intentions to seek help from a mental health professional following the program.

Figure 3.

Mediation effect.

Moderation Effects

Sex

There were no significant Sex × Group interactions for any of our outcome variables.

Adverse childhood experiences

There was a significant interaction effect between treatment group and ACEs for marijuana use at T4 (adjusted odds ratio [aOR]_int = 0.094, 95% CI = [0.010, 0.85], p = .035). Post hoc probing of this interaction revealed both a between- and within-groups effect. Between groups, individuals in the control group who reported four or more ACEs in their lifetime were somewhat more likely to use marijuana at T4 than individuals in the treatment group with four or more ACEs (aOR_Treatment = 0.18, 95% CI = [0.027, 1.17], p = .07; Figure 4a). Within-groups, control group participants with higher ACEs scores had somewhat higher odds of marijuana use at T4 than control group participants with lower ACEs scores (aOR_ACEs≥4 = 4.10, 95% CI = [0.97, 17.29], p = .06; Figure 4b). However, the cell sizes for this analysis were very small, and thus these findings should be interpreted as preliminary only. There were no significant ACEs × Group interactions for positive mental health or bullying.

Figure 4.

Interaction between ACEs and group for marijuana use at T4: (a) between groups (light gray line = control condition, dark gray line = treatment condition) and (b) within groups (light gray line = four or more ACEs, dark gray line = less than four ACEs).

Discussion

We conducted a preliminary randomized controlled trial to determine if participation in the universal HRP was associated with improved mental health and reductions in substance misuse and bullying victimization and perpetration, as compared with a strong attention-control condition. In this nonclinical sample of mid-adolescents, participation in the HRP was associated with significantly lower odds of physical bullying victimization 1 year later, and this effect was mediated by increased likelihood to seek help from a mental health professional following the program. Thus, hypotheses were partially confirmed.

Contrary to Hypothesis 1, we did not find either a main or subgroup effect for positive mental health in this sample. HRP content focuses on understanding stressors, signs, and symptoms of mental health issues, and the promotion of help-seeking (Figure 1). We did find that, immediately following the program, intervention participants were significantly more likely to report greater intentions to seek help from a mental health professional. However, this intention did not translate into improved positive mental health. The lack of a main effect for mental health may be because both groups’ mental health improved slightly over the course of the year or because positive mental health status at baseline in this nonclinical population was already fairly good, resulting in a ceiling effect. For example, we had limited power to detect effects in subgroups that may have had worse mental health at baseline (e.g., adolescents with four or more ACEs). Indeed, when the HRP was recently evaluated in higher risk groups of youth using a within-groups design, there were associations with reduced depression, particularly for those starting with higher levels (Lapshina, Crooks, & Kerry, 2018).

We also did not find a main effect for substance misuse in this sample, as predicted by Hypothesis 2. The treatment group was significantly less likely to have participated in binge drinking at baseline but, by 1-year follow-up, appears to have caught up with their control group counterparts and both the control and treatment groups increased in drinking over the follow-up period. While this increase in drinking is developmentally expected (Schulenberg & Maggs, 2002), it suggests no impact of intervention on drinking in this sample. In terms of marijuana use, the treatment group did decrease slightly, while the control group increased slightly (Table 2); however, this effect was not significant. When subgroups were analyzed, control group participants with a more severe trauma history were somewhat more likely to use marijuana at 1-year follow-up than treatment group participants with this same history, suggesting a potential protective effect for these more vulnerable youth. While this aligns with prior findings of a protective effect of Fourth R programming for maltreated youth (Crooks et al., 2011), this finding should be interpreted as preliminary due to the small sample size.

As predicted by Hypothesis 3, we did find an impact on physical bullying victimization in this sample. We coded bullying victimization as any experience in the past month, as we did not have the power to look at bullying as an ordinal variable, and the coding we used is typical in bullying research (e.g., Espelage, Low, Polanin, & Brown, 2015). However, an examination of frequencies for the ordinal variable suggests that the treatment effect was the result of reduced victimization across all categories (data not shown). We did not find an impact on bullying perpetration in this sample, but this may have been due to the small number of youth who reported perpetration at baseline (n = 19 youth).

We also found that the reduction in bullying victimization was mediated by an increased intention to seek help from a mental health professional following the program. If this intention translated into actual help-seeking, the bullying effect may be the result of feelings of increased social support (Hansen, Steenberg, Palic, & Elklit, 2012), better social competence (Cook, Williams, Guerra, Kim, & Sadek, 2010), or improved problem-solving strategies (Wilton, Craig, & Pepler, 2000), all of which are linked to reduced risk for bullying victimization and are likely outcomes of seeking help from a mental health professional. This finding aligns well with the goals and strategies of the program, in that help-seeking is intentionally taught and barriers to help-seeking are explored with participants.

Limitations

The most significant limitations to this study were the relatively modest sample size (particularly given our interest in looking at subgroup analyses) and the low-risk nature of the sample (which reflects the universal nature of the program). Given our final enrolled sample size (N = 212), we had 80% power to detect a medium effect size (Cohen’s d) at the p = .05 level (d = .39). Despite significant efforts to recruit youth (including adding an additional study site), we were unable to obtain a larger sample size. This was likely due to the implementation of the study over the summer session, as some students who might have participated were employed, away for the summer, or attending summer school. While this provides important information about the feasibility of running the HRP in a camp-like format, it is likely that implementation during the school year (and potentially during the school day) would increase program enrollment. The overrepresentation of girls in our sample (~2:1 ratio) also may have affected our ability to explore sex as a moderator of program outcomes and may also affect generalizability of study findings. We also note that, given other data collected in this study (particularly on executive functioning), we asked about sex at birth and not gender identity, which limits our analyses in some ways (i.e., we cannot explore differential outcomes for nonbinary individuals). However, in a recent large-scale national implementation study of the HRP (Lapshina et al., 2019), less than 2% of the participants reported a nonbinary identity, and thus it is unlikely we would have had the power to examine differential effects for this important subgroup. The compressed delivery of the program was another potential limitation with respect to assessing the possible impacts; although the program was designed to be flexible in terms of delivery, and is offered in this condensed format in some community-based settings, many skill-building approaches recognize the need for spaced practice and time to consolidate new skills and attitudes. Thus, it is possible that had the program been offered over an extended period of time, additional outcomes may have emerged. This is an important area for future study of this program.

Relevance to the Practice of School Psychology

In this sample of nonclinical mid-adolescents, participation in the HRP was associated with significantly lower rates of physical bullying victimization 1 year following the program, even when controlling for bullying experiences at baseline and compared with a strong attention-control condition. Given the high rates of bullying victimization in Canada (Freeman et al., 2016) and the limited availability of effective bullying prevention programs for older students (Yeater, Fong, Lee, & Espelage, 2015), this is an important finding, and school psychologists may wish to consider incorporating the HRP as a Tier 1 resource. This article thus adds to the school psychology evidence base, by demonstrating a potential intervention for implementation within the school setting, including as part of tiered mental health strategies. The HRP also fits within tiered systems designed to promote trauma-informed practice, particularly through its focus on social skill development and related potential contributions to positive school climate (Chafouleas, Johnson, Overstreet, & Santos, 2016), and through its intention to build healthy, safe, and supportive relationships within the school environment (Rossen & Cowan, 2013). Attending HRP training could also contribute to advancing school psychologists’ knowledge and skills base for universal, trauma-informed, healthy relationships intervention for middle school students (a trauma-informed focus is specifically included in program training content).

Mediation analyses suggested that our bullying finding was the result of an increased likelihood to seek help from a mental health professional. Help-seeking is viewed as a critical piece of bullying prevention (Eliot, Cornell, Gregory, & Fan, 2010), as adolescents cannot receive the adult support often needed to stop bullying until they reach out for this help. This finding is also promising as adolescents are generally hesitant to seek help following bullying and may become less likely to seek help from an adult with age (Oliver & Candappa, 2007). Thus, participation in the HRP may be one way to promote help-seeking—including from school psychologists—among mid-adolescents as part of tiered school mental health strategies. However, to confirm these findings, further evaluation of the HRP with larger samples and in different formats is ongoing.

Finally, we note that overall, fidelity in this study was high (as is expected in an efficacy study), but that where facilitators were unable to implement all program activities, time emerged as a key barrier (see Supplemental Material). Time is also a commonly reported barrier in past implementation work on other Fourth R programs (Crooks, Chiodo, Zwarych, Hughes, & Wolfe, 2008; Exner-Cortens, Spiric, Crooks, Syeda, & Wells, 2018b). Thus, as part of planning for HRP implementation, we urge school psychologists to examine how the time needs of the program might best fit within the school environment (e.g., deciding when and how often to offer the program, leaving time for extra sessions at the end) and to be planful about adaptations to the program that arise due to time constraints. For guidance on program adaptation, school psychologists may wish to review the Fourth R implementation manual for examples of this type of work (Crooks et al., 2015; an HRP-specific implementation manual is forthcoming). We also encourage school psychologists to use activity fidelity checklists (provided at program training) to track delivery of the program in their building, which will allow for making data-informed adjustments and improvements to program delivery.

Supplemental Material

CJSP_HRP_Supplemental_Material.R1.Clean – Supplemental material for A Preliminary Randomized Controlled Evaluation of a Universal Healthy Relationships Promotion Program for Youth

Supplemental material, CJSP_HRP_Supplemental_Material.R1.Clean for A Preliminary Randomized Controlled Evaluation of a Universal Healthy Relationships Promotion Program for Youth by Deinera Exner-Cortens, David Wolfe, Claire V. Crooks and Debbie Chiodo in Canadian Journal of School Psychology

Footnotes

Acknowledgements

We would like to thank the participating teachers and students. Thanks also to Hua Shen, PhD, for her support with statistical analysis and to Ray Hughes for his support with data collection and program implementation.

Declaration of Conflicting Interests

The authors declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: The second and third authors are creators of the Fourth R and Healthy Relationships Plus programs. All analyses in this article were conducted by the first author. The first and fourth authors have no conflicts of interest to report.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported in part by a Fellowship Award from the Canadian Institutes of Health Research (MFE-135532) and by funding from the Centre for Addiction and Mental Health (CAMH) Foundation.

Supplemental Material

Supplemental material for this article is available online.

Notes

Author Biographies

Deinera Exner-Cortens is an assistant professor in the Faculty of Social Work, University of Calgary, and is jointly appointed to the Department of Psychiatry, Cumming School of Medicine, University of Calgary. Her research focuses on evaluating healthy relationships activities in school and community settings, and developing and evaluating implementation support tools for school-based mental health/healthy relationships service delivery. In her work, Dr. Exner-Cortens collaborates with a number of community and research partners both provincially and nationally.

David Wolfe is an adjunct professor at the Centre for School Mental Health and Research Scholar at the Centre for Research and Education on Violence Against Women and Children, Faculty of Education, Western University. He is also director emeritus of the Centre for Addiction and Mental Health (CAMH) Centre for Prevention Science. For over 40 years, Dr. Wolfe has been active in developing and evaluating violence and abuse prevention initiatives for children and youth.

Claire V. Crooks is the director of the Centre for School Mental Health at Western University. Her research focuses on the development and evaluation of mental health promotion and violence prevention programs. She is particularly interested in effective programming for vulnerable groups of youth.

Debbie Chiodo holds a PhD in Educational Psychology and is an evaluator with the Provincial System Support Program at the Centre for Addiction and Mental Health and assistant professor at Western University. Her research focuses on understanding the implementation and sustainability of evidence-based interventions in schools and communities.

References

Ajzen

(1991). The theory of planned behavior. Organizational Behavior and Human Decision Processes, 50, 179-211.

Auerbach

R. P.

Mortier

Bruffaerts

Alonso

Benjet

Cuijpers

. . . WHO WMH-ICS Collaborators. (2018). WHO World Mental Health Surveys International College Student Project: Prevalence and distribution of mental disorders. Journal of Abnormal Psychology, 127, 623-638. doi:10.1037/abn0000362

Bandura

(1977). Self-efficacy: Toward a unifying theory of behavioral change. Psychological Review, 84, 191-215.

Botvin

G. J.

Griffin

K. W.

(2004). Life skills training: Empirical findings and future directions. Journal of Primary Prevention, 25, 211-232.

Brener

N. D.

Collins

J. L.

Kann

Warren

C. W.

Williams

B. I.

(1995). Reliability of the Youth Risk Behavior Survey Questionnaire. American Journal of Epidemiology, 141, 575-580.

Catalano

R. F.

Hawkins

J. D.

Berglund

Pollard

J. A.

Arthur

M. W.

(2002). Prevention science and positive youth development: Competitive or cooperative frameworks? Journal of Adolescent Health, 31, 230-239.

Centers for Disease Control and Prevention. (2011). 2011 National Youth Risk Behavior Survey. Retrieved from http://www.cdc.gov/healthyyouth/yrbs/pdf/questionnaire/2011_xxh_questionnaire.pdf

Chafouleas

S. M.

Johnson

A. H.

Overstreet

Santos

N. M.

(2016). Toward a blueprint for trauma-informed service delivery in schools. School Mental Health, 8, 144-162.

Cole

E. R.

(2009). Intersectionality and research in psychology. American Psychologist, 64, 170-180.

10.

Cook

C. R.

Williams

K. R.

Guerra

N. G.

Kim

T. E.

Sadek

(2010). Predictors of bullying and victimization in childhood and adolescence: A meta-analytic investigation. School Psychology Quarterly, 25, 65-83.

11.

Crooks

C. V.

Chiodo

Zwarych

Hughes

Wolfe

D. A.

(2008). Predicting implementation success of an evidence-based program to promote healthy relationships among students two to eight years after teacher training. Canadian Journal of Community Mental Health, 32, 125-138.

12.

Crooks

C. V.

Scott

K. L.

Broll

Zwarych

Hughes

Wolfe

D. A.

(2015). Does an evidence-based healthy relationships program for 9th graders show similar effects for 7th and 8th graders? Results from 57 schools randomized to intervention. Health Education Research, 30, 513-519.

13.

Crooks

C. V.

Scott

Ellis

Wolfe

D. A.

(2011). Impact of a universal school-based violence prevention program on violent delinquency: Distinctive benefits for youth with maltreatment histories. Child Abuse & Neglect, 35, 393-400.

14.

Crooks

C. V.

Zwarych

Hughes

Burns

(2015). The Fourth R implementation manual: Building success from adoption to sustainability. Retrieved from https://youthrelationships.org/implementation-manual

15.

Eliot

Cornell

Gregory

Fan

(2010). Supportive school climate and student willingness to seek help for bullying and threats of violence. Journal of School Psychology, 48, 533-553.

16.

Espelage

D. L.

Low

Polanin

J. R.

Brown

E. C.

(2015). Clinical trial of Second Step middle-school program: Impact on aggression & victimization. Journal of Applied Developmental Psychology, 37, 52-63.

17.

Exner-Cortens

Wells

Lee

Spiric

(2018a). Building a culture of intimate partner violence prevention in Alberta, Canada through the promotion of healthy youth relationships. Revision requested.

18.

Exner-Cortens

Spiric

Crooks

C. V.

Syeda

Wells

(2018b). Predictors of healthy youth relationships program implementation in a sample of Canadian middle school teachers. Revision requested.

19.

Felitti

V. J.

Anda

R. F.

Nordenberg

Williamson

D. F.

Spitz

A. M.

Edwards

. . . Marks

J. S.

(1998). Relationship of childhood abuse and household dysfunction to many of the leading causes of death in adults: The Adverse Childhood Experiences (ACE) study. American Journal of Preventive Medicine, 14, 245-258.

20.

Foster

Hagan

Brooks-Gunn

(2004). Age, puberty, and exposure to intimate partner violence in adolescence. Annals of the New York Academy of Sciences, 1036, 151-166.

21.

Freeman

J. G.

King

Pickett

(2016). Health Behaviour in School-Aged Children (HBSC) in Canada: Focus on relationships. Retrieved from http://healthycanadians.gc.ca/publications/science-research-sciences-recherches/health-behaviour-children-canada-2015-comportements-sante-jeunes/alt/health-behaviour-children-canada-2015-comportements-santé-jeunes-eng.pdf

22.

Freeman

J. G.

King

Pickett

Craig

Elgar

Janssen

Klinger

(2011). The health of Canada’s young people: A mental health focus. Retrieved from http://www.phac-aspc.gc.ca/hp-ps/dca-dea/publications/hbsc-mental-mentale/assets/pdf/hbsc-mental-mentale-eng.pdf

23.

Gardner

Steinberg

(2005). Peer influence on risk taking, risk preference, and risky decision making in adolescence and adulthood: An experimental study. Developmental Psychology, 41, 625-635.

24.

Greenberg

M. T.

Domitrovich

C. E.

Weissberg

R. P.

Durlak

J. A.

(2017). Social and emotional learning as a public health approach to education. Future of Children, 27(1), 13-32.

25.

Guerra

N. G.

Boxer

Cook

C. R.

(2006). What works (and what does not) in youth violence prevention: Rethinking the questions and finding new answers. New Directions for Evaluation, 110, 59-71.

26.

Gulliver

Griffiths

K. M.

Christensen

(2010). Perceived barriers and facilitators to mental health help-seeking in young people: A systematic review. BMC Psychiatry, 10, Article 113.

27.

Hale

D. R.

Fitzgerald-Yau

Viner

R. M.

(2014). A systematic review of effective interventions for reducing multiple health risk behaviors in adolescence. American Journal of Public Health, 104, e19-e41.

28.

Hansen

T. B.

Steenberg

L. M.

Palic

Elklit

(2012). A review of psychological factors related to bullying victimization in schools. Aggression and Violent Behavior, 17, 383-387.

29.

Hayes

A. F.

(2013). Introduction to mediation, moderation, and conditional process analysis: A regression-based approach. New York, NY: Guilford Press.

30.

Jessor

(2016). Problem behavior theory and adolescent risk behavior: A re-formulation. In Jessor

(Ed.), The origins and development of problem behavior theory (pp. 117-130). Basel, Switzerland: Springer.

31.

Keyes

C. L. M.

(2005). The subjective well-being of America’s youth: Toward a comprehensive assessment. Adolescent & Family Health, 4, 3-11.

32.

Klomek

A. B.

Marrocco

Kleinman

Schonfeld

I. S.

Gould

M. S.

(2007). Bullying, depression, and suicidality in adolescents. Journal of the American Academy of Child and Adolescent Psychiatry, 46, 40-49.

33.

Lapshina

Crooks

C. V.

Kerry

(2019). Changes in depression and positive mental health among youth in a healthy relationships program. Canadian Journal of School Psychology, 34, 300-316.

34.

MacKinnon

D. P.

Lockwood

C. M.

Williams

(2004). Confidence limits for the indirect effect: Distribution of the product and resampling methods. Multivariate Behavioral Research, 39, 99-128.

35.

Muthén

L. K.

Muthén

B. O.

(2010). Mplus user’s guide (6th ed.). Los Angeles, CA: Author.

36.

Oliver

Candappa

(2007). Bullying and the politics of “telling.” Oxford Review of Education, 33, 71-86.

37.

Pepler

Craig

(2011). Promoting relationships and eliminating violence in Canada. International Journal of Behavioral Development, 35, 389-397.

38.

Perou

Bitsko

R. H.

Blumberg

S. J.

Pastor

Ghandour

R. M.

Huang

L. N.

(2013). Mental health surveillance among children—United States, 2005-2011. MMWR Weekly Supplements, 62, 1-35.

39.

PREVNet. (2014). Bullying Evaluation & Strategies Tool (BEST). Retrieved from http://www.prevnet.ca/resources/assessment-tool

40.

Rickwood

Deane

F. P.

Wilson

C. J.

Ciarrochi

(2005). Young people’s help-seeking for mental health problems. Australian e-Journal for the Advancement of Mental Health, 3, 218-251.

41.

Rossen

Cowan

(2013). The role of schools in supporting traumatized students. Principal’s Research Review, 8(6), 1-8.

42.

Schulenberg

J. E.

Maggs

J. L.

(2002). A developmental perspective on alcohol use and heavy drinking during adolescence and the transition to young adulthood. Journal of Studies on Alcohol, (Suppl. 14), 54-70.

43.

Short

(2016). Intentional, explicit, systematic: Implementation and scale-up of effective practices for supporting student mental well-being in Ontario schools. International Journal of Mental Health Promotion, 18, 33-48.

44.

Stephan

S. H.

Sugai

Lever

Connors

(2015). Strategies for integrating mental health into schools via a multitiered system of support. Child and Adolescent Psychiatric Clinics of North America, 2, 4211-4231.

45.

Ttofi

M. M.

Farrington

D. P.

Losel

(2011). Health consequences of school bullying. Journal of Aggression, Conflict and Peace Research, 3, 60-62.

46.

Ttofi

M. M.

Farrington

D. P.

Losel

Loeber

(2011). Do the victims of school bullies tend to become depressed later in life? A systematic review and meta-analysis of longitudinal studies. Journal of Aggression, Conflict and Peace Research, 3, 63-73.

47.

Wilson

C. J.

Deane

F. P.

Ciarrochi

Rickwood

(2005). Measuring help-seeking intentions: Properties of the General Help-Seeking Questionnaire. Canadian Journal of Counselling, 39, 15-28.

48.

Wilton

M. M. M.

Craig

W. M.

Pepler

D. J.

(2000). Emotion regulation and display in classroom victims of bullying: Characteristic expressions of affect, coping styles and relevant contextual factors. Social Development, 9, 226-245.

49.

Wolfe

D. A.

Crooks

C. V.

Chiodo

Hughes

Ellis

(2012). Observations of adolescent peer resistance skills following a classroom-based healthy relationship program: A post-intervention comparison. Prevention Science, 13, 196-205.

50.

Wolfe

D. A.

Crooks

Jaffe

Chiodo

Hughes

Ellis

… Donner

(2009). A school-based program to prevent adolescent dating violence: A cluster randomized trial. Archives of Pediatrics and Adolescent Medicine, 163, 692-699.

51.

Wolfe

D. A.

Jaffe

P. G.

Crooks

C. V.

(2006). Adolescent risk behaviours: Why teens experiment and strategies to keep them safe. New Haven, CT: Yale University Press.

52.

Yeater

D. S.

Fong

C. J.

Lee

H. Y.

Espelage

D. L.

(2015). Declines in efficacy of anti-bullying programs among older adolescents: Theory and a three-level meta-analysis. Journal of Applied Developmental Psychology, 37, 36-51.

53.

Young

M. M.

Saewyc

Boak

Jahrig

Anderson

Clark

(2011). Cross-Canada report on student alcohol and drug use: Technical report. Ottawa, Ontario: Canadian Centre on Substance Abuse.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.14 MB