Abstract
Purpose:
Adolescent mental health is a public health priority. Considered an early intervention approach, Youth Mental Health First Aid (YMHFA) trains adults to provide initial assistance to adolescents experiencing a mental health problem or crisis. This study evaluated the effectiveness of the U.S. version of YMHFA (YMHFA-USA) among graduate social work students.
Method:
A quasi-experimental design (N = 73; intervention, n = 39; comparison, n = 34), using the Mental Health Beliefs and Literacy Scale, assessed the effectiveness of YMHFA-USA. Outcomes were measured at pretest, posttest, and 5 months in the intervention group. Differences between groups were compared at 5 months.
Results:
Statistically significant improvements in attitudes, beliefs, self-confidence, and knowledge were observed among intervention group students. At 5 months, these students had significantly better attitudes and greater knowledge and self-confidence than the comparison group.
Conclusion:
Results indicate YMHFA-USA may improve factors related to master’s level social work students’ abilities to engage with youth experiencing mental health problems.
One in the five youths aged 13–18 years is diagnosed with a mental health or addictive disorder, with approximately 10% of those considered severely impacted (Merikangas et al., 2010). Although symptoms of many mental health problems manifest in childhood and adolescence (Kessler et al., 2007), diagnosis and treatment typically begin much later (Foliaki, Kokaua, Schaaf, & Tukuitonga, 2006; Kessler et al., 2007). This gap in treatment can lead to adverse long-term outcomes such as unemployment, homelessness, poverty, health and mental health problems, suicide (O’Connell, Boat, & Warner, 2009), and increased involvement in the juvenile justice system (Institute of Medicine, 2006). Prevention and early intervention programs are associated with better health and educational outcomes, reduced involvement in the criminal justice system, and are more cost-effective (O’Connell et al., 2009).
Delay in early intervention among children and adolescents is due, in part, to limited inclination to seek help; thus, youth are dependent on others to observe their signs and symptoms and refer them to the appropriate resources (Kitchener, Jorm, & Kelly, 2010). Youth who seek help often turn to people they are comfortable talking with such as family, friends, neighbors, and community members (Jorm, Wright, & Morgan, 2007). However, these natural helpers may lack the knowledge and skills to identify mental health problems and are thus unable to refer or assist the young person with getting access to prevention and treatment services (Kitchener et al., 2010). The limited number of practitioners with specialty training in child and adolescent mental health along with the lack of access to available evidence-based and developmentally appropriate services has also been identified as barriers to service delivery for this vulnerable population (Patel, Flisher, Hetrick, & McGorry, 2007; Tolan & Dodge, 2005). Additionally, the range of mental health disorders prevalent in youth varies greatly from depression, trauma, anxiety, and suicide, to substance abuse and eating disorders, and professionals trained in one skill area may not have the expertise to be aware of or address problems outside of their specialization (Blom, 2004; Smyth, Goodman, & Glenn, 2006). For example, social workers, who represent the largest providers of mental health services in the United States, have a variety of professional specializations, only some of which include skills specifically related to assessment and treatment of adolescent mental health (Gibelman, 1995; National Association of Social Workers [NASW], 2016).
Due to the profound risks associated with delayed mental health treatment coupled with the social and monetary gains from prevention, several U.S. governmental agencies have identified adolescent mental health as a public health priority warranting early intervention using research-supported and evidence-based practices (Healthy People 2020, 2012; O’Connell et al., 2009). However, there is little information available about the implementation and effectiveness of evidence-based or research-supported youth mental health prevention and early treatment programs (Office of Applied Studies, 2008).
Mental Health First Aid
Mental Health First Aid (MHFA) is a research-supported mental health prevention and early treatment program on the Substance Abuse and Mental Health Services Administration’s (SAMHSA, 2014) National Registry of Evidence-based Programs and Practices. Developed in Australia in 2000 and adapted for use in the United States in 2008 as MHFA-USA (Kitchener & Jorm, 2002a, 2002b), over 680,000 people have been trained to implement this intervention (MHFA-USA, 2015). MHFA is an 8-hr program that trains the public or laypeople (e.g., clergy, teachers, coworkers, parents, peers, and others) to build mental health literacy, which then enables them to assist adults experiencing mental health crises or who are in the early stages of developing a mental health problem.
MHFA is set within the context of the first-aid principles of providing immediate assistance until more specialized helpers can intervene (American Red Cross, 2014). Although no specific theoretical framework is identified by MHFA, the intervention is consistent with dimensions of the unified theory of behavior (UTB; Fishbein et al., 2001; Jaccard, Dodge, & Dittus, 2002). For example, in UTB, behaviors originate from beliefs, attitudes, social norms, and self-efficacy, all of which contribute to the intent to perform a behavior (i.e., behavioral intention). Subsequently, behavioral intention fosters actual behavior but is also affected by factors such as cues to action, habits, knowledge and skills, and environmental constraints and facilitators (Anthony, Banh, & Goldman, 2015). Thus, MHFA can be described as targeting determinants of behavioral intention and actual behaviors, as proposed by UTB, in order to foster helping behaviors intrinsic to the interventions’ core model —Assess for risk of suicide or harm, Listen nonjudgmentally, Give reassurance and information, Encourage appropriate professional help, and Encourage self-help and other support strategies (ALGEE; Kitchener, Jorm, & Kelly, 2009). The MHFA course also covers basic mental health knowledge including signs, symptoms, and treatments (Jorm, Blewitt, Griffiths, Kitchener, & Parslow, 2005; Kitchener et al., 2009) as well as stigma associated with mental health disorders (Corrigan, 2004).
Early evaluation of MHFA shows four key outcomes: (1) increase in mental health prevention behavior and behavioral intention, (2) increase in knowledge about mental illness and treatment, (3) increase in confidence and efficacy to assist those in mental health distress, and (4) decrease in stigma (Jorm, Kitchener, Fischer, & Cvetkovski, 2010; Kelly et al., 2011; Kitchener & Jorm, 2002b; Lipson, Speer, Brunwasser, Hahn, & Eisenberg, 2014; Morawska et al., 2013; Pierce, Liaw, Dobell, & Anderson, 2010; Svensson & Hansson, 2014; Wong, Collins, & Cerully, 2015; Yap & Jorm, 2012). Although most research evaluating MHFA is international, initial research of MHFA-USA shows participants have increased knowledge and confidence, combined with decreased mental health–related stigma (Lucksted, Mendenhall, Frauenholtz, & Aakre, 2015; Mendenhall, Jackson, & Hase, 2013). Participants in the Mendenhall, Jackson, and Hase’s (2013) study were mainly from the education field (51%) with approximately 9% employed in social service or mental health fields.
Youth Mental Health First Aid USA (YMHFA-USA)
A youth version of MHFA was specifically developed for adolescents aged 12–18 years with mental health problems who are at heightened risk for more serious long-term problems (Kelly et al., 2011). Like MHFA, YMHFA is taught by a certified instructor and generally implemented in two, 4-hr sessions. The targeted trainees for YMHFA range from laypeople who are unfamiliar with mental health concepts to mental health professionals who do not have experience with youth. Teaching methods using the ALGEE framework combine didactic instruction with experiential exercises, media clips, case studies, and group reflection. At the end of the training, participants are expected to have basic knowledge about how to recognize mental health problems among adolescents as well as feeling more confident about how to provide help to an adolescent in crisis (Maryland Department of Health and Mental Hygiene, 2012).
To date, there is minimal published research evaluating YMHFA including one study in North East England, two studies conducted in Australia, and one exploring the adaptation of YMHFA-USA specifically. Findings from a two-stage study set in North East England showed that participants’ confidence in helping youth as well as knowledge of mental health increased as a result of participation in YMHFA (Borrill & Kuczynska, 2011). A low response rate prevented these researchers from demonstrating effects of participation in the training, as intended, 3–6 months after participation in YMHFA. Studies of YMHFA in Australia have associated participation with improvement in mental health literacy, decreased stigma, increased confidence, and use of the YMHFA action plan among a sample of Australian adults (Kelly et al., 2011). These results are similar to the findings of Jorm, Kitchener, Sawyer, Scales, and Cvetkovski (2010), who determined that high school teachers who received YMHFA training exhibited increased mental health knowledge, beliefs about treatment more consistent with mental health professionals, reduction in personal and perceived stigma, increased intention to help, and confidence in providing help to students, compared to those who were not trained. Finally, recent research examining YMHFA uses a version adapted for the United States in 2011 (YMHFA-USA). The study, using social service employees, showed that participation in YMHFA-USA increased confidence in, the likelihood of, and comfort with helping a young person in emotional distress or crisis (Aakre, Lucksted, & Browning-McNee, 2016).
Study Objectives and Hypotheses
Adolescent mental health has been identified as a public health priority warranting early intervention and mental health promotion approaches (e.g., Healthy People 2020, 2012). There is growing recognition of YMHFA in the United States, evidenced by increased implementation of the training in various settings such as social service agencies (Mental Health First Aid Training for DJS Staff, 2015; MHFA-USA, 2015) and schools (SAMHSA, 2015). However, given that most of the current evidence supporting YMHFA is based on international studies, there is a need for more empirical research to substantiate the use of YMHFA in the United States. Further, although social workers are employed across a variety of professional specializations, both clinical and nonclinical social workers potentially encounter youth in their practice. As part of the preparation to work across service settings, it is important, then, that social work students have foundational training on providing initial assistance to youth with mental health needs. Subsequently, the objective of the current research is to examine the effects of the YMHFA-USA training in a sample of master’s level social work (MSW) students. To our knowledge, this is the first quasi-experimental study of its kind in the United States as well as the first among social work students.
Outcomes examined in this study were selected to assess the influence of YMHFA-USA, based on previous research, and rooted in the framework of the UTB (Anthony et al., 2015). These outcomes addressed determinants of behavioral intention as well as factors affecting actual promotion behavior including (1) attitude toward and beliefs about carrying out ALGEE actions, (2) individual’s estimate of the social pressure to engage or not engage in targeted ALGEE behaviors (subjective norms), (3) confidence in aptitude to learn and perform targeted ALGEE actions (self-efficacy), and (4) mental health specific knowledge and skills. Subsequently, hypotheses for the study are (1) MSW students trained in YMHFA-USA will significantly improve their attitudes, beliefs, subjective norms (e.g., stigma), self-efficacy, and knowledge about mental health from pretest to 5-month follow-up, and (2) MSW students trained in YMHFA-USA will show greater improvements in attitudes, beliefs, subjective norms (e.g., stigma), self-efficacy, and knowledge about mental health than MSW students who were not trained in YMHFA-USA at 5-month follow-up.
Method
Study Design and Sample
This study uses a nonrandomized, quasi-experimental comparison group design to observe whether changes in the intervention (i.e., YMHFA-USA training) group on identified outcomes differed significantly from those who did not receive the training (Shaddish, Cook, & Campbell, 2002). This research was conducted at a school of social work in a large mid-Atlantic university during the 2014–2015 academic year and with the approval of the institutional review board. The intervention group was assessed at baseline (pretest), 2 weeks following the intervention (posttest), and 5 months following the intervention (follow-up). The comparison group was assessed at the same baseline and 5 months follow-up time frame as the intervention group.
Sample Selection and Groups
Researchers used a list of students furnished by the school’s Office of Field Education to identify full- and part-time MSW students enrolled in field education that were placed in internships with youth (N = 194). This list served as the sampling frame for the study, and purposive samples of both intervention and comparison group students were drawn from this list. Eligibility criteria for the intervention group included students who were enrolled in field education, assigned to a field placement that served youth aged 12–18 years, had a field instructor certified in YMHFA-USA that had scheduled a YMHFA-USA training, and were willing to participate in the study. Eligibility criteria for the comparison group were similar to the intervention group, as these students were enrolled in field education, assigned to a field placement that serves youth aged 12–18 years, and were willing to participate in the study. However, these students did not have a field instructor certified in YMHFA-USA (see Figure 1 for CONSORT diagram). Comparison group students were not trained in YMHFA-USA during the project but were referred to the Mental Health Association (MHA) of Maryland for training opportunities at the completion of the study period. Students in both intervention and comparison groups were excluded from the study if they were taking classes with the study's principal investigator at the time of study inception.
Flow of participants through Youth Mental Health First Aid study.
All data were collected online. Eligible intervention and comparison group students were sent an e-mail to learn more about the study including information about the online pretest, posttest (intervention group only), and follow-up surveys. Intervention group students were sent e-mails 2 weeks prior to their scheduled YMHFA-USA training. The e-mail also included a weblink for the online informed consent form. Once students read the online informed consent and agreed to participate in the study, they were directed to the online pretest survey to be completed in a location of their choice. Of 67 students who received the training, 39 (58%) agreed to participate in the study and formed the intervention group. Of students meeting eligibility for the comparison group (127), 34 students (27%) consented to be a part of the study.
All pretest surveys were completed by mid-November 2014 for the intervention group and early December 2014 for the comparison group. After the trainings were completed, intervention group students who had consented to participate in the study and had completed a pretest were sent an e-mail asking them to link to the online survey to complete the posttest survey. All posttest surveys were completed by early December 2014. A 5-month follow-up survey link was e-mailed to both the intervention and comparison groups in April 2015. Based on the academic year schedule, follow-up surveys were administered using a 5-month modified time line. All 5-month surveys were completed by May 2015. Students in both the intervention and comparison groups were given an online US $10 gift card for each completed survey.
Demographic information for both groups is summarized in Table 1. Students in the intervention and comparison groups are majority female (94%) and Caucasian (62% intervention; 56% comparison). Both intervention and comparison groups were similar in age (M = 28, intervention; M = 26, comparison). Sixty-seven percent of students assigned to the intervention group were in the advanced year of the MSW program, whereas 53% of students assigned to the comparison group were in the foundation year of the curriculum. Thirty-one percent of the intervention group compared to 53% of comparison group students had previous mental health training. Finally, the majority of both groups (>85%) had previous experience working with youth. No statistically significant differences between groups on any of the sample characteristics were observed, based on bivariate analyses.
Student and Sample Demographics.
N = 73.
YMHFA Intervention
All field instructors administering the YMHFA-USA training were certified through the MHA of Maryland. Following the training, field instructors administered the manualized YMHFA-USA training to students whom they supervised at the field instruction placement agency in either one 8-hr or two 4-hr sessions. The first 4-hr session of YMHFA-USA covers the topics of general mental health, adolescent development, signs and symptoms of mental health problems, self-injury, risk and protective factors, and suicidality assessment. The second 4-hr session introduces skills and allows time for participants to practice the ALGEE model as well as talking about mental health crises. Lessons are taught using examples of common mental health problems experienced by adolescents including depression, trauma, anxiety, eating disorders, psychosis, substance use and abuse, and attention and disruption disorders. Students were grouped with a minimum of nine other trainees for group activities. Although no incentive was given for participation in the training, students received the training at no cost. Field instructors were unaware of which students consented to participate in the evaluation study.
Measures
The survey instrument, Mental Health Beliefs and Literacy Scale, was developed by researchers at Georgetown University in collaboration with the National Council for Behavioral Health and is used with permission (Anthony et al., 2015). The instrument, grounded in the UTB framework (e.g., Jaccard et al., 2002), evaluates MHFA (both adult and youth trainings), and included a pre- and posttest as well as 3-month and 6-month follow-ups. For this study, we used the 6-month follow-up survey at 5 months. The survey was designed to assess the implementation of the training (including the utility of structure and content), attitudes about mental health first-aid principles (i.e., ALGEE), knowledge of mental health, the motivation for and confidence in carrying out action steps, intention to use skills taught during training, and actual behaviors. Respondents were also provided open-ended questions to describe the quality of their training experience; those results are not included in the current study.
For this research, 10 subscales from the survey, specifically reflecting immediate and longer term influences of the training, were used in the analyses to address our study hypotheses. Both of the personal stigma scales and the behavioral intentions scale showed unacceptable reliability in our sample (Cronbach α < .60) and thus were not included in the analysis. The subscales are grouped by construct below as delineated by the survey instrument. Although we use the term ALGEE to describe the scales and outcomes, the questions from each subscale were principles of UTB and related to mental health literacy so would be recognizable by laypeople as well as our comparison group.
Attitudes and beliefs about performing ALGEE actions
The first construct was attitudes toward and beliefs about carrying out ALGEE actions. Specifically, attitudes incorporated overall favorableness toward the behavior including (1) perceptions of the level of difficulty, reward, and usefulness of ALGEE actions and (2) beliefs that implementing ALGEE will lead to positive results. Question prompts for difficulty, reward, and usefulness were the same (e.g., I believe that for me talking with someone experiencing a mental health problem about his/her problem is…). Response sets for each of the 8 items were (1) not at all difficult to (5) extremely difficult, (1) not at all rewarding to (5) extremely rewarding, and (1) not at all useful to (5) extremely useful. For the difficulty scale, lower scores mean less difficulty with performing the action. Current sample reliability was .82 for difficulty, .87 for reward, and .91 for usefulness. Four questions assessed whether respondents believed that implementing ALGEE would lead to positive results. This scale included questions such as, “If I listen to a person without judging, it will help that person talk to me about his/her mental health.” The response set ranged from (1) not at all likely to (5) extremely likely. Reliability for the belief scale in this sample was .63. Mean scores were computed for all variables, and possible scores ranged from 1 to 5.
Subjective norms
The second construct assessed subjective norms or an individual’s estimate of the social pressure to engage or not engage in the targeted ALGEE behaviors. This construct included individual perceptions of behavior as well as how they believe others would like them to behave. Three variables from the survey were used to assess this construct. The first assessed stigmatizing beliefs. That is, the variable assessed what people, important to the individual, believed about others with mental health problems. A sample item is “people important to me believe that people with mental health problems are seeking attention.” Eight items were used to assess this variable and the response set ranged from (1) do not agree at all to (5) strongly agree. Three items were recoded, and lower scores indicate less negative perception of others’ stigmatizing beliefs. Reliability in the current sample was .81. The second variable examined personal beliefs about performing ALGEE actions. A sample item is “I believe I should listen compassionately to someone about their mental health problem(s).” Five items were used to assess this variable. The response set ranged from (1) do not agree at all to (5) strongly agree. Reliability was .77 in the current sample. The third variable examined beliefs about how others, who may be important to the individual, may want them to behave (e.g., other people important to me believe I should perform ALGEE). Five items were used to assess this variable. A sample item is “People important to me believe I should give practical resources to someone experiencing a mental health problem.” The response set ranged from (1) do not agree at all to (5) strongly agree. Reliability was .88 in this sample. Mean scores were computed for all variables, and possible scores ranged from 1 to 5.
Self-efficacy
The third construct assessed confidence in aptitude to learn and perform targeted ALGEE actions. The first variable represented a person’s confidence in their ability to carry out ALGEE actions. This variable included 14 items with questions such as “I am confident I can refer a person with a mental health problem to appropriate help.” The response set ranged from (1) do not agree at all to (5) strongly agree. Reliability in the current sample was .88. The second variable measured controllability or how much a person feels they have control over the behavior (e.g., likelihood of performing ALGEE actions if prepared). Six items represented this variable; a sample item from this scale is “I will approach someone with a mental health problem(s) if I feel I have the knowledge to talk to them about their problem.” The response set ranged from (1) not at all likely to (5) extremely likely. Reliability in the current sample was .86. Based on the response sets, mean scores were computed for all variables and ranged from 1 to 5.
Knowledge
The fourth construct was knowledge of common mental health concepts. Sixteen statements were used to assess knowledge. A sample statement is “Schizophrenia is one of the most common mental health disorders in the United States.” Respondents described their level of agreement with each item (agree, do not agree, and don’t know). Correct responses were summed, and then the mean number of correct responses across the sample was calculated. Higher scores indicate a greater number of correct responses, and scores ranged from 0 to 16 correct responses.
Statistical Methods and Analysis
Repeated measures analysis of variance (ANOVA) was used to examine within-group changes among intervention group participants across the three data collection time points: pretest, posttest, and 5-month follow-up. An a priori power analyses performed in G*Power indicated a sample size of 22 was required to detect a small effect size of .25 at 80% power (Faul, Erdfelder, Buchner, & Lang, 2009; Faul, Erdfelder, Lang, & Buchner, 2007). Analysis of covariance (ANCOVA) was used to assess differences between intervention and comparison groups on all study variables at 5-month follow-up, controlling for the pretest scores. Statistical assumptions associated with ANCOVA and repeated measures ANOVA were assessed for each model and generally met. Any violations are discussed below with results. For ANCOVA, the pretest score was the only covariate examined due to sample size. All hypotheses were tested as two-tailed using α = .10. All post hoc analyses were performed with a Bonferroni adjustment (α = .03). Appropriate effect sizes and 95% confidence intervals (CIs) were used in assessing practical significance of findings. Partial η2 is provided as an effect size of the full models and is interpreted as the percentage of variance explained by the model. Based on Cohen’s (1988) suggestions, .01 is a small effect, .9 is a moderate effect, and .25 is a large effect. Cohen’s d is provided as an effect size for post hoc analyses; it is interpreted as the standardized difference between the mean scores. Based on Cohen’s suggestions, .2 is a small effect, .5 is a moderate effect, and .8 is a large effect. All quantitative data analyses were conducted using SPSS, Version 22 (IBM Corp, 2013).
Results
Hypothesis 1: Improvement in Attitudes, Beliefs, Subjective Norms, Self-Efficacy, and Knowledge Over Time for the Intervention Group
A series of repeated measures ANOVAs were conducted to assess statistically significant change over time in all study variables within the intervention group only. The findings (e.g., means, standard deviations) are presented in Table 2. Overall, 11–14 cases (28.2–35.9%) were missing at T3. At this time point, missing data were determined to be missing completely at random (MCAR) based on Little’s (1988) test of MCAR. SPSS uses listwise deletion for ANOVA tests, which utilizes only observed data and is determined to be appropriate for data that are MCAR (Allison, 2001). Several variables violated statistical tests of normality, but all skewness and kurtosis values were very low (<1). The Greenhouse-Geisser corrected statistic was reported for any analysis that violated sphericity.
Descriptive Statistics for Repeated Measures Analysis of Change Over Time for the Intervention Group.
Note.
aα = .03.
Attitudes and beliefs about performing ALGEE actions
There was a statistically significant effect of time on difficulty, signifying changes in scores across time points, F(2, 48) = 14.12, p < .001. A large effect size (partial η2) indicates that 37% of variance in scores is attributable to changes over time. Bonferroni-adjusted (α = .03) post hoc analyses revealed a statistically significant decrease in difficulty between T1 and T2 (p = .02; 95% CIΔ [0.05, 0.74]; Cohen’s d = .56), and between T1 and T3 (p < .001; 95% CIΔ [0.37, 0.89]; Cohen’s d = .96), but not between T2 and T3 (p = .18), indicating a quadratic trend in the data. Thus, students reported less difficulty in implementing ALGEE actions after receiving the YMHFA-USA training, and this was sustained at 5-month follow-up.
The repeated measures ANOVA yielded statistically significant results for mean changes in perceptions of personal reward or satisfaction from implementing ALGEE over time, F(2, 54) = 3.24, p = .047. Change over time yielded a modest effect size (partial η2) of 11% explained variance. Bonferroni-adjusted (α = .03) post hoc analyses revealed a statistically significant increase between T1 and T3 (p = .01; 95% CIΔ [−060, −0.06]; Cohen’s d = .51). Examination of tests of shape indicates a linear trend, and no statistically significant changes were observed between T1 and T2 (p = 1.00) or between T2 and T3 (p = .51). Thus, students in this sample perceived that implementing ALGEE would be rewarding over the course of the 5-month time period. No statistically significant changes in scores were observed over time for usefulness of implementing ALGEE based on the repeated measures ANOVA, F(1.3, 31.91) = 1.55, p = .22; results suggest students’ perception about the usefulness of ALGEE was high at T1 and remained high over the course of the study.
There was a statistically significant effect of time on belief that implementing ALGEE would lead to a positive result, F(1.54, 41.70) = 27.74, p < .001. A large effect size (partial η2) indicated that approximately 51% of variance of scores was associated with change over time. Bonferroni-adjusted (α = .03) post hoc analyses revealed a statistically significant increase in beliefs between T1 and T2 (p = .001; 95% CIΔ [−0.80, −0.20]; Cohen’s d = 1.39) and between T1 and T3 (p < .001; 95% CIΔ [−0.90, −0.48]; Cohen’s d = .85), but not between T2 and T3 (p = .085). Tests of shape indicate a linear trend. Accordingly, with training, there appeared to be an increase in student belief that implementing ALGEE would result in more beneficial outcomes.
Subjective norms
Students’ perception of what important people in their lives believe about others with mental health problems did not change significantly over the course of the time. Although there was a slight decrease from T1 to T3, no statistically significant changes in scores were observed over time based on the repeated measures ANOVA, F(2, 50) = 1.81, p = .174. The repeated measures ANOVA for students’ perspectives on whether they should implement ALGEE yielded statistically significant results for mean changes over time, F(1.4, 36.58) = 5.36, p = .02. The partial η2 of 17% indicated a modest effect for explained variance. Bonferroni-adjusted (α = .03) post hoc analyses revealed a statistically significant increase in scores between T1and T2 (p = .04; 95% CIΔ [−0.40, −0.01]; Cohen’s d = .42), but not between T1 and T3 (p = .08) or between T2 and T3 (p = 1.00). Tests of shape indicate an overall quadratic trend. No statistically significant changes in students’ perspectives that people believe they should carry out ALGEE actions were observed over time based on the repeated measures ANOVA, F(2, 52) = .64, p = .51.
Self-efficacy
The repeated measures ANOVA yielded statistically significant results for student’s confidence in their ability to carry out ALGEE actions over time, F(2, 44) = 22.74, p < .001. A large effect size (partial η2) suggested that nearly 60% of variance in scores could be attributed to changes over time. Bonferroni-adjusted (α = .03) post hoc analyses revealed a statistically significant increase between T1 and T2 (p < .001; 95% CIΔ [−0.91, −0.28]; Cohen’s d = 1.20) as well as T1 and T3 (p < .001; 95% CIΔ [−1.01, −0.37]; Cohen’s d = .98), but not between T2 and T3 (p = .81). Tests of shape indicate an overall quadratic trend. No statistically significant changes in scores were observed over time for controllability (e.g., students feeling that if prepared they would be able to carry out ALGEE actions) based on the repeated measures ANOVA, F(1.3, 31.18) = 1.34, p = .27.
Knowledge
There was a statistically significant effect of time on knowledge, F(2, 44) = 9.25, p < .001, indicating greater change in student knowledge about mental health concepts post YMHFA-USA training. Changes over time were associated with approximately 30% explained variance in scores (partial η2). Tests of shape indicate a linear trend. Bonferroni-adjusted (α = .03) post hoc analyses revealed a statistically significant increase in knowledge between T1 and T2 (p = .01; 95% CIΔ [−3.29, −0.36]; Cohen’s d = 1.07) and between T1 and T3 (p = .004; 95% CIΔ [−4.76, −0.80]; Cohen’s d = .48), but not between T2 and T3 (p = .42). Thus, students increase in knowledge right after the training remained consistent at 5-month follow-up.
Hypothesis 2: Difference in Attitudes, Beliefs, Subjective Norms, Self-Efficacy, and Knowledge Between the Intervention and Comparison Group
A series of one-way ANCOVA’s were conducted to determine statistically significant differences between the intervention and comparison group on each of the outcomes at the 5-month follow-up (T3), controlling for their respective pretest scores. Overall, 21–25 cases (28.8–34.2%) were missing data at T3. At this time point, missing data were determined to be MCAR based on Little’s (1988) test of MCAR. SPSS uses listwise deletion for ANCOVA tests, which utilizes only observed data and is determined to be appropriate for data that are MCAR (Allison, 2001). No statistically significant differences in pretest scores were observed between the intervention and comparison groups on any of the outcomes. The sample size, means, and standard deviations for T1 and T3 by group are reported in Table 3. Partial η2, a commonly reported measure of effect size for ANOVA tests (Bakeman, 2006; Richardson, 2011), is also reported and indicates the percentage of explained variance attributable to group membership. Several variables violated statistical tests of normality, but all skewness and kurtosis values were very low (<1). Homogeneity of regression was violated for the analysis of difficulty with performing ALGEE actions and for perceptions of personal reward or satisfaction from implementing ALGEE; thus, in these analyses, the covariate was removed, and a one-way ANOVA was used. The assumption of homogeneity of variance was violated for perceptions of personal reward or satisfaction from implementing ALGEE, belief that implementing ALGEE would lead to a positive result and knowledge. However, in all cases, the ratio of highest to lowest variance was <4, and any impact on the results was likely negligible with the equivalent sample sizes.
Descriptive Statistics for Original Means/SD for ANCOVA or ANOVA Tests of Difference Between the Intervention and Comparison Group.
Note.
*p < .05. **p < .01.
The one-way ANOVA revealed significantly less difficulty with performing ALGEE actions, F(1, 48) = 4.80, p = .03, among intervention group students compared to their counterparts who were not trained (95% CIΔ [0.05, 0.63]). A small effect size (partial η2) indicated that approximately 9% of variance in scores was associated with group membership. The ANCOVA tests showed statistically significant differences between groups at 5-month follow-up on three more outcomes. Findings showed higher beliefs that performing YMHFA activities will result in positive outcomes, F(1, 49) = 27.89, p < .001, among intervention group participants compared to those who did not receive the training (95% CIΔ [−.90, −0.41]). The results indicated a large effect size (partial η2) with 36% of variance explained by group membership. Additionally, there was a statistically significant difference in confidence in implementing ALGEE actions, F(1, 42) = 10.54, p = .002, between the intervention and comparison group (95% CIΔ [−0.67, −0.16]). A moderate effect size (partial η2) of approximately 20% explained variance was noted. Students receiving the intervention reported significantly greater confidence in carrying out these actions at 5-month follow-up. Finally, knowledge of mental health concepts at 5-month follow-up, F(1, 44) = 17.41, p < .001, was significantly greater in the intervention group compared to the comparison group (95% CIΔ [−2.73, −0.95]). Results suggest a large effect size (partial η2) with 28% explained variance in scores attributable to group differences. No significant differences were observed between groups at 5-month follow-up on any of the other variables examined.
Discussion and Applications to Practice
Findings from this study show potential benefits of the YMHFA-USA training in a small sample of graduate social work students. Minimal research has examined the effect of the training in the United States and particularly among social work students. As social workers are the largest providers of mental health services in the United States (NASW, 2016) and nonclinical social workers may encounter youth in their practice or program settings, it is important that social work students have, at a minimum, foundational training on youth mental health.
In regard to our first hypothesis, study findings showed that similar to previous literature, students trained in YHMFA-USA exhibited greater confidence (Aakre et al., 2016; Kelly et al., 2011) and knowledge of mental health concepts (Kelly et al., 2011) over time. Further, intervention group students reported less difficulty and greater reward in carrying out ALGEE actions, and higher belief that implementing ALGEE would produce positive benefits 5 months following the training. We also found some support for our second hypothesis. That is, trained students exhibited greater confidence, knowledge of mental health concepts (Jorm, Kitchener, Sawyer, et al., 2010), as well as less difficulty in carrying out ALGEE actions and higher belief that implementing ALGEE would produce positive benefits compared to students who were not trained. Thus, although MSW students in the United States take foundation and advanced courses in theory, practice, and research related to mental health, behavioral health, crisis intervention, and stigma/social justice, our study showed that the additional training helped intervention group students to build their knowledge and hone their skills. They felt more adept in screening and intervening across time than their peers. Summarily, the YMHFA-USA training appears to benefit graduate social work in that it can provide students with a specific framework through which to begin to support youth with mental health concerns. This seems true both for students who have had previous mental health training as well as for students who did not have a background in mental health. Findings provide some initial support for the use of YMHFA-USA training with MSW students and extend the knowledge base for the effectiveness of the training.
All social work students participate in a field internship, in which they interact with the community in settings such as schools, hospitals, and community-based organizations. In these placements, social work students may have formal clients with specific mental health needs that the student will be expected to address; however, students lacking training in mental health or students early in their social work education careers may not have had sufficient experience to prepare them to work with these clients. Additionally, social work students who do not take specialized courses in child welfare, health, behavioral health, or mental health, such as macro or gerontology focused students, may not learn basic youth mental health assessment and referral skills, in spite of often working in community settings in which they may encounter youth. Although social work programs provide students with general knowledge and skills, field placements typically begin at the start of the social work program, and even those who will take coursework on youth mental health may not have sufficient course participation at the time of their first field placements to guide their interactions with youth. This is especially problematic due to the incidence, prevalence, and high-risk nature of youth mental health concerns. YMHFA-USA helps to address this gap by providing a brief, foundational training that can provide social work students with specific knowledge and skills to assess and refer youth for mental health treatment as well as to foster the confidence to use these skills as necessary. The training gives field instructors another tool to use as part of their own orientation or even ongoing supervision with social work interns to ensure they have the basic mental health knowledge and confidence to work with youth clients. Further, the training could provide a foundational knowledge of youth mental health for all social workers to draw on as they develop their professional social work careers. Finally, social workers who may transition from other areas of the field, for example, macro settings to direct practice with youth, may benefit from participation in the training as they begin to build skills to work with a new population.
There are many practical factors to consider with the implementation of the YMHFA-USA training with social work students. The first major consideration is when the training should be provided. As college students, bachelors of social work students (BSW) may be excellent candidates for participation in the YMHFA-USA training. However, given the potential breadth of student experiences, and the varying backgrounds from which students enter an MSW program, it is possible that foundation year may also be a good time to implement the training on an MSW level. A second consideration is that YMHFA-USA could be implemented with targeted groups of MSW students, such as those who have field placements in which they work directly with youth, or students who have less clinical backgrounds (such as macro students) and who are likely to interact with youth but receive less training in mental health practice. For MSW students placed in school systems or other places with a large concentration of youth (e.g., community-based organizations with youth programs), it could be beneficial to consider training these students to become trainers of YMHFA-USA. These students would then be able to train colleagues at their placements, such as teachers and other school staff, thus bringing YMHFA-USA skills to a broader audience. The actual timing of administering YMHFA-USA is a third important consideration. This program could be incorporated into the orientation of an MSW program, within the field education orientation or provided as a pre-MSW summer program to those who have no or minimal previous mental health experience. YMHFA-USA could also be incorporated into existing coursework such as foundation practice or human behavior courses.
As with all pilot studies, conducted in “real-world” settings (e.g., Proctor et al., 2009), there are limitations to consider in interpreting study findings. Although this research used a quasi-experimental design, thus reducing some potential threats to internal validity (e.g., history, maturation), the threat of selection bias needs to be acknowledged. That is, random assignment to groups was not possible in this study as intervention group students had to have a field instructor trained in YMHFA. The absence of randomization increases the potential for selection effects between conditions on observed and unobserved factors and decreases the ability to fully control for all factors that could have contributed to outcome differences between groups. Causality, also, cannot be established. Although not applied in this pilot study, future research could utilize alternative statistical approaches such as regression discontinuity design (Shadish, 2011), which may improve the comparability of comparison groups in quasi-experimental designs. However, there were no statistically significant differences by group on demographic or pretest measures, which provides some support for findings on observed group differences. The data were also self-report which increases the possibility of social desirability bias. Further, the study used purposive sampling and was only conducted in one school of social work in the mid-Atlantic region, which limits generalizability of the findings to other schools of social work nationally. Future studies could utilize random sampling to be able to generalize beyond schools with similar student bodies.
Additionally, although there is initial research supporting the measures used to assess YMHFA-USA, the personal stigma scale did not meet reliability criteria in our sample. Reduction in personal stigma was a key factor in the Australian studies (e.g., Kelly et al., 2011), and stigmatizing attitudes have been related to help seeking in much mental health research. Improvements to measures that assess the effect of this training should be addressed, such as ensuring reliability of a personal stigma scale. Further, the study did not use an imputation method, but rather listwise deletion, which could have reduced the power to detect changes, particularly with the ANCOVA test. We also recommend future research that builds on our initial study of YMHFA-USA by replicating this study with a larger sample but varying the population and timing of the training. That is, it would be beneficial to compare the impact of YMHFA-USA between BSW students, foundation MSW students, advanced MSW students, and students with and without mental health backgrounds. Similarly, we recommend research on the timing of YMHFA-USA training, comparing administering the training as part of orientation to implementing it at other times, like as its own short course, or integrated into an existing course. Effects of YMHFA-USA should be measured over a longer period of time to determine whether refresher modules should be developed. Ultimately, research should be conducted to examine the distal effects of YMHFA-USA, such as changes in the actions and behavior of the youth being served in response to trained participants. However, to accurately assess these results, a large-scale implementation of YMHFA-USA will be necessary.
Notwithstanding, there are several strengths of the study. We used a comparison group that allows us to begin to examine group differences and provides some support for training effectiveness. The study also assessed a variety of outcomes related to YMHFA-USA, and data were collected over three time points, which help us understand the effect of the training over time. Importantly, this is one of the first studies, to our knowledge, to evaluate YMHFA-USA using a quasi-experimental approach as well as the first study to explore the effectiveness of this training with graduate social work students, adding to the developing research base for the training intervention.
As one of the first implementation studies of YMHFA-USA, this study provides promising early findings of the value of the training to MSW students in social work education. There has been a glaring gap of standardized trainings that provide a specific set of skills aimed at allowing the layperson to recognize and provide support for adolescents who may be developing a mental health problem or may be in crisis. Courses within MSW programs such as psychopathology for social work practice, assessment of mental and behavioral health, and theories of human development are excellent benchmarks to begin helping students to use their knowledge, attitudes, and skills in assessment, diagnosis, and intervention. However, our study indicated that students who completed these standard courses in our MSW program benefited from the YMHFA-USA training. They were more confident in their skills and appreciated a specific manualized way of screening and intervening with youth. We believe continued refinement and use of YMHFA-USA within social work curriculum will only benefit our client base in many clinical and nonclinical settings and our profession as a whole.
Footnotes
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The first four authors received financial support for the research, authorship, and/or publication of this article through a University of Maryland School of Social Work Teaching Scholars Award.