A first-level evaluation of a school-based family programme for adolescent social,emotional and behavioural difficulties

Abstract

This study is a first-level evaluation of an intervention targeted at adolescents with social, emotional and behavioural difficulties in Irish post-primary schools. It is a combined implementation of the Working Things Out adolescent programme and the Parents Plus Adolescent Programme (WTOPPAP). Overall, 47 parents and their children (mean age: 13.81 years) took part in the study. The study used a repeated measures design to assess change at pre- and post-intervention and 5-month follow-up using the Strengths and Difficulties Questionnaire, McMaster General Functioning Scale, Goal Attainment (parent- and adolescent-rated), Parent Stress Scale and Kansas Parenting Satisfaction Scale (parent-rated) as assessment measures. This study found that parent-rated child total difficulties and adolescent-rated emotional difficulties significantly improved from pre-test to 5-month follow-up. Parent- and adolescent-rated goals, and parent-rated child conduct difficulties, parental stress and satisfaction with parenting also significantly improved from pre- to post-test. These gains were largely maintained at 5-month follow-up. These findings indicate that the WTOPPAP may be an effective intervention for adolescents with emotional and behavioural difficulties and their parents. It was demonstrated that a manualised family intervention could be effectively rolled out at a number of school locations, with delivery and evaluation being conducted by school staff. Further implications are also discussed.

Keywords

Parents Plus family intervention school-based intervention social emotional and behavioural difficulties adolescent difficulties

Introduction

Social, emotional and behavioural difficulties in adolescents

Social, emotional and behavioural difficulties (SEBD) are a relatively common problem for adolescents, and the prevalence rates have been increasing (Banks & McCoy, 2011; Collishaw, Maughan, Goodman, & Pickles, 2004). SEBD can negatively impact their mental and physical health, social relationships, and engagement in school and educational attainment and increase the risk of poor outcomes in later life (e.g. A. Goodman, Joyce, & Smith, 2011; Ingul, Klöckner, Silverman, & Nordahl, 2012; Reijntjes, Kamphuis, Prinzie, & Telch, 2010; Wolitzky-Taylor, Bobova, Zinbarg, Mineka, & Craske, 2012). They can also lead to disrupted routines, financial strains and caregiver burden in families and schools as young people with SEBD require more mental health and academic support than their peers (Brannan & Heflinger, 2001; Suhrcke, Pillas, & Selai, 2008; Turnbull & Turnbull, 2001). The development of SEBD may be best understood from a bio-psycho-social model (see Cooper & Jacobs, 2011). There is a considerable body of research evidence indicating that negative parenting behaviours, for example, can contribute to SEBD in adolescents (Hughes & Ensor, 2007; Hurth-Bocks & Hughes, 2008).

Rationale for family interventions

A wide range of interventions that aim to improve young people’s SEBD have been developed. These interventions can involve adolescents alone or can take a family-based approach where only parents or both parents and children are the focus of the intervention (e.g. Baruch, Vrouva, & Wells, 2011; Gillham et al., 2006; Schoenfeld & Janney, 2008). The most up-to-date empirically supported interventions for improving adolescent mental health problems include cognitive behaviour therapy (CBT) and parent management training (PMT) where positive parenting behaviours (e.g. responsiveness, good communication, rule setting and monitoring) are strengthened. However, CBT may be most effective at improving internalising problems (Chorpita et al., 2011), while PMT has consistently shown to improve behaviour difficulties with gains maintained at a 1-year follow-up (see Carr, 2014). Although not the focus during intervention, PMT has also shown to improve parental functioning and mental health (Barlow, Smailagic, Huband, Roloff, & Bennett, 2012).

Research suggests that family interventions targeting parents and children, rather than children or parents alone, may be more effective at improving SEBD, but in particular behavioural problems (see Carr, 2014). Carr’s (2014) review of meta-analyses, systematic literature reviews and controlled trials on the effectiveness of family interventions on youth mental health difficulties and disorders indicates that such interventions are more effective at improving emotional and behavioural difficulties than treatment as usual with young people (e.g. Kaslow, Broth, Smith, & Collins, 2012; Woolfenden, Williams, & Peat, 2002). Moreover, the review identified two studies which showed that the combination of PMT and CBT for young people with SEBD is more effective than either treatment alone (Kazdin, 2010; Webster-Stratton & Reid, 2010).

Rationale for delivering family interventions in the school setting

Despite the advantages of family interventions, the majority are conducted in clinical settings (Shriver & Allen, 2010), and this can limit both the generalisability of such programmes to ‘real-world’ settings and the access to the intervention (Michelson, Davenport, Dretzke, Barlow, & Day, 2013). Young people are, in general, hesitant to seek professional support when encountering difficulties (Rickwood, Deane, & Wilson, 2007), and families can be reluctant to seek support in clinical settings because of the stigma and expense associated with them (Hoganbruen, Clauss-Ehlers, Nelson, & Faenza, 2003). Thus, the provision of easily accessible, familiar and non-threatening interventions for adolescents with SEBD is crucial, and schools are increasingly being considered the most natural and effective setting to facilitate such support (Fantuzzo, McWayne, & Bulotsky, 2003; Lean & Colucci, 2010; McLennan, Reckord, & Clarke, 2008).

Basing mental health interventions in schools can reduce the impact of issues such as cost and transportation to families (Gulliver, Griffiths, & Christensen, 2010). It can allow for greater sharing of information among health professionals, schools and families (Van Acker & Mayer, 2009). As well as being less stigmatising for families, it could help demystify the institution for those parents with memories of school as an unwelcoming place (LaBahn, 1995) and, in turn, increase parent involvement (PI). Increasing PI in school is widely known to have benefits for teachers, parents and students including improving students’ SEBD (Christenson & Havsy, 2004; Hornby & Witte, 2010; Patrikakou & Weissberg, 2007).

There is a relatively large body of research dealing with school-based intervention programmes for students’ externalising problems (Barnes, Smith, & Miller, 2014; Wilson & Lipsey, 2007). For example, a meta-analysis conducted by Wilson and Lipsey (2007) indicated that such programmes can be effective at reducing problem behaviours in young people, with the most effective programmes being CBT- and counselling-based. There is also a growing number of reviews of school-based emotional and social learning interventions for internalising problems (Cooper & Jacobs, 2011; Durlak, Weissberg, Dymnicki, Taylor, & Schellinger, 2011; Sancassiani et al., 2015). A meta-analysis by Durlak et al. (2011) revealed that these interventions yield significant positive effects on young people’s social-emotional competencies and internalising problems. Cooper and Jacobs’ (2011) review identified the CBT-based FRIENDS programme (often delivered by teachers) as highly effective at improving the anxiety and self-esteem of children.

However, family interventions with PMT alone, or in combination with a child intervention, are rarely delivered in school settings (Cooper & Jacobs, 2011; Valdez, Carlson, & Zanger, 2005). Valdez et al. (2005) identified eight studies between 1980 and 2002 that evaluated family interventions delivered in schools. They reported the efficacy of the majority of these in improving young people’s SEBD at home and in school. While Cooper and Jacobs (2011) did not identify any PMT provided in schools in their review, they referenced that many parents attending an Incredible Years Parenting Programme reported a concern that their child’s teachers were not involved and felt that they could benefit from the intervention they themselves received (Kelleher & McGilloway, 2006). Teachers may be well placed to facilitate parenting programmes in schools, as they too could apply skills in managing SEBD, which can lead to higher levels of student engagement and, in turn, reduce the risk of SEBD (Buyse, Verschueren, Doumen, Van Damme, & Maes, 2008; Cooper & McIntyre, 1996; Corkum, McKinnon, & Mullane, 2005; LaRusso, Romer, & Selman, 2008).

The intervention

The National Behaviour Support Service (NBSS) works in partnership with second-level schools in Ireland providing whole-school support at three levels to address the SEBD and academic needs of students. The first tier is the universal level that targets all pupils with basic positive behaviour support, which is generally sufficient for 80% of the population. The second tier focuses on students who are at-risk of SEBD (~15%) by supplementing mainstream education with behavioural support and learning support teams. Finally, the third tier targets high-risk students with specific challenges (~5%) who are managed within the school with support from external agencies and guided by Individual Education Plans. The NBSS prioritised increasing PI as a means of making their interventions more effective and sought to implement the combined version of the existing Parents Plus Adolescents Programme (PPAP; Sharry & Fitzpatrick, 2012) and Working Things Out (WTO; Fitzpatrick, Brosnan, & Sharry, 2009) Programme (WTOPPAP) for parents and adolescents, respectively. There are three other Parents Plus Programmes including Parents Plus Early Years (PPEY; Griffin, Guerin, Sharry, & Drumm, 2010), Parents Plus Children’s Programme (PPCP; Coughlin, Sharry, Fitzpatrick, Guerin, & Drumm, 2009) and Parents Plus–Parenting When Separated (PP-PWS; Keating, Sharry, Murphy, Rooney, & Carr, 2015).

The PPAP is a solution-focused positive parent training programme for parents of adolescents aged 11–16 years. PPAP draws on social learning principles and employs DVD footage to model parenting skills and techniques. The aim of PPAP is to build on parents’ existing strengths and resources and provide them with the skills to build good relationships with their teenagers, while also being firm and influential in their lives. Beattie, Fitzpatrick, Guerin, and O’Donoghue (2007) found that parents of children attending a Child and Adolescent Mental Health Service (CAMHS) who completed the PPAP (n = 38) reported significant improvements in their children’s total, peer and conduct difficulties when compared to the routine clinical care group (n = 17). A randomised controlled trial (RCT) study evaluating the PPAP within secondary schools also revealed that the PPAP group (n = 70) reported significant reductions in adolescent total and conduct difficulties, decreased parental stress, increased parental satisfaction and greater progress in achieving their goals compared to the waiting list control group (n = 39; Nitsch, 2011).

The WTO programme is an intervention for adolescents that promotes their positive mental health and supports them in overcoming specific problems. It is skill oriented and draws on CBT principles to highlight the connection between thoughts, feelings and actions. Findings from an evaluation of the WTO programme in CAMHS revealed that the adolescent participants (n = 33) reported a significant reduction in their emotional and behavioural difficulties and used significantly more ‘good’ coping strategies 3 months after completion of the programme. This study is currently being written up for publication.

Aims

The purpose of this study is to evaluate the impact of running the WTO and PPAP programmes together as a family intervention in a school setting with adolescents and their parents. It is expected that participants will see improved outcomes in terms of parent- and adolescent-rated SEBD, general family functioning, parent stress and satisfaction with parenting from baseline to 5-month follow-up. It is also expected that participants will see improvements in goal attainment from baseline to post-intervention.

Method

Study design

This study utilised a repeated measures design to assess change within the intervention group. Time was the primary independent variable. Schools and participants were offered the intervention in January 2012. Participants were assessed pre-intervention (Time 1), post-intervention (Time 2) and at 5-month follow-up (Time 3). A number of dependent variables were examined; these were adolescent SEBD, participant goals, general family functioning, parent stress and satisfaction with parenting. Participant’s goals were measured at Times 1 and 2.

Participants

Participants were a targeted group of 47 adolescents and their 47 parents/guardians who were accessing one of the three tiers of the NBSS in their school. Participants were recruited by Home School Community Liaison (HSCL) and Positive Behaviour Liaison (PBL) teachers. The participants were drawn from eight schools in the Republic of Ireland, with a relatively wide geographic spread (e.g. Dublin, Cork, Wicklow, Cavan, Wexford) ensuring a diverse range of participants both from rural and urban settings and from a broad socioeconomic background. Of the adolescents who took part in the study, 30 (64%) were male. The average age for adolescents was 13.81 years, with ages ranging from 11 to 17 years. All of the parents who took part in the study were mothers; one father took part in the intervention, but has been excluded from the present analysis in order to control for parental gender. Only one parent per family participated. Ethical approval for the study was granted by the Research Ethics Committee of the Mater Misericordiae Hospital, Dublin. Parents/guardians provided written informed consent for them and their children to participate, and informed assent was obtained from adolescents on the day.

Procedure

PBL and HSCL teachers attended 3 days of training in the delivery of both the PPAP and the WTO programmes. In order to maintain fidelity in the implementation of the programmes across sites, facilitators were provided with manualised programmes and were given supervision and support as they facilitated the groups. After an initial invitation to take part in the evaluated programme, families were invited to attend an individual screening meeting to obtain further information and complete assessment measures. The programmes for adolescents and parents ran in parallel over 8 weeks. Two joint individual family sessions were held after sessions 3 and 6. Topics covered in the courses (see Table 1) and the materials distributed to participants were integrated to ensure that the intervention reflected a whole-family.

Table 1.

List of topics covered over 8-weekly sessions in the PPAP and WTO programmes.

Topic	PPAP	WTO
1	Positive communication	Getting started
2	Getting along with your teenager	How we think affects what we feel and do
3	Encouraging your teenager	Managing feeling down
4	Listening to your teenager	New ways of thinking
5	Establishing rules	Stop and think – the key to solving problems
6	Using consequences/having a discipline plan	Dealing with anger and conflict
7	Dealing with conflict	Communicating well and building relationships
8	Problem solving	Planning for the future and making positive choices

WTO: Working Things Out; PPAP: Parents Plus Adolescent Programme.

Upon completion of the two programmes, adolescents and parents completed the assessment measures for the second time and again at 5-month follow-up. Goal attainment was assessed at Times 1 and 2 only.

Assessment measures

The Strengths and Difficulties Questionnaire (SDQ) is a 25-item questionnaire that assesses young people’s behaviours, emotions and relationships (R. Goodman, 1997, 2001). The 25 items are divided into five scales: conduct, hyperactivity, emotional, peer problems and pro-social. The scores from these scales, excluding the pro-social scale, are summed to generate a total difficulties score. The Cronbach’s alpha of the parent- and self-rated versions was satisfactory at .81.

The McMaster Family Assessment Device (FAD) is made up of seven subscales which measure problem solving, communication, roles, affective responsiveness, affective involvement, behaviour control and general functioning in families (Epstein, Baldwin, & Bishop, 1983). The General Functioning scale used in this study assesses the overall health/pathology of the family. Parents and children rated their agreement with how well an item describes their families (Cronbach’s alpha was satisfactorily above .80).

The Kansas Parental Satisfaction Scale (KPS) is a three-item scale; it measures parental satisfaction with their child’s behaviour, with their parenting role and with their relationship with their child (Schumm & Hall, 1994). The three questions are summed to give an overall satisfaction score, and items are scored on a 7-point Likert scale. Cronbach’s alpha approached satisfactory at .65.

The Parental Stress Scale (PSS) is a self-report scale that contains 18 items representing themes of parenthood that are positive (emotional benefits, self-enrichment, personal development) and negative (demands on resources, opportunity costs and restrictions; Berry & Jones, 1995). Respondents are asked to agree or disagree with items in terms of their typical relationship with their child and to rate each item on a 5-point Likert scale: ‘strongly disagree’ (1), ‘disagree’ (2), ‘undecided’ (3), ‘agree’ (4) and ‘strongly agree’ (5). The eight positive items are reverse scored so that possible scores on the scale can range between 18 and 90. Higher scores on the scale indicate greater stress. Cronbach’s alpha was satisfactory at .81.

The Goal Attainment measure designed for this study requires parents to pick two goals for their child and two personal goals and adolescents to pick two goals for their family and two personal goals that they would work towards during the intervention. Participants rated them using a visual analogue scale (i.e. 0 = not very close to achieving goal and 10 = have reached the goal).

Statistical strategy

Data were analysed using SPSS (version 20). Repeated measures analysis of variance (ANOVA) was used to examine changes in the intervention group over the three assessment times. Post hoc analyses were conducted with paired-samples t-tests and a Bonferroni-corrected significance value of p < .017. Only paired-samples t-tests were used to analyse goal attainment.

Results

Descriptive statistics

Participants’ mean scores, standard deviations and range of scores at baseline on the assessment measures completed can be seen in Table 2. The number of adolescents with SEBD in the clinical range and participants rating their family’s general functioning above the cut-off can be seen in Table 3.

Table 2.

Descriptives of assessment total scores at baseline.

	N	Minimum score	Maximum score	M (SD)
Parent measures
SDQ	46	2	30	15.48 (6.64)
PSS	44	25	71	40.68 (9.56)
KPS	44	8	20	14.45 (3.07)
McMaster	44	1.16	3	1.97 (.41)
Personal goal	13	1	6	2.50 (1.60)
Child goal	23	0	9	2.60 (2.10)
Adolescent measures
SDQ	46	0	29	13.59 (6.19)
McMaster	45	1	3.17	2.05 (.54)
Personal goal	31	1	8	3.53 (1.96)
Family goal	25	0	8	3.91 (2.30)

SDQ: Strengths and Difficulties Questionnaire; PSS: Parent Stress Scale; KPS: Kansas Parenting Satisfaction; McMaster: McMaster General Functioning; SD: standard deviation.

Table 3.

Numbers of participants in the clinical range for the McMaster and SDQ at all three time points.

	n	T1	T2	T3
Parent measures
SDQ	32
Abnormal		17 (53%)	13 (41%)	13 (41%)
Normal		15 (47%)	19 (59%)	19 (59%)
McMaster	31
Abnormal		19 (61%)	11 (35%)	13 (42%)
Normal		12 (39%)	20 (65%)	18 (58%)
Adolescent measures
SDQ	33
Abnormal		9 (27%)	13 (39%)	12 (36%)
Normal		24 (73%)	20 (61%)	21 (64%)
McMaster	31
Abnormal		20 (65%)	13 (42%)	15 (48%)
Normal		11 (35%)	18 (58%)	16 (52%)

The ‘clinical’ category for the SDQ includes cases in the ‘borderline’ range.

SDQ: Strengths and Difficulties Questionnaire; McMaster: McMaster General Functioning; T1: baseline; T2: post-intervention; T3: 5-month follow-up.

Repeated measures ANOVA and post hoc paired-samples t-test

Repeated measures ANOVA results for parent- and adolescent-rated assessment measures at baseline, post-intervention and 5-month follow-up can be seen in Table 4. Post hoc paired-samples t-test results can be seen in Table 5.

Table 4.

Repeated measures ANOVA results for assessment total scores at all three time points.

	N	F (p)	η²	T1	T2	T3
				M (SD)	M (SD)	M (SD)
Parent measures
SDQ	32	3.93 (.030^)	.21^a	14.56 (7.02)	12.63 (7.28)	12.16 (7.63)
McMaster	31	3.24 (.054)	.18^a	1.96 (.46)	1.79 (.46)	1.81 (.45)
PSS	30	8.69 (.001^)	.38^a	40.57 (10.52)	35.03 (9.78)	37.53 (11.32)
KPS	30	5.69 (.008^)	.29^a	14.67 (3.14)	16.63 (3.06)	16.40 (3.06)
Adolescent measures
SDQ	34	.182 (.834)	.011	13.27 (6.52)	12.94 (6.1)	12.79 (6.54)
McMaster	31	2.97 (.067)	.170^a	2.02 (.53)	1.84 (.47)	1.85 (.44)

ANOVA: analysis of variance; SDQ: Strengths and Difficulties Questionnaire; PSS: Parent Stress Scale; KPS: Kansas Parenting Satisfaction; McMaster: McMaster General Functioning; SD: standard deviation.

Large effect size.

Significant at p < .05.

Table 5.

Paired-samples t-test comparing assessment total scores across all three time points.

	n	T1–T2		T2–T3		T1–T3
		p	η²	p	η²	p	η²
Parent measures
SDQ	32	.043	.13^a	.610	.010	.011^^b	.19^a
McMaster	31	.020	.17^a	.783	.003	.078	.10^b
PSS	30	.000^	.38^a	.274	.040	.154	.07^b
KPS	30	.002^	.29^a	.603	.010	.010^	.21^a
Personal goal	14	.000^	.75^a	–	–	–	–
Child goal	15	.002^	.50^a	–	–	–	–
Adolescent measures
SDQ	34	.651	.006	.819	.002	.549	.011
McMaster	31	.030	.15^a	.822	.002	.042	.13^b
Personal goal	17	.000^b	.67^a	–	–	–	–
Family goal	15	.000^b	.63^a	–	–	–	–

SDQ: Strengths and Difficulties Questionnaire; PSS: Parent Stress Scale; KPS: Kansas Parenting Satisfaction; McMaster: McMaster General Functioning.

Large effect size.

Moderate effect size.

Significant at p < .017.

Parent-rated adolescent difficulties

As can be seen in Table 4, parents reported a significant improvement in total difficulty scores according to repeated measures ANOVA, and the effect size for this improvement was large. Post hoc paired-samples t-tests (see Table 5) indicated that there was a large effect size for the improvement in scores from T1 to T3.

Further repeated measures ANOVA analyses with the SDQ subscales indicated a significant change over time only for the Conduct subscale, F(2, 30) = 8.381, p = .001, η² = .36, with a large effect size for this change. Post hoc paired-samples t-tests indicated that there was a similar significant change from T1 (M = 3.19, SD = 2.25) to T2 (M = 2.00, SD = 1.98), t(31) = 3.881, p = .001, η² = .327, and from T1 to T3 (M = 1.97, SD = 1.91), t(31) = 3.849, p = .001, η² = .323, with large effect sizes observed.

Parent-rated general functioning

Improvements in parent-rated general functioning scores only approached significance, with a large effect size (Table 4). Likewise, post hoc paired-samples t-tests showed improvements from T1 to T2 that only approached significance.

Parent-rated stress

There was a strongly significant improvement in parent stress scores (see Table 4), with a very large effect size accompanying this. Post hoc tests showed a strongly significant decrease in parent stress from T1 to T2 (Table 5). There was a slight, non-significant increase in parent stress from T2 to T3 (small effect size), which meant that the T1–T3 improvement was not statistically significant.

Parent-rated satisfaction with parenting

Improvement for satisfaction with parenting scores was strongly significant, with a very large effect size for the change from baseline to 5-month follow-up (Table 4). Post hoc analyses showed that parenting satisfaction improved significantly from T1 to T2, with a large effect size. This improvement was maintained at follow-up, with a large effect size from T1 to T3.

Adolescent-rated difficulties

Repeated measures ANOVA analyses showed a non-significant improvement in adolescents’ total difficulties, with a small effect size for this improvement. However, further repeated measures ANOVA analyses with the SDQ subscales indicated a significant change for the Emotional subscale, F(2, 32) = 3.735, p = .035, η² = .189, with a large effect size for this change. Post hoc t-tests indicated a significant improvement in emotional difficulties from T1 (M = 3.88, SD = 2.68) to T3 (M = 3.15, SD = 2.78), t (33) = 2.616, p = .013, η² = .172, with a large effect size observed.

Adolescent-rated general functioning

Improvements in adolescent-rated general functioning scores only approached significance, with a large effect size (see Table 4). Likewise, post hoc paired-samples t-tests showed improvements from T1 to T2 and T1 to T3 that only approached significance.

Parent- and adolescent-rated goal attainment

Participants showed significant movement towards their goals from T1 to T2 with large effect sizes observed (Table 5).

Discussion

This study found that parents rated their children’s overall difficulties and adolescents rated their emotional difficulties as having significantly improved from pre-test to 5-month follow-up of the WTOPPAP intervention. Parents also rated their children’s conduct and their satisfaction with parenting as having significantly improved and their stress levels as having decreased significantly over the course of the intervention, with gains maintained at follow-up. In addition, parents and adolescents reported significant movement towards their goals by the end of the group.

These findings are consistent with outcomes from group family interventions delivered in school settings (e.g. Moretti & Obsuth, 2009; Valdez et al., 2005; Vitaro, Brendgen, & Tremblay, 2001; Wilson & Lipsey, 2007) and with previous investigations of the PPAP and WTO programmes (e.g. Beattie et al., 2007; Nitsch, 2011). The finding that parents reported a greater severity of adolescent difficulties and that parents and adolescents disagree on ratings of emotional and behavioural difficulties is relatively common in the literature (Martin, Ford, Dyer-Friedman, Tang, & Huffman, 2004; Van der Meer, Dixon, & Rose, 2008; Van Roy, Groholt, Heyerdahl, & Clench-Aas, 2010). Young people with externalising disorders are more likely to be identified as having difficulties and needing intervention due to the more visible nature of their problematic behaviours (Bradshaw, Buckley, & Ialongo, 2008), as was the case for many of the participants in this study. Moreover, parents are more likely to rate externalising behaviours as problematic (Van der Meer et al., 2008), while young people are more likely to rate internalising problems as being clinically significant than externalising problems they are experiencing (Martin et al., 2004; Van der Meer et al., 2008). Hence, it is considered best practice to obtain data from numerous sources.

As expected, the WTOPPAP intervention improved parental stress and satisfaction with parenting. This finding may partially explain the improvements in adolescent SEBD; parents’ ability to effectively implement adaptive parenting strategies and manage their children’s problematic behaviours can be adversely affected by stress and low confidence in parenting (T. L. Jones & Prinz, 2005; Morawska, Winter, & Sanders, 2009; Webster-Stratton, 1990). However, it is important to note that parental stress increased from the final session of the programme to the follow-up session, although not significantly. This finding emphasises that there is no ‘magic bullet’ intervention and that engagement and support for parents is an on-going and crucial aspect of the process of supporting young people with SEBD. Previous research has indicated that the provision of follow-up support sessions may help to maintain treatment effects after the conclusion of an intervention (Eyberg, Edwards, Boggs, & Foote, 1998). Therefore, it may be effective to increase contact with parents and adolescents in the months following the end of the WTOPPAP programme with additional family or parent-only sessions.

Strengths and limitations

The main strength of this research is how it demonstrated that a manualised family intervention can be effectively rolled out and evaluated in Irish schools in partnership with the NBSS and parents. There is always a trade-off between evaluating the effectiveness and efficacy of an intervention (Hoagwood, Hibbs, Brent, & Jensen, 1995), with both having positive and negative aspects (e.g. greater control over confounding variables, generalisability, attrition). As previous research into the PPAP and WTO programmes has shown the efficacy of both programmes in clinical settings (e.g. Beattie et al., 2007; Nitsch, 2011), this study was concerned more with the effective delivery of these programmes in a novel setting, and so the positive findings in relation to adolescent and parent outcomes are encouraging. Due to the number of different sites the study was conducted across, the findings of the study are also more readily generalisable and thus provide relatively robust evidence for the effectiveness of the intervention.

It has previously been stated that good implementation of an intervention is predictive of better outcomes (Wilson & Lipsey, 2007). Factors that are crucial to a high standard of implementation are the adequate training of facilitators, close and receptive supervision during the intervention and the provision of principal support (Gottfredson & Gottfredson, 2002). Another strength of this research was the adherence to these factors in the roll-out and evaluation of WTOPPAP, through the use of session rating forms, close supervision from the administrators of the programme and from NBSS staff, and the provision of intensive 3-day facilitator training prior to implementation. Furthermore, and due to the well-structured and manualised nature of the intervention coupled with on-going support provided to facilitators, it was possible to ensure a high degree of fidelity in the delivery of the programme across different schools.

A further strength of this research is that of the attrition rate: from the beginning of the intervention to its conclusion, four students (8.5%) and five parents (11%) dropped out of the programme. These figures are relatively low and compare favourably to similar interventions conducted in school settings (e.g. Baruch et al., 2011; Koning, Van den Eijnden, Verdurmen, Engels, & Vollebergh, 2011). Although the rate of attrition increases at the time of follow-up, this could be attributed to the relatively long duration of the follow-up period (5 months) and that the follow-up interview was not a therapeutic component of the intervention, meaning that families’ motivation to attend may have been reduced. This could potentially be offset in future evaluations using ‘booster’ family sessions in the period after the conclusion of the programme.

Naturally, a weakness of this study was the lack of a comparison group, to determine whether the outcome gains reported here can be directly attributed to participation in WTOPPAP. The provision of a comparison group was initially pursued during the design stage of the research; however, due to the complexity involved in the multi-location roll-out of the intervention, it was not possible to secure a comparison group which would have been of a size sufficient to conduct robust statistical analyses. However, due to the relative success of the implementation of the WTOPPAP, the use of a comparison group in future research is more feasible. Based on the positive findings reported here, comparative analyses are considered a priority for further evaluation of the intervention, which is currently being developed.

It is a commonly encountered occurrence that mothers are more likely to participate in parenting programmes than fathers (see Fabiano, 2007); however, one potential limitation of this study is that all parents who participated were mothers. It is, therefore, not possible to infer what impact participation in WTOPPAP may have on fathers based on the present findings. Furthermore, in cases where both parents reside together, the implementation of parenting strategies can be more effective when they are consistently applied (Kaminski, Valle, Filene, & Boyle, 2008), which is facilitated by both parents attending parenting courses together. As this was a first-level intervention and the fidelity of implementation was prioritised, it was not possible to focus on recruiting parent dyads across the different settings in this study. However, future implementation of the WTOPPAP could place greater emphasis on facilitators recruiting both parents, where possible, as this could potentially result in greater improvements and maintenance of improvement in outcomes.

Conclusion

The findings of this first-level evaluation of the delivery of the combined WTOPPAP in Irish secondary schools broadly supported the effectiveness of the intervention. They also demonstrated that it is possible to effectively deliver a multi-modal family intervention in an Irish school setting and for school staff to conduct a valid evaluation of the intervention. These findings are encouragingly positive and support the further roll-out and evaluation of the WTOPPAP programme.

Footnotes

Funding

The author(s) received no financial support for the research, authorship and/or publication of this article.

Author biographies

Eóin D Rickard is a Psychologist in training within the Clinical Psychology Doctorate Programme in Trinity College Dublin.

Eileen Brosnan is a Senior Trainer with Parents Plus and also the lead developer of the Working Things Out Programme. She is completing her PHD in University College Dublin.

Aoife O’Laoide is a psychologist within the Clinical Psychology Doctorate Programme in University College Dublin.

Ciara Wynne holds a PhD from Trinity College Dublin. She has worked on projects evaluating a Health Promoting Schools intervention and Parents Plus Programmes and is now embarking on a Clinical Psychology Doctorate Programme in University College Dublin.

Mary Keane is the National Coordinator of the National Behaviour Support Service and was was involved in the design of the PPAP/ WTO intervention

Margaret McCormack worked as a Regional Development Officer in the National Behaviour Support Service and had a special role supporting schools delivering the PPAP/ WTO intervention.

John Sharry, is Co-founder/ CEO of Parents Plus, co-developer of the Parents Plus Programmes and an Adjunct Senior Lecturer in the School of Psychology in University College Dublin.

References

Banks

McCoy

(2011). A study on the prevalence of special educational needs. Dublin, Ireland: NCSE.

Barlow

Smailagic

Huband

Roloff

Bennett

(2012). Group-based parent training programmes for improving parental psychosocial health. Campbell Systematic Reviews, 15, doi:10.4073/csr.2012.15

Barnes

T. N.

Smith

S. W.

Miller

M. D.

(2014). School-based cognitive-behavioral interventions in the treatment of aggression in the United States: A meta-analysis. Aggression and Violent Behavior, 19, 311–321.

Baruch

Vrouva

Wells

(2011). Outcome findings from a parent training programme for young people with conduct problems. Child and Adolescent Mental Health, 16, 47–54.

Beattie

Fitzpatrick

Guerin

O’Donoghue

(2007). Parent management training for adolescent mental health disorders: A controlled trial. In Ireland Branch Research Meeting, The Association for Child and Adolescent Mental Health, Dublin.

Berry

J. O.

Jones

W. H.

(1995). The Parental Stress Scale: Initial psychometric evidence. Journal of Social and Personal Relationships, 12, 463–472.

Bradshaw

C. P.

Buckley

J. A.

Ialongo

N. S.

(2008). School-based service utilization among urban children with early onset educational and mental health problems: The squeaky wheel phenomenon. School Psychology Quarterly, 23, 169–186.

Brannan

A. M.

Heflinger

C. A.

(2001). Distinguishing caregiver strain from psychological distress: Modeling the relationship between child, family, and caregiver variables. Journal of Child and Family Studies, 10, 405–418.

Buyse

Verschueren

Doumen

Van Damme

Maes

(2008). Classroom problem behavior and teacher-child relationships in kindergarten: The moderating role of classroom climate. Journal of School Psychology, 46, 367–391.

10.

Carr

(2014). The evidence base for family therapy and systemic interventions for child-focused problems. Journal of Family Therapy, 36, 107–157.

11.

Chorpita

B. F.

Daleiden

E. L.

Ebesutani

Young

Becker

K. D.

Nakamura

B. J.

. . . Starace

(2011). Evidence-based treatments for children and adolescents: An updated review of indicators of efficacy and effectiveness. Clinical Psychology: Science and Practice, 18, 154–172.

12.

Christenson

S. L.

Havsy

L. H.

(2004). Family-school-peer relationships: Significance for social, emotional and academic learning. In Zins

J. E.

Weissberg

Wang

M. C.

Walberg

H. J.

(Eds.), Building academic success on social and emotional learning: What does the research say (pp. 59–75). New York, NY: Teachers College Press.

13.

Collishaw

Maughan

Goodman

Pickles

(2004). Time trends in adolescent mental health. Journal of Child Psychology and Psychiatry, 45, 1350–1362.

14.

Cooper

Jacobs

(2011). Evidence of best practice models and outcomes in the education of children with emotional disturbance/behavioural difficulties: An International Review. Dublin, Ireland: NCSE.

15.

Cooper

McIntyre

(1996). Effective teaching and learning: Teachers’ and students’ perspectives. New York, NY: McGraw-Hill.

16.

Corkum

P. V.

McKinnon

M. M.

Mullane

J. C.

(2005). The effect of involving classroom teachers in a parent training program for families of children with ADHD. Child & Family Behavior Therapy, 27(4), 29–49.

17.

Coughlin

Sharry

Fitzpatrick

Guerin

Drumm

(2009). A controlled clinical evaluation of the Parents Plus Children’s Programme: A video-based programme for parents of children aged 6–11 years with behavioural and developmental problems. Clinical Child Psychology and Psychiatry, 14, 541–558.

18.

Durlak

J. A.

Weissberg

R. P.

Dymnicki

A. B.

Taylor

R. D.

Schellinger

K. B.

(2011). The impact of enhancing students’ social and emotional learning: A meta-analysis of school-based universal interventions. Child Development, 82, 405–432.

19.

Epstein

N. B.

Baldwin

L. M.

Bishop

D. S.

(1983). The McMaster family assessment device. Journal of Marital and Family Therapy, 9, 171–180.

20.

Eyberg

S. M.

Edwards

Boggs

S. R.

Foote

(1998). Maintaining the treatment effects of parental training: The role of booster sessions and other maintenance strategies. Clinical Psychology: Science and Practice, 5, 544–554.

21.

Fabiano

G. A.

(2007). Father participation in behavioral parent training for ADHD: Review and recommendations for increasing inclusion and engagement. Journal of Family Psychology, 21, 683–693.

22.

Fantuzzo

McWayne

Bulotsky

(2003). Forging strategic partnerships to advance mental health science and practice for vulnerable children. School Psychology Review, 32, 17–37.

23.

Fitzpatrick

Brosnan

Sharry

(2009). ‘Working Things Out’: A therapeutic resource for professionals working with young people. In Essau

(Ed.), Treatment for adolescent depression: Theory and practice (pp. 283–312). Oxford, UK: Oxford University Press.

24.

Gillham

J. E.

Reivich

K. J.

Freres

D. R.

Lascher

Litzinger

Shatté

Seligman

M. E.

(2006). School-based prevention of depression and anxiety symptoms in early adolescence: A pilot of a parent intervention component. School Psychology Quarterly, 21, 323–348.

25.

Goodman

Joyce

Smith

J. P.

(2011). The long shadow cast by childhood physical and mental problems on adult life. Proceedings of the National Academy of Sciences, 108, 6032–6037.

26.

Goodman

(1997). The Strengths and Difficulties Questionnaire: A research note. Journal of Child Psychology and Psychiatry, 38, 581–586.

27.

Goodman

(2001). Psychometric properties of the Strengths and Difficulties Questionnaire. Journal of American Academy of Child Adolescent Psychiatry, 40, 1337–1347.

28.

Gottfredson

D. C.

Gottfredson

G. D.

(2002). Quality of school-based prevention programs: Results from a national survey. Journal of Research in Crime & Delinquency, 39, 3–36.

29.

Griffin

Guerin

Sharry

Drumm

(2010). A multicentre controlled study of an early intervention parenting programme for young children with behavioural and developmental difficulties. International Journal of Clinical and Health Psychology, 10, 279–294.

30.

Gulliver

Griffiths

K. M.

Christensen

(2010). Perceived barriers and facilitators to mental health help-seeking in young people: A systematic review. BMC Psychiatry, 10, Article 113.

31.

Hoagwood

Hibbs

Brent

Jensen

(1995). Introduction to the special section: Efficacy and effectiveness in studies of child and adolescent psychotherapy. Journal of Consulting and Clinical Psychology, 63, 683–687.

32.

Hoganbruen

Clauss-Ehlers

Nelson

Faenza

M. M.

(2003). Effective advocacy for school-based mental health programs. New York, NY: Kluwer Academic.

33.

Hornby

Witte

(2010). Parent involvement in rural elementary schools in New Zealand: A survey. Journal of Child and Family Studies, 19, 771–777.

34.

Hughes

Ensor

(2007). Positive and protective: Effects of early theory of mind on problem behaviours in at-risk preschoolers. Journal of Child Psychology and Psychiatry, 48, 1025–1032.

35.

Hurth-Bocks

A. C.

Hughes

H. M.

(2008). Parenting stress, parenting behaviour, and children’s adjustment in families experiencing intimate partner violence. Journal of Family Violence, 23, 243–251.

36.

Ingul

J. M.

Klöckner

C. A.

Silverman

W. K.

Nordahl

H. M.

(2012). Adolescent school absenteeism: Modelling social and individual risk factors. Child and Adolescent Mental Health, 17, 93–100.

37.

Jones

T. L.

Prinz

R. J.

(2005). Potential roles of parental self-efficacy in parent and child adjustment: A review. Clinical Psychology Review, 25, 341–363.

38.

Kaminski

J. W.

Valle

L. A.

Filene

J. H.

Boyle

C. L.

(2008). A meta-analytic review of components associated with parent training program effectiveness. Journal of Abnormal Child Psychology, 36, 567–589.

39.

Kaslow

N. J.

Broth

M. R.

Smith

C. O.

Collins

M. H.

(2012). Family-based interventions for child and adolescent disorders. Journal of Marital and Family Therapy, 38(1), 82–100.

40.

Kazdin

A. E.

(2010). Problem-solving skills training and parent management training for oppositional defiant disorder and conduct disorder. In Weisz

Kazdin

(Eds.), Evidence-based psychotherapies for children and adolescents (2nd ed., pp. 211–226). New York, NY: Guilford Press.

41.

Keating

Sharry

Murphy

Rooney

Carr

(2015). An evaluation of the Parents Plus–Parenting When Separated programme. Clinical Child Psychology and Psychiatry. Advance online publication, doi:10.1177/1359104515581717

42.

Kelleher

McGilloway

(2006). Incredible years in Ireland: A pilot evaluation of the Parent Training (BASIC) programme. Dublin: The Clondalkin Partnership.

43.

Koning

I. M.

Van den Eijnden

R. J.

Verdurmen

J. E.

Engels

R. C.

Vollebergh

W. A.

(2011). Long-term effects of a parent and student intervention on alcohol use in adolescents: A cluster randomized controlled trial. American Journal of Preventive Medicine, 40, 541–547.

44.

LaBahn

(1995). Education and parental involvement in secondary schools: Problems, solutions, and effects. Educational Psychology Interactive, 1. Valdosta, GA: Valdosta State University. Retrieved from: http://www.edpsycinteractive.org/files/parinvol.html

45.

LaRusso

M. D.

Romer

Selman

R. L.

(2008). Teachers as builders of respectful school climates: Implications for adolescent drug use norms and depressive symptoms in high school. Journal of Youth and Adolescence, 37, 386–398.

46.

Lean

D. S.

Colucci

V. A.

(2010). Barriers to learning: The case for integrated mental health services in schools. Lanham, MD: Rowman & Littlefield.

47.

Martin

J. L.

Ford

C. B.

Dyer-Friedman

Tang

Huffman

L. C.

(2004). Patterns of agreement between parent and child ratings of emotional and behavioral problems in an outpatient clinical setting: When children endorse more problems. Journal of Developmental and Behavioral Pediatrics, 25, 150–155.

48.

McLennan

Reckord

Clarke

(2008). A mental health outreach program for elementary schools. Journal of the Canadian Academy of Child and Adolescent Psychiatry, 17, 122–130.

49.

Michelson

Davenport

Dretzke

Barlow

Day

(2013). Do evidence-based interventions work when tested in the ‘real world?’ A systematic review and meta-analysis of parent management training for the treatment of child disruptive behavior. Clinical Child and Family Psychology Review, 16, 18–34.

50.

Morawska

Winter

Sanders

M. R.

(2009). Parenting knowledge and its role in the prediction of dysfunctional parenting and disruptive child behaviour. Child: Care, Health and Development, 35, 217–226.

51.

Moretti

M. M.

Obsuth

(2009). Effectiveness of an attachment-focused manualized intervention for parents of teens at risk for aggressive behaviour: The connect program. Journal of Adolescence, 32, 1347–1357.

52.

Nitsch

(2011). Positive parenting: A randomized controlled trial evaluation of the Parents Plus Adolescent Programme in schools (PhD dissertation). Department of Psychology, University of Limerick, Limerick, Ireland.

53.

Patrikakou

Weissberg

(2007). School-family partnerships to enhance children’s social, emotional, and academic learning. In Bar-on

Maree

Elias

(Eds.), Educating people to be emotionally intelligent (pp. 49–77). Westport, CT: Praeger.

54.

Reijntjes

Kamphuis

J. H.

Prinzie

Telch

M. J.

(2010). Peer victimization and internalizing problems in children: A meta-analysis of longitudinal studies. Child Abuse & Neglect, 34, 244–252.

55.

Rickwood

D. J.

Deane

F. P.

Wilson

C. J.

(2007). When and how do young people seek professional help for mental health problems? The Medical Journal of Australia, 187(7), S35–S39.

56.

Sancassiani

Pintus

Holte

Paulus

Moro

M. F.

Cossu

. . . Lindert

(2015). Enhancing the emotional and social skills of the youth to promote their wellbeing and positive development: A systematic review of universal school-based randomized controlled trials. Clinical Practice & Epidemiology in Mental Health, 11(Suppl. 1, M2), 21–40.

57.

Schoenfeld

N. A.

Janney

D. M.

(2008). Identification and treatment of anxiety in students with emotional or behavioral disorders: A review of the literature. Education and Treatment of Children, 31, 583–610.

58.

Schumm

W. R.

Hall

(1994). Kansas parental satisfaction scale (KPS). In Fischer

Corcoran

(Eds.), Measures for clinical practice: A sourcebook (2nd ed., Vol. 1: Couples, families, and children, pp. 345–346). New York, NY: The Free Press.

59.

Sharry

Fitzpatrick

(2012). Parents Plus Adolescents Progamme: A DVD based parenting course on managing conflict and getting on better with older children and teenagers aged 11–16 years (3rd ed.). Dublin, Ireland: Parents Plus.

60.

Shriver

M. D.

Allen

K. D.

(2010). Parent training: Working with families to develop and implement interventions. In Peacock

G. G.

Ervin

R. A.

Daly

E. J.

Merrell

K. W.

(Eds.), Practical handbook of school psychology: Effective practices for the 21st century (pp. 408–421). New York, NY: Guilford Press.

61.

Suhrcke

Pillas

Selai

(2008). Economic aspects of mental health in children and adolescents (Social cohesion for mental well-being among adolescents). Copenhagen, Denmark: WHO Regional Office for Europe.

62.

Turnbull

A. P.

Turnbull

H. R.

(2001). Families, professionals, and exceptionality: Collaborating for empowerment (4th ed.). Upper Saddle River, NJ: Prentice Hall.

63.

Valdez

C. R.

Carlson

Zanger

(2005). Evidence-based parent training and family interventions for school behavior change. School Psychology Quarterly, 20, 403–433.

64.

Van Acker

Mayer

M. J.

(2009). Cognitive-behavioral interventions and the social context of the school: A stranger in a strange land. In Mayer

M. J.

Van Acker

Lochman

J. E.

Gresham

F. M.

(Eds.), Cognitive-behavioral interventions for emotional and behavioral disorders (pp. 82–108). New York, NY: Guilford Press.

65.

Van der Meer

Dixon

Rose

(2008). Parent and child agreement on reports of problem behaviour obtained from a screening questionnaire, the SDQ. European Child & Adolescent Psychiatry, 17, 491–497.

66.

Van Roy

Groholt

Heyerdahl

Clench-Aas

(2010). Understanding discrepancies in parent-child reporting of emotional and behavioural problems: Effects of relational and socio-demographic factors. BMC Psychiatry, 10, Article 56.

67.

Vitaro

Brendgen

Tremblay

R. E.

(2001). Preventive intervention: Assessing its effects on the trajectories of delinquency and testing for mediational processes. Applied Developmental Science, 5, 201–213.

68.

Webster-Stratton

(1990). Long-term follow-up of families with young conduct problem children: From preschool to grade school. Journal of Clinical Child Psychology, 19, 144–149.

69.

Webster-Stratton

Reid

M. J.

(2010). The Incredible Years Parents, Teachers and Children Training Series: A multifaceted treatment approach for young children with conduct problems. In Weisz

Kazdin

(Eds.), Evidence-based psychotherapies for children and adolescents (2nd ed., pp. 194–210). New York, NY: Guilford Press.

70.

Wilson

S. J.

Lipsey

M. W.

(2007). School-based interventions for aggressive and disruptive behavior: Update of a meta-analysis. American Journal of Preventive Medicine, 33, S130–S143.

71.

Wolitzky-Taylor

Bobova

Zinbarg

R. E.

Mineka

Craske

M. G.

(2012). Longitudinal investigation of the impact of anxiety and mood disorders in adolescence on subsequent substance use disorder onset and vice versa. Addictive Behaviors, 37, 982–985.

72.

Woolfenden

S. R.

Williams

Peat

J. K.

(2002). Family and parenting interventions for conduct disorder and delinquency: A meta-analysis of randomised controlled trials. Archives of Disease in Childhood, 86, 251–256.