Multiple Family Group Service Delivery Model for Children With Disruptive Behavior Disorders: Impact on Caregiver Stress and Depressive Symptoms

Abstract

Disproportionately high rates of caregiver stress and depression are found among poverty-impacted communities, with high levels of caregiver stress and depression putting youth at heightened risk for the onset and perpetuation of disruptive behavior disorders. The purpose of this study was to examine the effects of a behavioral parent training program called the 4Rs and 2Ss for Strengthening Families Program (4R2S) on caregiver stress and depressive symptoms among 320 youth aged 7 to 11 and their families assigned to either the 4R2S or services as usual (SAU) condition. Among caregivers with clinically significant (CS) scores at baseline, 4R2S participants manifested significantly reduced scores on the stress and depressive symptom scores to SAU participants at 6-month follow-up. Findings suggest that 4R2S may reduce caregiver stress and depressive symptoms among those caregivers initially manifesting CS levels of stress or depressive symptoms.

Keywords

child disruptive behavior disorders caregiver stress caregiver depression poverty-impacted communities 4Rs and 2Ss for Strengthening Families Program multiple family groups behavioral parent training

Introduction

The combination of stressors associated with poverty (e.g., community violence, housing instability, limited economic and social supports) frequently result in a common “package” of circumstances within impacted communities, where families manifest elevated rates of parental stress and depression, as well as child behavioral difficulties (Bannon & McKay, 2007; Ertel, Rich-Edwards, & Koenen, 2011; G. W. Evans, 2004; Ghandour, Kogan, Blumberg, Jones, & Perrin, 2012; Kazdin & Whitley, 2003; Siefert, Finlayson, Williams, Delva, & Ismail, 2007; Yoshikawa, Aber, & Beardslee, 2012). Economic hardship often contributes to parental stress and depression (Conger, Conger, & Martin, 2010), leading to interruptions to the parent–child bonding process, as well as increases in inconsistent, harsh parenting practices (Coiro, Riley, Broitman, & Miranda, 2012; Deater-Deckard, 1998; Hall, 2012). Such parenting practices, in concert with poverty-related stressors, increase children’s vulnerability to the onset and perpetuation of conduct disorder (CD) and oppositional defiant disorder (ODD; American Psychiatric Association, 2013), which are part of a larger category of disruptive behavior disorders (DBDs; Forgatch & Patterson, 2010). Characterized by serious and chronic oppositionality, impulsivity, aggression, and delinquent acts (Kilgore, Snyder, & Lentz, 2000), these disorders are a costly public health issue (Scott, Knapp, Henderson, & Maughan, 2001) as they are associated with multiple maladaptive future outcomes for children (e.g., child maltreatment, comorbid mental health, substance use/abuse, delinquency; Beauchaine & Hinshaw, 2013). Moreover, children’s behavioral difficulties can, in turn, further exacerbate parental stress and depressive symptoms (Marchand, Hock, & Widaman, 2002; Munson, McMahon, & Spieker, 2001; Spieker, Larson, Lewis, Keller, & Gilchrist, 1999).

Unfortunately, untreated parent stress and depression can also hinder treatment for DBDs. Behavioral parent training (BPT) is one of the most effective types of treatment for DBDs (S. W. Evans, Owens, & Bunford, 2014; Eyberg, Nelson, & Boggs, 2008), with the common goal of decreasing problematic youth behaviors and promoting positive behaviors through enhancing caregivers’ parenting skills (Chacko et al., 2015). However, parental stress and depression negatively impact caregivers’ ability to implement effective parenting skills for managing child behavior (Chronis, Chacko, Fabiano, Wymbs, & Pelham, 2004). Specifically, caregivers struggling with stress and depression are more likely to discipline children harshly through coercive parenting and physical discipline (Deater-Deckard, & Dodge, 1997). Stress and depression also increase the likelihood that caregivers will be less responsive to children’s needs, instead manifesting increased irritability in their interactions with children. As a result, parent–child interactions become increasingly conflict-ridden (Chronis et al., 2004; Kazdin & Whitley, 2003; Lovejoy, Graczyk, O’Hare, & Neuman, 2000). Consequently, children with DBDs are unlikely to improve if caregiver stress and depression concurrently impede the use of corrective parenting practices. This suggests that addressing both child- and caregiver-level concerns may maximize treatment success for BPTs.

At the same time, there is some debate regarding ideal strategies to address both parent and child needs in treatment. Stress and depression have been shown to abate when parental emotional health is targeted by integrating enhancements to typical BPT such as problem-solving sessions (Chacko et al., 2009; Kazdin, Siegel, & Bass, 1992; Kazdin & Whitley, 2003). However, such enhancements may potentially increase treatment costs and length, due to the addition of separate sessions and/or concurrently run groups. Alternatively, other studies suggest that BPT without enhancements reduces parental stress and depression by decreasing child behavior problems, which subsequently lead to increased parental sense of efficacy and competence in their parenting skills (DeGarmo, Patterson, & Forgatch, 2004; Pisterman et al., 1992). This suggests that BPT without enhancements may be a more cost-effective, first line of intervention to address both child behavioral difficulties and parental distress (Barth, 2009).

To date, however, there is uneven evidence that BPT without enhancements for youth with behavioral difficulties can also lead to significant reductions in parental stress and depression. While some studies have found significant reductions in parental stress and depression when comparing treatment with control groups (DeGarmo et al., 2004; Hutchings et al., 2007; Larsson et al., 2009; Pisterman et al., 1992; Stattin, Enebrink, Özdemir, & Giannotta, 2015), others have found no differences (Braet et al., 2009; Charles, Bywater, Edwards, Hutchings, & Zou, 2013; Gardner, Burton, & Klimes, 2006; Sigmarsdóttir, Thorlacius, Guðmundsdóttir, & DeGarmo, 2015; Taylor, Schmidt, Pepler, & Hodgins, 1998).

Importantly, the existing literature on effectiveness of BPTs for children and parents living in poverty is limited by the fact that few studies have been conducted predominantly among low-income families, delivered within publicly funded routine clinical practice (e.g., treatment provided by existing practitioners rather than research staff), and offered to youth referred to treatment by their caregivers rather than those recruited for a particular study (Michelson, Davenport, Dretzke, Barlow, & Day, 2013; Weisz, Ugueto, Cheron, & Herren, 2013). Moreover, in routine community-based services, children typically receive multiple services concurrently (e.g., individual therapy and medication monitoring; Hutchings et al., 2007; Larsson et al., 2009; Pisterman et al., 1992), underscoring the importance of understanding the incremental benefit of an intervention within the context of possibly receiving additional services. Research that fails to address these key factors will have limited generalizability when these interventions are translated into routine, under-resourced service settings with poverty-impacted clientele. Moreover, studies often comparing evidence-based interventions with usual care among clinic-referred youth with serious, complex diagnoses typical of “real world practice” often have indicated nonsignificant findings (Weisz et al., 2013). Our review of the BPT treatment literature for child behavioral difficulties that also examined parental stress and depression as outcomes yielded only a handful of studies that reflected the challenges and realities of poverty-impacted families accessing routine, publicly available services (e.g., treatment provided by existing practitioners in publicly funded settings, treatment provided to predominantly low-income sample of referred youth and families, multiple services provided concurrently). We have discussed them in more detail below.

Efforts to integrate the Incredible Years parent training in the U.K. equivalent of Head Start, called Sure Start, among socioeconomically disadvantaged families in Wales, found significant reductions in child behavioral difficulties and parental stress, although mixed findings on parental depression when compared with families in the wait-list control groups (Charles et al., 2013; Hutchings et al., 2007). Larsson et al. (2009) examined the effectiveness of parent training in community-based child psychiatric clinics in Norway, reporting significant reductions in child conduct problems and parental stress for those in the experimental group versus wait-list control. However, a number of other studies have found no significant benefits of BPT on measures of parental stress and depression even when concluding that BPT reduced child behavioral difficulties (Gardner et al., 2006; Sigmarsdóttir et al., 2013; Taylor et al., 1998).

Notably, only two of these studies reviewed above compared BPT with services as usual (SAU; that is, Sigmarsdóttir et al., 2013; Taylor et al., 1998). Comparing BPT with a nonactive comparison condition like wait-list control is more likely to yield larger treatment effect sizes favoring the active treatment, as wait-list control conditions are meant to have no beneficial therapeutic effect and, thus by design, are weaker than the active treatment being tested. Conversely, usual care, which is designed to have a therapeutic benefit, is a stronger standard to test BPTs. In addition, efforts to improve mental health services must ultimately focus on the incremental benefit of novel innovative approaches compared with existing models rather than just to wait-list control or alternative, relatively less-utilized interventions/models. Given the substantial costs of implementing many evidence-based interventions within existing clinical practice (due to associated manuals, measures, training, supervision, etc.), the question of whether a novel approach actually provides added value to usual care becomes even more salient (Weisz, Ugueto, et al., 2013), particularly in resource-poor communities. To our knowledge, the incremental value of BPT over and above the benefit of SAU has yet to be tested regarding parental stress and depression.

The current study extends the existing literature base by examining the incremental benefit of an evidence-based multiple family group BPT program, called the 4Rs and 2Ss for Strengthening Families Program (4R2S; http://nrepp.samhsa.gov/ProgramProfile.aspx?id=41), to existing services in outpatient, community-based child mental health clinics. Importantly, this article reports on findings from an effectiveness study conducted with families predominantly impacted by poverty and within the context of routine community-based clinical settings (Chacko et al., 2015; Gopalan et al., 2015; McKay et al., 2011). Using a common elements approach (Chorpita & Daleiden, 2009; Garland, Hawley, Brookman-Frazee, & Hurlburt, 2008), 4R2S integrates essential practices of BPT and family therapy as well as factors known to affect service utilization, translated into six core intervention components known as the 4Rs and 2Ss: Roles, Responsibilities, Relationships, Respectful communication, Social support, and Stress. Specifically, 4R2S targets key family processes commonly addressed across evidence-based BPTs which have been successful in treating DBDs (e.g., consistent discipline, use of rewards/consequences to shape behavior, and enhanced quality of family organization, relationships, and communication), through a service delivery model that also combines group and family therapy strategies. As stress and lack of social support have been known to inhibit treatment engagement (McKay & Bannon, 2004), 4R2S further helps families identify and cope with sources of stress, as well as enhance their social support network (see “Method” section for more details).

The incremental value of 4R2S (4R2S + SAU) was recently evaluated in a large clinical effectiveness trial in comparison with SAU within community-based, publicly funded child mental health clinics for families with school-aged children (ages 7–11). Compared with participants in the SAU condition, findings of previous studies indicated 4R2S participants manifested significantly reduced levels of child behavior difficulties and increased social skills immediately post treatment (Chacko et al., 2015). Six months post treatment, 4R2S continued to demonstrate benefits over SAU in terms of reduced child behavioral difficulties, as well as diminished functional impact on children’s relationships with peers and overall severity of child behavior difficulties (Gopalan et al., 2015).

While 4R2S has been evaluated for child behavioral difficulties and functional outcomes, as well as the relationship between caregiver stress and depression to 4R2S attendance (Jackson, 2015), the incremental value 4R2S has not been evaluated regarding its impact on caregiver stress and depressive symptoms relative to SAU. Given the impact of caregiver stress and depressive symptoms on effective parenting, child functioning, and BPT treatment success, examining these outcomes is particularly important. Based on the prior research suggesting that BPT without enhancements may lead to lower parental stress and depression (e.g., DeGarmo et al., 2004), we hypothesize that caregivers in the 4R2S condition will report lower rates of parental stress and depressive symptoms at posttest and 6-month follow-up compared with participants in the SAU condition.

Importantly, we also explore whether the 4R2S incremental treatment effect differs based on whether caregivers enter the study with clinically significant (CS) levels of stress and depressive symptoms. Prior research has demonstrated that caregivers with initially high levels of stress and depressive symptoms are more likely (compared with those caregivers with normal levels) to demonstrate reductions in these outcomes resulting from BPT (e.g., DeGarmo et al., 2004; Jackson, 2015). As a result, we anticipate that individuals with CS levels of stress and depressive symptoms will demonstrate incremental benefits from 4R2S, whereas those caregivers eliciting non-CS levels of the same outcomes will reveal little to no effect of the 4R2S intervention. Similar to the main effects analyses, we examine the incremental benefit of 4R2S over SAU at posttest and 6-month follow-up, separated by CS status.

Method

Procedures

Institutional Review Board approval was obtained for this study. Recruitment occurred between October 2006 and October 2010 in 13 community-based outpatient mental health clinics serving families who were socioeconomically disadvantaged. Each site included in the study provided an array of mental health services to youth and families from neighborhoods in the New York City metropolitan area. Most often, families served at participating clinics identified as being of Latino or African American/Black descent. Clinic staff identified children between 7 and 11 years of age with behavior difficulties at intake and informed their adult caregivers about the study. Interested caregivers were then referred to research staff. The child’s legal guardian provided written informed consent and youth provided verbal assent. Families could not be consented where the legal guardian was not available (i.e., children residing in foster care) or where there was significant cognitive or psychiatric impairment (as determined by research staff upon face-to-face meeting) which prohibited understanding of the consent process or intervention components.

Eligibility criteria included English- or Spanish-speaking families with youth who were between the ages of 7 and 11 years old, and willing to be a part of treatment at one of the participating clinics. Among those families who met this initial eligibility criteria, caregivers completed the screening measure, the Disruptive Behavior Disorders Rating Scale (Pelham, Gnagy, Greenslade, & Milich, 1992), to determine whether the child met criteria for a diagnosis of ODD or CD as specified in the Diagnostic and Statistical Manual of Mental Disorders (5th ed.; DSM-5; American Psychiatric Association, 2013). Only families where youth met diagnostic criteria for ODD or CD were eligible to participate in the study.

Once determined eligible, participants were assigned to the 4R2S experimental condition (4R2S + SAU; n = 225) or SAU comparison condition (n = 95). Project coordinators utilized a 2:1 allocation ratio, block comparison design to ensure that there were a sufficient number of families to begin the 4R2S group at each participating clinic. Specifically, the first six to eight families where the youth screened in as eligible were assigned by research staff to 4R2S condition. Subsequently, the next three to four eligible youth and their families were allocated to the SAU treatment condition. As indicated by Dumville, Torgerson, and Hewitt (2006), threats to statistical validity are a concern only when the allocation ratio is 3:1 or greater. Because treatment conditions can be populated quickly and efficiently, block comparison designs like the one utilized in this study are frequently employed for health services trials with similar logistical constraints (e.g., Goodwin et al., 2001). To reduce bias, decisions regarding treatment condition assignment were not managed by field staff, rather by project coordinators who were removed from direct family contact. In addition, research staff and investigators were blind to youth and family profiles during allocation.

Participants

Of the 416 initially screened, 320 youth and their families were enrolled in the study. Prior studies examining the same data (Chacko et al., 2015; Gopalan et al., 2015) have reported 91 children did not meet diagnosis for CD or ODD, three children were considered not appropriate for treatment due to their pervasive developmental delay issues, and two caregivers declined to participate. The majority of caregiver participants identified as Latino (54%) or Black/African American (31%), were the primary caregivers of boys (68%), and reported a family income of less than US$30,000 (79%). Two thirds of families (n = 215) identified as having a single-parent household, and 70% (n = 223) received publicly funded health insurance. At baseline, caregivers in 4R2S were significantly younger (M = 35.72, SD = 8.39) than caregivers in SAU (M = 38.18, SD = 9.30, t = −2.29, df = 308, p = .02). There were no differential rates of attrition by treatment condition between participants at baseline, midtest, and posttest assessment. Six-month postintervention data were collected from 221 (69%) families. Of the total families at 6-month follow-up, a significantly greater number of families who received SAU (79%, n = 75) were included compared with families in the 4R2S condition (65%, n = 146; χ² = 6.18, df = 1, p = .01). A significant association between response status and ethnicity (χ² = 14.30, df = 5, p = .01) at 6-month follow-up was found, indicating that African American/Black caregivers were the most represented (79%), followed by Hispanic caregivers (68%). All families who identified as Native American responded (100%), while Caucasian/White caregivers responded the least (50%). It should be noted, however, that there were no differences on baseline demographic variables between treatment conditions among those participants who responded at the 6-month follow-up assessment.

4R2S

4R2S is a manualized, time-limited (16 weeks, 90–120 min per session) mental health service that targets school-age (7–11 years), children meeting diagnostic criteria for ODD or CD and their families, including adult caregivers, and siblings over 6 years of age. 4R2S typically involves six to eight families, and at least two generations of a family are present in each session. The multiple family group service delivery model allows for an in vivo focus on changing family interactions by keeping children and parents together. Moreover, as multiple families can be seen at one time, this service delivery model addresses the chronic shortages in qualified child mental health providers within poverty-impacted settings, which often results in long waiting lists for families to access treatment (Asen, 2002). In addition, 4R2S sessions are co-facilitated by teams of clinicians and family advocates (parents with the “lived” experience of caring for a child with emotional/behavioral difficulties and navigating the child mental health service system). Family advocates have been shown to deepen caregivers’ engagement in 4R2S (Gopalan, Acri, Lalayants, Hooley, & Einbinder, 2014) as feedback from peers with similar shared “lived” experiences is often perceived as more credible than suggestions offered by professionals (McKay, Gonzales, Stone, Ryland, & Kohner, 1995). In combination, the 4R2S model uniquely allows families the opportunity to validate each other strengths, normalize family struggles, and empower each other through mutual aid, thus augmenting family members’ commitment to treatment and maximizing change (McKay et al., 1995). Session content is delivered via a mixture of didactic and experiential activities. At each session, families are provided with child care for children 5 years old and younger, a meal, as well as transportation expenses. As indicated earlier, participants in the 4R2S condition were not restricted from obtaining other services within the participating site (e.g., medication management). Less than half of study participants (47%) received additional services, including individual therapy, medical management, school-based mental health, case management, and crisis management services. See Chacko et al. (2015) for a description of services received, facilitator training, fidelity, and attendance.

In this comparison condition, youth and their families received any service typically provided by the participating site. This included medication management, case management, as well as individual, family, or group therapy (see Chacko et al., 2015; Gopalan et al., 2015, for a full description of SAU services).

Measures

Depressive symptoms were measured using the Center for Epidemiologic Studies Depression Scale (CESD; Radloff, 1977). The CESD is a free and publicly available screening tool consisting of 20 items that assess the frequency of depressive symptoms occurring in the past week (0 = rarely/none of the time, 1 = some or a little of the time, 2 = occasionally or a moderate amount of time, 3 = most or all of the time). CESD scores are summed and range from 0 to 60, with a score of 16 and above considered CS (Zich, Attkisson, & Greenfield, 1990). The current study had good internal consistency with Cronbach’s alpha at baseline (.89), midtest (.90), posttest (.90), and 6-month follow-up (.91). For the current study, 63% (n = 200) of caregivers demonstrated CS CESD scores (≥16) at baseline.

Parenting stress in the parent–child system was assessed using the Parenting Stress Index–Short Form (PSI-SF; Abidin, 1995). Caregivers completed a 36-item questionnaire (e.g., “I often have the feeling that I cannot handle things very well”) with response options ranging from 1 (strongly disagree) to 5 (strongly agree). Possible scores on the PSI-SF range from 36 to 180, with higher scores indicating increased levels of parent stress. Cronbach’s alphas are reported at baseline, midtest, posttest, and 6-month follow-up, respectively: .91, .92, .94, and .94. For the current study, 69% (n = 221) of caregivers demonstrated CS scores (≥90) on the PSI-SF (Schaeffer, Alexander, Bethke, & Kretz, 2005).

Data Analysis

We used an intent-to-treat (ITT) analysis strategy to compare the treatment effect at posttest and 6-month follow-up. All analyses were conducted on Stata 14 (StataCorp, 2015) and utilized mixed-effects regression modeling recommended for analysis of longitudinal data with missing data (Gueorguieva & Krystal, 2004). Rather than deleting whole cases where there may be missing data on a single assessment point, mixed-effects regression includes cases in analyses as long as there is at least one nonmissing data point across all assessment periods. Furthermore, this method takes into consideration correlation between assessment periods within each case. All analyses allowed intercepts to vary randomly and utilized three levels of nesting: assessments (mid, post, 6-month follow-up) were nested within (a) individual participant IDs, which were nested within (b) family ID (22 families had more than one child in the 4R2S condition), which were then nested within (c) 4R2S group ID. Preliminary reliability analyses examined the level of clustering by clinic site for each outcome variable using reliability analysis. As all intraclass correlation coefficients (ICCs) were nonsignificant at all time points, with Cronbach’s alphas all close to zero, we concluded that there was no evidence of clustering by clinic site. As a result, we determined that it was not necessary to account for this level of nesting in analyses.

Main effects analyses

Main effects analyses included a dichotomous variable identifying the treatment condition (1 = 4R2S, 0 = SAU); dummy variables indicating the midtest, posttest, and 6-month follow-up assessment periods (baseline as a reference); as well as condition by assessment period dummy variable interactions (e.g., Condition × 6-Month Follow-Up). As indicated earlier, preliminary analyses indicated baseline differences in caregiver age by treatment condition. To determine if caregiver age impacted the treatment effect on outcome variables, we conducted preliminary sensitivity analyses including as covariates caregiver age and its interaction by all other model parameters. No systematic bias was detected, as no interaction coefficients of caregiver age by treatment condition were statistically significant. Consequently, we did not include caregiver age as a covariate for the final main effects analyses. After running multivariate models, linear contrasts were conducted to test for significant differences between 4R2S versus SAU at each assessment point (between-group differences). As recommended by Kraemer, Wilson, Fairburn, and Agras (2002), differences in treatment effects are best determined by combining the value of all parameters in the multivariate equation and subsequently comparing treatment effect sizes, as interpreting individual interaction parameters (e.g., Condition × Midtest) is considered insufficient. Finally, Cohen’s d effect sizes were calculated by dividing each linear contrast estimate with the pooled sample baseline standard deviation for that outcome variable.

Moderation analyses

As indicated in the “Introduction” section, examining the 4R2S treatment effect with all caregivers would include those who did not have any CS levels of stress and depression at baseline, thus likely resulting in minimal change over time on such outcomes. Consequently, moderation analyses examined the 4R2S treatment effect separated by caregiver CS status of stress and depression at baseline. Multivariate regression models examining the 4R2S treatment effect by CS status included all covariates from the main effects models as well as a dummy variable indicating CS status (1 = CS, 0 = not CS), and all interactions between the CS dummy variable, treatment condition, and assessment point. As a result, multivariate models for each outcome included two-way (Condition × Posttest, CS × Condition) and three-way interactions (CS × Condition × Post).

Preliminary analyses examined differences in the baseline levels of the outcome and demographic variables by treatment condition separately for CS and non-CS cases, as significant differences between 4R2S and SAU treatment condition within each CS status could bias final mixed-effects regression results. Among non-CS participants for the PSI-SF and the CESD, caregivers in the 4R2S condition were significantly younger than SAU caregivers (PSI-SF: t = −2.04, df = 96, p = .04; CESD: t = −2.15, df = 116, p = .03). In addition, non-CS SAU participants for the CESD were more likely to have a child identified as “Other” (e.g., not White/Caucasian, Black/African American, Hispanic/Latino, Native American, Asian/Pacific islander) race compared with 4R2S participants (χ² = 6.69, df = 1, p = .01).

Consequently, we conducted an additional set of sensitivity analyses utilizing the same procedure documented for the main effects models. Caregiver age and the child race “Other” categories were added as covariates along with their interactions with other model parameters for the relevant outcome variables (i.e., PSI-SF—Caregiver Age; CESD—Caregiver Age, Child Race—Other). No systematic bias was noted in these analyses for PSI-SF or CESD. Consequently, multivariate models examining moderation effects of CS status did not include caregiver age or child race as covariates. As recommended by Kraemer et al. (2002), computation of multivariate regression models were followed by linear contrasts to test for significant between-group differences at each assessment point (4R2S vs. SAU at posttest, 6-month follow-up). Linear contrasts for between-group analyses were conducted separately for CS and non-CS subgroups. Cohen’s d was calculated to determine treatment effect sizes for all contrasts.

Results

Table 1 presents means and SDs for all outcomes at each assessment point and by treatment condition. Multivariate regression parameters for main effects models are reported in Table 2. Despite the presence of significant interaction parameters between treatment condition and multiple assessment points (e.g., Condition × Posttest) for the PSI-SF and CESD in Table 2, the linear contrasts reported in Table 3 indicate that there were no significant between-group effects found for the PSI-SF or CESD at any assessment time point, with small treatment effect sizes reported throughout. Post hoc analyses of trends over time (not shown) indicate that 4R2S participants’ scores on all outcomes decreased from baseline to 6-month follow-up, while SAU participants’ scores either stayed constant or increased over time.

Table 1.

Means and SD of Outcomes by Treatment and Assessment Point.

	4Rs and 2Ss group (4R2S), n = 225 M (SD)				SAU, n = 95 M (SD)
Outcome	Baseline (n = 220)	Midtest (n = 174)	Posttest (n = 177)	6-month follow-up (n = 146)	Baseline (n = 95)	Midtest (n = 70)	Posttest (n = 83)	6-month follow-up (n = 75)
PSI-SF	100.05 (22.98)	93.52 (24.37)	88.88 (24.40)	86.72 (25.32)	95.41 (21.42)	94.67 (20.05)	94.46 (21.55)	93.04 (22.64)
CESD	19.84 (11.15)	18.52 (11.81)	17.95 (10.83)	16.82 (11.47)	18.09 (11.59)	16.51 (10.90)	19.20 (12.16)	19.62 (12.51)

Note. 4R2S = 4Rs and 2Ss for Strengthening Families Program; SAU = services as usual; PSI-SF = Parenting Stress Index-Short Form; CESD = Center for Epidemiologic Studies Depression Scale.

Table 2.

Multivariate Regression Main Effects Model Parameters for Outcome Variables.

	PSI-SF		CESD
Variable	b	SE	b	SE
Intercept	95.26	2.52***	18.13	1.24***
Condition	5.23	3.05	2.03	1.50
Midtest dummy	−0.08	2.33	−0.73	1.13
Posttest dummy	−0.70	2.16	0.85	1.05
6-month follow-up dummy	−1.74	2.20	1.43	1.09
Condition × Mid	−5.80	2.77*	−0.50	1.35
Condition × Post	−8.53	2.64**	−2.73	1.27*
Condition × 6-Month Follow-Up	−10.52	2.72***	−4.34	1.34**

Note. PSI-SF = Parenting Stress Index-Short Form; CESD = Center for Epidemiologic Studies Depression Scale.

p < .05. **p < .01. ***p < .001.

Table 3.

Linear Contrasts for Between-Group Main Effects.

			Multiple family group (4R2S)–SAU
Outcome variable	n	Assessment	Contrast estimate (b)	SE	Z	p	Effect size (Cohen’s d)
PSI-SF	223	Midtest	−0.57	3.31	−0.17	.863	0.03*
	236	Posttest	−3.30	3.19	−1.03	.301	0.15
	208	6-month follow-up	−5.29	3.27	−1.62	.105	0.23
CESD	224	Midtest	1.53	1.61	0.95	.342	0.14
	245	Posttest	−0.70	1.55	−0.45	.651	0.06
	207	6-month follow-up	−2.31	1.60	−1.45	.148	0.20

Note. 4R2S = 4Rs and 2Ss for Strengthening Families Program; SAU = services as usual; PSI-SF = Parenting Stress Index-Short Form; CESD = Center for Epidemiologic Studies Depression Scale.

p < .05.

Table 4 presents the multivariate regression parameters for moderation effects models for all outcomes. Significant interaction parameters for treatment condition, CS indicator, and assessment point were found for the CESD outcome, suggesting the potential moderating effect of baseline CS status on depressive symptoms at posttest. However, Table 5 presents linear contrast estimates for between-group moderation effects, with no significant between-group differences on the CESD at posttest. At the same time, Table 5 indicates that among participants who scored in the CS range on each outcome variable, 4R2S participants report significantly lower scores at 6-month follow-up on the PSI-SF and CESD compared with SAU participants, with medium effect sizes for both outcomes. No significant between-group differences were found for the non-CS participants on either outcome.

Table 4.

Multivariate Regression Moderation Effects Model Parameters.

	PSI-SF		CESD
Variable	b	SE	b	SE
Intercept	75.04	3.19***	7.95	2.21***
Condition	0.47	4.03	2.01	2.45
Midtest dummy	7.26	3.64	2.35	1.68
Posttest dummy	5.74	3.40	2.60	1.58
6-month follow-up dummy	2.11	3.44	3.40	1.58*
Condition × Mid	−3.39	4.43	0.52	2.01
Condition × Post	−8.26	4.23	0.82	1.94
Condition × 6-Month Follow-Up	−3.70	4.36	−2.44	1.98
CS dummy	32.86	4.04***	18.58	1.96***
Condition × CS	3.98	5.02	−1.86	2.35
Midtest Dummy × CS	−12.05	4.63**	−6.08	2.24**
Posttest Dummy × CS	−10.78	4.33*	−3.26	2.09
6-Month Follow-Up Dummy × Outcome	−6.39	4.39	−3.32	2.16
Condition × Midtest Dummy × CS	−2.11	5.59	−0.67	2.67
Condition × Posttest Dummy × CS	0.57	5.35	−5.47	2.54*
Condition × 6-Month Follow-Up Dummy × CS	−10.69	5.51	−3.07	2.66

Note. PSI-SF = Parenting Stress Index-Short Form; CESD = Center for Epidemiologic Studies Depression Scale.

p < .05. **p < .01. ***p < .001.

Table 5.

Linear Contrasts for Between-Group Moderation Effects.

			Clinically significant					Nonclinically significant
Outcome variable	n	Assessment	Contrast estimate (b)	SE	Z	p	Effect size (Cohen’s d)	Contrast estimate (b)	SE	Z	p	Effect size (Cohen’s d)
PSI-SF	223	Midtest	−1.05	3.50	−0.30	.765	0.05	−2.92	4.53	−0.64	.520	0.13
	236	Posttest	−3.24	3.38	−0.96	.338	0.14	−7.79	4.34	−1.79	.073	0.35
	208	6-month follow-up	−9.94	3.47	−2.86	.004**	0.44	−3.23	4.46	−0.72	.470	0.14
CESD	224	Midtest	−0.00	2.45	−0.00	.999	0.00	2.53	2.63	0.96	.335	0.22
	245	Posttest	−4.50	2.37	−1.90	.058	0.40	2.83	2.58	1.10	.272	0.25
	207	6-month follow-up	−5.36	2.46	−2.18	.029*	0.47	−0.43	2.61	−0.16	.870	0.04

Note. PSI-SF = Parenting Stress Index-Short Form; CESD = Center for Epidemiologic Studies Depression Scale.

p < .05. **p < .01. ***p < .001.

Discussion

Given the high prevalence of DBDs among children living in poverty (Bannon & McKay, 2007; Ghandour et al., 2012), and uneven findings regarding the impact of BPT alone upon parent-level outcomes for families living in poverty-impacted communities (Charles et al., 2013; Gardner et al., 2006; Hutchings et al., 2007; Larsson et al., 2009; Sigmarsdóttir et al., 2013; Taylor et al., 1998), this study examined the incremental effects of the 4R2S to decrease caregiver stress and depressive symptoms within a resource-poor, high-risk population-seeking services in routine clinical settings. Specifically, caregivers in the 4R2S treatment condition were hypothesized to have significantly improved levels of parental stress and depressive symptoms over time and in comparison with participants in the SAU condition. Importantly, we also explored whether the incremental treatment benefit of 4R2S differed based on whether or not caregivers entered the study with CS levels of stress and depressive symptoms. We hypothesized that individuals with CS levels of stress and depressive symptoms, in particular, would demonstrate significant 4R2S incremental treatment benefits compared with SAU.

Although main effects analyses manifested no significant differences between 4R2S and SAU participants’ PSI-SF or CESD scores, moderation effects analyses indicated beneficial between-group effects of 4R2S (vs. SAU) at 6-month follow-up for those caregivers who scored in the CS range at baseline on the PSI-SF and CESD. This is consistent with our a priori hypothesis that only those caregivers who demonstrated CS stress and depressive symptoms would demonstrate significant 4R2S incremental treatment benefits.

Among those previously reviewed studies with poverty-relevant contextual conditions, moderate to large treatment main effects sizes (d = 0.48–1.07) in favor of BPT were reported for parental stress and depression outcomes. However, none of these studies utilized SAU as control condition, evaluated the incremental benefit of BPT over SAU, nor examined the treatment effect among those caregivers with CS levels of stress and depressive symptoms. In our review, we were able to find one study that conducted analyses among participants with CS levels of depressive symptoms (Stattin et al., 2015), which found significant improvements for participants involved in three of the four BPT interventions (Comet, Cope, and Connect) in comparison with the wait-list control group, with effect sizes ranging from 0.23 to 0.30. Importantly, our study found larger effect sizes for participants with CS levels of stress and depressive symptoms (d = 0.44–0.47), in comparison with the more conservative control condition (SAU), and more accurately reflects the realities of poverty-impacted families utilizing publicly available community-based services.

Collectively, these findings align with other studies that suggest that interventions, largely derived from BPT, can have a significant impact on both child DBDs (e.g., Eyberg et al., 2008) and, perhaps equally important, parent-level outcomes (Chronis-Tuscano et al., 2013; Miller & Prinz, 1990; Rajwan, Chacko, Wymbs, & Wymbs, 2014). Importantly, the current study suggests that relatively large group intervention formats, such as 4R2S, can provide benefits for families in key areas that are putatively related to the persistence of DBDs. Unlike other group-based formats, 4R2S focuses on multiple family members working together within the group, allowing for greater efficiency in service delivery relative to group-based intervention that are parent- or child-only groups. The data suggest that the 4R2S format, while more complex, may lead to significant improvements for those caregivers who have CS levels of depressive symptoms and parental stress.

The clinical significance of the data should also be put in context. 4R2S is one among a handful of BPT-based interventions focused on school-age children and has undergone an effectiveness trial in resource-poor communities primarily with families of color. Importantly, participants in our study manifested numerous risk factors, owing to high rates of public health insurance as a proxy of poverty (70%), single-parent status (67%), as well as CS levels of depressive symptoms (63%) and parental stress (69%). Based on our review, few BPT effectiveness trials examining parental stress and depression as outcomes demonstrated relevant methodological features for high-risk, poverty-impacted families accessing routine, publicly available social services. Consequently, the results of the current study are among the few that highlight the potential incremental benefits of BPT for treating DBDs among school-age youth, and potentially caregiver stress and depression, in resource-poor communities with poverty-impacted populations.

In light of these findings, the study has several limitations suggesting the need for replication. Although attention was placed on reducing selection bias, the assignment to treatment condition was not randomized. In addition, 4R2S participants responded less at the 6-month follow-up assessment compared with SAU participants. This attrition raises concerns that those in the sample at 6-month follow-up may not be representative of the full sample at baseline. However, there were no significant baseline demographic differences between treatment conditions among those who did complete the 6-month follow-up assessment. Moreover, because the study sample is predominantly populated by ethnic minority participants, the response pattern at the 6-month follow-up assessment suggests that those participants remaining in the sample were representative of the target population. As a result, we concluded that the threat to sample representativeness due to attrition at the 6-month follow-up assessment was minimal. All families in the current study were self-referred to child mental health treatment and may be substantially different from those families who may not have overcome the stigma of seeking mental health treatment for their children. Finally, due to the exclusion criteria for this study, findings for this study may not generalize to younger and older children, as well as those youths with subthreshold levels of DBD symptoms.

Taken together, the results of this study suggest that 4R2S may have the potential to impact a substantial number of poverty-impacted, school-age children and families given the influence of caregiver stress and depression on child externalizing behaviors, as well as the risk that parental stress and depressive symptoms may undermine therapeutic child outcomes (Beauchaine, Webster-Stratton, & Reid, 2005; Chronis et al., 2004; Kazdin & Whitley, 2003). Future research should examine the difference in child behavior outcomes by CS status of caregiver stress and depression at baseline, as well as how reductions in stress and depressive symptoms through the course of 4R2S ultimately impact behavior among school-age children.

These findings suggest that clinicians working with children should be cognizant of how interventions for children may impact the emotional health of their caregivers. Ideally, practice-level recommendations include screening caregivers throughout treatment for their children for common mental health problems such as depression (Chacko et al., 2015; Rajwan et al., 2014), particularly in clinics where poverty-impacted families frequent. There is a fairly robust literature that shows screening programs, particularly in settings where caregivers are likely to attend, facilitate detection of untreated mental health problems (Acri et al., 2015; Smith et al., 2004). When putative parent-level issues (e.g., stress and depressive symptoms) are not sufficiently addressed through BPTs, enhancements with demonstrated effectiveness (e.g., Chacko et al., 2009), as well referring caregivers to receive their own mental health treatment, may be warranted (Rajwan et al., 2014). Ideally, colocating child and adult mental health services within a comprehensive care model addressing the needs of the entire family has the greatest potential for coordinating and streamlining care, decreasing logistical barriers to help-seeking, and ultimately improving child and family outcomes.

Footnotes

Authors’ Note

1. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institute of Mental Health (NIMH) or the National Institutes of Health.

2. Kyle O’Brien is now affiliated with Southern Connecticut State University, New Haven, CT.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Funding for this study was obtained through R01 MH072649 (principal investigator: McKay). Salary support for this study also came from National Institute of Mental Health (NIMH; F32MH090614; Gopalan). Dr. Gopalan is also an investigator with the Implementation Research Institute (IRI), at the George Warren Brown School of Social Work, Washington University in St. Louis, through an award from the NIMH (R25MH080916-01A2) and the Department of Veterans Affairs, Health Services Research and Development Service, Quality Enhancement Research Initiative (QUERI).

References

Abidin

R. R.

(1995). Parenting Stress Index (3rd ed., Professional manual). Odessa, FL: Psychological Assessment Resources.

Acri

M. C.

Frank

Olin

S. S.

Burton

Ball

J. L.

Weaver

Hoagwood

K. E.

(2015). Examining the feasibility and acceptability of a screening and outreach model developed for a peer workforce. Journal of Child and Family Studies, 24, 341–350.

American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). Arlington, VA: American Psychiatric Publishing.

Asen

(2002). Multiple family therapy: An overview. Journal of Family Therapy, 24(1), 3–16.

Bannon

W. M. J.

McKay

M. M.

(2007). Addressing urban African American youth externalizing and social problem behavioral difficulties in a family oriented prevention project. Social Work in Mental Health, 5, 221–240.

Barth

R. P.

(2009). Preventing child abuse and neglect with parent training: Evidence and opportunities. The Future of Children, 19, 95–118.

Beauchaine

T. P.

Hinshaw

S. P.

(Eds.). (2013). Child and adolescent psychopathology (2nd ed.). Hoboken, NJ: John Wiley.

Beauchaine

T. P.

Webster-Stratton

Reid

M. J.

(2005). Mediators, moderators, and predictors of 1-year outcomes among children treated for early-onset conduct problems: A latent growth curve analysis. Journal of Consulting and Clinical Psychology, 73, 371–388.

Braet

Meerschaert

Merlevede

Bosmans

Van Leeuwen

De Mey

(2009). Prevention of antisocial behaviour: Evaluation of an early intervention programme. European Journal of Developmental Psychology, 6, 223–240.

10.

Chacko

Gopalan

Franco

Dean-Assael

Jackson

Marcus

. . . McKay

(2015). Multiple family group service model for children with disruptive behavior disorders: Child outcomes at post-treatment. Journal of Emotional and Behavioral Disorders, 23, 67–77.

11.

Chacko

Wymbs

B. T.

Wymbs

F. A.

Pelham

W. E.

Swanger-Gagne

M. S.

Girio

. . . O’Connor

(2009). Enhancing traditional behavioral parent training for single mothers of children with ADHD. Journal of Clinical Child & Adolescent Psychology, 38, 206–218.

12.

Charles

J. M.

Bywater

T. J.

Edwards

R. T.

Hutchings

Zou

(2013). Parental depression and child conduct problems: Evaluation of parental service use and associated costs after attending the Incredible Years Basic Parenting Programme. BMC Health Services Research, 13, Article 523.

13.

Chorpita

B. F.

Daleiden

E. L.

(2009). Mapping evidence-based treatments for children and adolescents: Application of the distillation and matching model to 615 treatments from 322 randomized trials. Journal of Consulting and Clinical Psychology, 77, 566–579.

14.

Chronis

A. M.

Chacko

Fabiano

G. A.

Wymbs

B. T.

Pelham

W. E.

(2004). Enhancements to the behavioral parent training paradigm for families of children with ADHD: Review and future directions. Clinical Child and Family Psychology Review, 7(1), 1–27.

15.

Chronis-Tuscano

Clarke

T. L.

O’Brien

K. A.

Raggi

V. L.

Diaz

Mintz

A. D.

. . . Lewinsohn

(2013). Development and preliminary evaluation of an integrated treatment targeting parenting and depressive symptoms in mothers of children with attention-deficit/hyperactivity disorder. Journal of Consulting and Clinical Psychology, 81, 918–925.

16.

Coiro

M. J.

Riley

Broitman

Miranda

(2012). Effects on children of treating their mothers’ depression: Results of a 12-month follow-up. Psychiatric Services, 63, 357–363.

17.

Conger

R. D.

Conger

K. J.

Martin

M. J.

(2010). Socioeconomic status, family processes, and individual development. Journal of Marriage and Family, 72, 685–704.

18.

Deater-Deckard

(1998). Parenting stress and child adjustment: Some old hypotheses and new questions. Clinical Psychology: Science and Practice, 5, 314–332.

19.

Deater-Deckard

Dodge

K. A.

(1997). Externalizing behavior problems and discipline revisited: Nonlinear effects and variation by culture, context, and gender. Psychological Inquiry, 8, 161–165.

20.

DeGarmo

D. S.

Patterson

G. R.

Forgatch

M. S.

(2004). How do outcomes in a specified parent training intervention maintain or wane over time? Prevention Science, 5(2), 73–89.

21.

Dumville

J. C.

Torgerson

D. J.

Hewitt

C. E.

(2006). Research methods: Reporting attrition in randomised controlled trials. British Medical Journal, 332, 969–971.

22.

Ertel

K. A.

Rich-Edwards

J. W.

Koenen

K. C.

(2011). Maternal depression in the United States: Nationally representative rates and risks. Journal of Women’s Health, 20, 1609–1617.

23.

Evans

G. W.

(2004). The environment of childhood poverty. American Psychologist, 59(2), 77–92.

24.

Evans

S. W.

Owens

J. S.

Bunford

(2014). Evidence-based psychosocial treatments for children and adolescents with attention-deficit/hyperactivity disorder. Journal of Clinical Child & Adolescent Psychology, 43, 527–551.

25.

Eyberg

S. M.

Nelson

M. M.

Boggs

S. R.

(2008). Evidence-based psychosocial treatments for children and adolescents with disruptive behavior. Journal of Clinical Child & Adolescent Psychology, 37, 215–237.

26.

Forgatch

M. S.

Patterson

G. R.

(2010). Parent management training—Oregon model: An intervention for antisocial behaviors in children and adolescents. In Weisz

J. R.

Kazdin

A. E.

(Eds.), Evidenced-based psychotherapies for children and adolescents (2nd ed., pp. 159–178). New York, NY: Guilford Press.

27.

Gardner

Burton

Klimes

(2006). Randomised controlled trial of a parenting intervention in the voluntary sector for reducing child conduct problems: Outcomes and mechanisms of change. The Journal of Child Psychology and Psychiatry, 47, 1123–1132.

28.

Garland

A. F.

Hawley

K. M.

Brookman-Frazee

Hurlburt

M. S.

(2008). Identifying common elements of evidence-based psychosocial treatments for children’s disruptive behavior problems. Journal of the American Academy of Child & Adolescent Psychiatry, 47, 505–514.

29.

Ghandour

R. M.

Kogan

M. D.

Blumberg

S. J.

Jones

J. R.

Perrin

J. M.

(2012). Mental health conditions among school-aged children: Geographic and sociodemographic patterns in prevalence and treatment. Journal of Developmental & Behavioral Pediatrics, 33(1), 42–54.

30.

Goodwin

P. J.

Leszcz

Ennis

Koopmans

Vincent

Guther

. . . Hunter

(2001). The effect of group psychosocial support on survival in metastatic breast cancer. New England Journal of Medicine, 345, 1719–1726.

31.

Gopalan

Acri

Lalayants

Hooley

Einbinder

(2014). Child welfare involved caregiver perceptions of family support in child mental health. Journal of Family Strengths, 14(1), Article 5.

32.

Gopalan

Chacko

Franco

L. M.

Dean-Assael

K. M.

Rotko

Marcus

S. M.

. . . McKay

M. M.

(2015). Multiple Family Group service delivery model for youth with disruptive behaviors: Child outcomes at Six-Month follow-up. Journal of Child and Family Studies, 24, 2721–2733.

33.

Gueorguieva

Krystal

J. H.

(2004). Move over ANOVA: Progress in analyzing repeated-measures data and its reflection in papers published in the Archives of General Psychiatry. Archives of General Psychiatry, 61, 310–317.

34.

Hall

(2012). Current considerations of the effects of untreated maternal perinatal depression and the National Perinatal Depression Initiative. Journal of Developmental Origins of Health and Disease, 3, 293–295.

35.

Hutchings

Gardner

Bywater

Daley

Whitaker

Jones

. . . Scott

(2007). Parenting intervention in sure start services for children at risk of developing conduct disorder: Pragmatic randomised controlled trial. British Medical Journal, 334, Article 678.

36.

Jackson

J. M.

(2015). Multi-family groups for multi-stressed families: Initial outcomes and future implications. Research on Social Work Practice, 25, 537–548.

37.

Kazdin

A. E.

Siegel

T. C.

Bass

(1992). Cognitive problem-solving skills training and parent management training in the treatment of antisocial behavior in children. Journal of Consulting and Clinical Psychology, 60, 733–747.

38.

Kazdin

A. E.

Whitley

M. K.

(2003). Treatment of parental stress to enhance therapeutic change among children referred for aggressive and antisocial behavior. Journal of Consulting and Clinical Psychology, 71, 504–515.

39.

Kilgore

Snyder

Lentz

(2000). The contribution of parental discipline, parental monitoring, and school risk to early-onset conduct problems in African American boys and girls. Developmental Psychology, 36, 835–845.

40.

Kraemer

H. C.

Wilson

G. T.

Fairburn

C. G.

Agras

W. S.

(2002). Mediators and moderators of treatment effects in randomized clinical trials. Archives of General Psychiatry, 59, 877–883.

41.

Larsson

Fossum

Clifford

Drugli

M. B.

Handegård

B. H.

Mørch

(2009). Treatment of oppositional defiant and conduct problems in young Norwegian children: Results of a randomized controlled trial. European Child & Adolescent Psychiatry, 18(1), 42–52.

42.

Lovejoy

M. C.

Graczyk

P. A.

O’Hare

Neuman

(2000). Maternal depression and parenting behavior: A meta-analytic review. Clinical Psychology Review, 20, 561–592.

43.

Marchand

J. F.

Hock

Widaman

(2002). Mutual relations between mothers’ depressive symptoms and hostile-controlling behavior and young children’s externalizing and internalizing behavior problems. Parenting: Science and Practice, 2, 335–353.

44.

McKay

M. M.

Bannon

W. M. J.

(2004). Engaging families in child mental health services. Child and Adolescent Psychiatric Clinics of North America, 13, 905–921.

45.

McKay

M. M.

Gonzales

J. J.

Stone

Ryland

Kohner

(1995). Multiple family therapy groups: A responsive intervention model for inner city families. Social Work with Groups, 18(4), 41–56.

46.

McKay

M. M.

Gopalan

Franco

Dean-Assael

Chacko

Jackson

J. M.

Fuss

(2011). A collaboratively designed child mental health service model: Multiple family groups for urban children with conduct difficulties. Research on Social Work Practice, 21, 664–674.

47.

Michelson

Davenport

Dretzke

Barlow

Day

(2013). Do evidence-based interventions work when tested in the “real world”? A systematic review and meta-analysis of parent management training for the treatment of child disruptive behavior. Clinical Child and Family Psychology Review, 16(1), 18–34.

48.

Miller

G. E.

Prinz

R. J.

(1990). Enhancement of social learning family interventions for child conduct disorder. Psychological Bulletin, 108, 291–307.

49.

Munson

J. A.

McMahon

R. J.

Spieker

(2001). Structure and variability in the developmental trajectory of children’s externalizing behavior problems: Impact of infant attachment, maternal depressive symptomatology, and child sex. Development and Psychopathology, 13, 277–296.

50.

Pelham

W. E.

Gnagy

E. M.

Greenslade

K. E.

Milich

(1992). Teacher ratings of DSM-III-R symptoms for the disruptive behavior disorders. Journal of the American Academy of Child & Adolescent Psychiatry, 31, 210–218.

51.

Pisterman

Firestone

McGrath

Goodman

J. T.

Webster

Mallory

Coffin

(1992). The effects of parent training on parenting stress and sense of competence. Canadian Journal of Behavioural Science/Revue Canadienne Des Sciences Du Comportement, 24(1), 41–58.

52.

Radloff

L. S.

(1977). The CES-D scale: A self-report depression scale for research in the general population. Applied Psychological Measurement, 1, 385–401.

53.

Rajwan

Chacko

Wymbs

B. T.

Wymbs

F. A.

(2014). Evaluating clinically significant change in mother and child functioning: Comparison of traditional and enhanced behavioral parent training. Journal of Abnormal Child Psychology, 42, 1407–1412.

54.

Schaeffer

C. M.

Alexander

P. C.

Bethke

Kretz

L. S.

(2005). Predictors of child abuse potential among military parents: Comparing mothers and fathers. Journal of Family Violence, 20, 123–129.

55.

Scott

Knapp

Henderson

Maughan

(2001). Financial cost of social exclusion: Follow up study of antisocial children into adulthood. British Medical Journal, 323, 191–194.

56.

Siefert

Finlayson

T. L.

Williams

D. R.

Delva

Ismail

A. I.

(2007). Modifiable risk and protective factors for depressive symptoms in low-income African American mothers. American Journal of Orthopsychiatry, 77, 113–123.

57.

Sigmarsdóttir

Degarmo

D. S.

Forgatch

M. S.

Guðmundsdóttir

E. V.

(2013). Treatment effectiveness of PMTO for children’s behavior problems in Iceland: Assessing parenting practices in a randomized controlled trial. Scandinavian Journal of Psychology, 54, 468–476.

58.

Sigmarsdóttir

Thorlacius

Ö.

Guðmundsdóttir

E. V.

DeGarmo

D. S.

(2015). Treatment effectiveness of PMTO for children’s behavior problems in Iceland: Child outcomes in a nationwide randomized controlled trial. Family Process, 54, 498–517.

59.

Smith

M. V.

Cavaleri

M. A.

Howell

Poschman

Yonkers

K. A.

(2004). Screening for and detection of depression, panic disorder and PTSD in public-sector obstetrical clinics. Psychiatric Services, 55, 407–414.

60.

Spieker

S. J.

Larson

N. C.

Lewis

S. M.

Keller

T. E.

Gilchrist

(1999). Developmental trajectories of disruptive behavior problems in preschool children of adolescent mothers. Child Development, 70, 443–458.

61.

StataCorp. (2015). Stata statistical software: Release 14. College Station, TX: Author.

62.

Stattin

Enebrink

Özdemir

Giannotta

(2015). A national evaluation of parenting programs in Sweden: The short-term effects using an RCT effectiveness design. Journal of Consulting and Clinical Psychology, 83, 1069–1084.

63.

Taylor

T. K.

Schmidt

Pepler

Hodgins

(1998). Original research: A comparison of eclectic treatment with Webster-Stratton’s parents and children series in a children’s mental health center: A randomized controlled trial. Behavior Therapy, 29, 221–240.

64.

Weisz

J. R.

Kuppens

Eckshtain

Ugueto

A. M.

Hawley

K. M.

Jensen-Doss

(2013). Performance of evidence-based youth psychotherapies compared with usual clinical care: A multilevel meta-analysis. JAMA Psychiatry, 70, 750–761.

65.

Weisz

J. R.

Ugueto

A. M.

Cheron

D. M.

Herren

(2013). Evidence-based youth psychotherapy in the mental health ecosystem. Journal of Clinical Child & Adolescent Psychology, 42, 274–286.

66.

Yoshikawa

Aber

J. L.

Beardslee

W. R.

(2012). The effects of poverty on the mental, emotional, and behavioral health of children and youth implications for prevention. American Psychologist, 67, 272–284.

67.

Zich

J. M.

Attkisson

C. C.

Greenfield

T. K.

(1990). Screening for depression in primary care clinics: The CESD-D and the BDI. International Journal of Psychiatry in Medicine, 20, 259–277.