BPT for Early-Onset Behavior Disorders: Examining the Link Between Treatment Components and Trajectories of Child Internalizing Symptoms

Abstract

Behavioral Parent Training (BPT) is the standard of care for early-onset Behavior Disorders (BDs). Preliminary evidence suggests that BPT may also lead to improvement in comorbid symptomatology, particularly internalizing problems, in children with BDs, yet less is currently known about how BPT produces such cascading effects. To begin to address this gap in the literature, trajectory analyses were used to examine the link between treatment components of one mastery-based BPT program, Helping the Noncompliant Child (HNC), and child internalizing symptoms over the course of treatment. Findings revealed that parental use of the Attends skill (i.e., parental description of child activity with warmth and enthusiasm) over time was significantly associated with decreases in trajectories of child internalizing symptoms across treatment. Further probing of these effects revealed that parent use of average or above-average levels of Attends across treatment sessions led to significant reductions in child internalizing symptoms by Sessions 7 to 10 of treatment. Consistent with the movement toward a modular approach to the treatment of children, findings highlight the importance of identifying particular BPT skills that can be used in treatment to target multiple comorbid child symptom clusters. Clinical implications and future directions are discussed.

Keywords

Behavioral Parent Training internalizing symptoms attends trajectory analysis

Internalizing disorders (i.e., Anxiety and Mood Disorders) and externalizing disorders (i.e., Attention Deficit-Hyperactivity Disorder, Oppositional Defiant Disorder, Conduct Disorder) are the two most common mental health referrals for children (Merikangas, Nakamura, & Kessler, 2009). Moreover, early childhood (3-8 years) onset internalizing and externalizing problems often demonstrate chronic, stable trajectories within and across generations (e.g., Carpenter, Puliafico, Kurtz, Pincus, & Comer, 2014; Luby, Si, Belden, Tandon, & Spitznagel, 2009; Rothenberg, Hussong, & Chassin, 2018). If left untreated, these trajectories are associated with up to 10-fold increases in educational, health, and criminal justice costs before age 30 (Pelham, Foster, & Robb, 2007; Reinke, Eddy, Dishion, & Reid, 2012), seriously impacting employment opportunities and peer and family relationships (e.g., O’Connell, Boat, & Warner, 2009; Rothenberg, Solis, Hussong, & Chassin, 2017; Scott, Knapp, Henderson, & Maughan, 2001). In addition to the frequency and severity of these conditions, early childhood internalizing and externalizing disorders demonstrate significant comorbidity, as multiple studies have estimated that between 35% and 45% of all children diagnosed with an externalizing disorder also met criteria for an internalizing disorder (Armstrong, Lycett, Hiscock, Care, & Sciberras, 2015; National Institute of Mental Health [NIMH], 2015). Such comorbidity has been linked with maladaptive family relations, academic performance, treatment outcomes, and global functioning (Cummings, Caporino, & Kendall, 2014). Consequently, the NIMH has highlighted the importance of capitalizing on treatment approaches that effectively target both early-onset internalizing and externalizing symptom domains (NIMH, 2015).

One class of interventions that shows promise for treating such early-childhood comorbidities is Behavioral Parent Training (BPT) programs (see Forehand, Jones, & Parent, 2013; Kaehler, Jacobs, & Jones, 2016, for reviews). Essentially, BPT is a category of evidence-based interventions characterized by common behavior modification theory and associated treatment techniques. One set of BPT skills typically focuses on child behaviors that parents and other caregivers want to increase (i.e., appropriate child behaviors that parents want to see more of, including behaviors that demonstrate appropriate emotion regulation, compliance, prosociality, etc.). Using one BPT program, Helping the Noncompliant Child (HNC; McMahon & Forehand, 2003) as an example, this first set or phase of skills includes Attends (i.e., running commentary describing the child’s appropriate behaviors) and Rewards (i.e., physical touch-based attention such as giving a “high five” or hug, unlabeled verbal rewards such as “good job,” and labeled verbal rewards such as “thank you for . . .”). The last skill taught in the first phase of HNC is Ignoring. Notably, Ignoring is different from other Phase I skills as it is the first to focus on decreasing the probability of child inappropriate or negative-attention seeking behavior (e.g., whining) via the removal of verbal and physical attention (i.e., “no look, talk, or touch”). Phase II BPT skills build upon those learned in Phase I via parental use of Clear Instructions (to increase the probability of child compliance) and Time-Out (i.e., removal of positive attention as a consequence for noncompliance).

It is hypothesized that the aforementioned repertoire of BPT skills targets the coercive cycle of parent–child interaction implicated in the etiology and maintenance of externalizing disorders and, in turn, improves child behavior. Indeed, evidence-based BPT programs are considered standard-of-care interventions for early childhood externalizing disorders, with a robust and extensive literature demonstrating significant symptom improvement at both posttreatment and follow-up (e.g., Chorpita et al., 2011; Eyberg, Nelson, & Boggs, 2008; Kaehler et al., 2016; Kaminski & Claussen, 2017, for review). Although not an intended target of BPT interventions, emerging preliminary data also suggest that BPT may be effective in treating comorbid child internalizing symptoms (see Gonzalez & Jones, 2016, for a review). For example, both standard and modified BPT (i.e., including additional internalizing disorder-specific modules) programs have yielded greater reductions in a range of child internalizing symptomatology (e.g., separation anxiety, parent-rated child mood, parent-rated child internalizing symptoms) compared with wait-list controls (e.g., Carpenter et al., 2014; Chase & Eyberg, 2008; Webster-Stratton & Herman, 2008).

Chase and Eyberg (2008) demonstrated that a BPT used to treat 3- to 6-year-old (n = 15) children with comorbid Oppositional Defiant Disorder and Separation Anxiety Disorder could simultaneously reduce both disorders from pre- to posttreatment at clinically significant levels. Similarly, Carpenter and colleagues (2014) reviewed seven different modified BPT programs that built on Chase and Eyberg’s work by examining the effectiveness of BPTs in treating a range of early childhood internalizing problems. Collectively, these seven programs demonstrated preliminary evidence that BPTs utilized with children ages 2 to 8 can lead to reductions in Separation Anxiety Disorder, Social Anxiety Disorder, Generalized Anxiety Disorder, Specific Phobias, preschool depression, and general behavioral disinhibition symptoms (Carpenter et al., 2014). In turn, these investigations inform the current study in two ways. First, this prior literature provides support for the inference that BPTs can reduce internalizing symptoms in children experiencing comorbid clinically elevated externalizing symptoms (like children in the current study; Chase & Eyberg, 2008). Second, extant literature supports the inference that BPTs can be used to treat a wide range of internalizing symptoms in early childhood (i.e., depression and anxiety symptoms, which are both measured in the current study; Carpenter et al., 2014). Moreover, this study aims to extend these prior findings by examining what components of BPT programs (e.g., Phase I skills, Phase II skills, etc.) might bring about such improvements in internalizing symptoms. Therefore, we next review existing theories about why BPTs may be effective in reducing internalizing symptoms to postulate about how different BPT skills may improve such symptoms.

Some have hypothesized that such promising, albeit unintended, side effects of BPT may occur as a function of the skills that parents are learning in treatment and, in turn, the impact of those skills on the parent–child relationship (similar to how BPT shapes child externalizing behavior; Gonzalez & Jones, 2016). BPT programs were originally designed to treat child externalizing behavior and are efficacious in doing so (Kaehler et al., 2016). BPT programs ameliorate child externalizing behavior by positively reinforcing appropriate child behavior with positive parent attention and withdrawing such reinforcement when child behavior is inappropriate (through both Ignoring and parental use of the Time-Out sequence; Kaehler et al., 2016). In addition, BPT programs teach parents to deliver such positive parental attention within the context of play (i.e., parents are initially taught to begin practicing rewarding their child as they play alongside their child for 15 min every day, and then taught to reward appropriate child behavior throughout the day; Kaehler et al., 2016; McMahon & Forehand, 2003). Therefore, BPT programs also build warmth in the parent–child relationship through play-based interactions, ensuring parent positive attention is even more reinforcing (Gonzalez & Jones, 2016). This combination of the development of a warm parent–child relationship and parental use of differential attention (positive attention for appropriate behaviors, no attention for inappropriate behaviors) leads to rapid decreases in child externalizing problems as a result of BPT programs (Kaehler et al., 2016).

Building upon this more established literature, BPT tenets (i.e., developing warm parent-child relationships and using differential attention) also hold promise for ameliorating child internalizing problems (e.g., Chase & Eyberg, 2008). Specifically, when faced with insensitive or unresponsive parenting, children may learn to withdraw from parent–child interactions (Tronick & Gianino, 1986). This learned behavior is negatively reinforced because children learn that withdrawal leads to avoidance of an aversive stimulus (i.e., insensitive/unresponsive parenting; Gonzalez & Jones, 2016). Yet such withdrawal also leads to the emergence of internalizing symptoms over time, as a result of the consequent parent–child relational insecurity, lack of social support, and lack of interaction the child receives (Rothenberg et al., 2018).

Consistent with Levine and Ducharme’s (2013) work, Phase I HNC Skills (i.e., Attends and Rewards) may serve as an establishing operation that encourages child approach-oriented, as opposed to withdrawing, behaviors (Levine & Ducharme, 2013). Specifically, parental use of Attends and Rewards increases the likelihood that a child will begin to approach their parents once again by enhancing the value of parent–child interactions (Levine & Ducharme, 2013). Specifically, value is enhanced by teaching parents how to make such interactions warm and sensitive (Levine & Ducharme, 2013). As the child begins to approach, instead of withdraw, such behavior should become further reinforced as the child experiences new support, warmth, and safety from their parents (Gonzalez & Jones, 2016). Consequently, child anxiety, depression, and other internalizing symptoms related to child withdrawal from the parent–child relationship should then begin to dissipate (Gonzalez & Jones, 2016). In addition, Ignoring, as well as Phase II skills (i.e., Clear Instructions and Time-Out) may be especially effective in reducing internalizing symptoms because they establish more effective rules, expectations, and consequences for child behavior and consequently reduce parental hostility. By implementing both Phase I and Phase II skills, parents are responding to their children in more supportive and consistent, yet firm ways rather than in confrontations characterized by hostility, which may present as salient daily stressors that increase the child’s internalizing symptoms (Gonzalez & Jones, 2016). In sum, the same combination of BPT-taught skills posited to decrease child externalizing behaviors may also decrease child internalizing behaviors (Carpenter et al., 2014; Chase & Eyberg, 2008).

In support of these hypotheses, a related, albeit basic rather than applied line of research examining parenting more generally suggests that increased structure and consistency, and decreased parental hostility, have been associated with decreases in a range of internalizing symptoms (including low self-worth, hopelessness, and worthlessness) in prior studies (Burge & Hammen, 1991; Downey & Coyne, 1990; Ge, Best, Conger, & Simons, 1996; Gonzalez & Jones, 2016). Yet the interrelationship of the Attends, Rewards, and Ignoring skills in particular has not been examined in the context of BPT specifically. Accordingly, this study aims to build upon prior preliminary research (e.g., Carpenter et al., 2014; Chase & Eyberg, 2008) by characterizing the course of child internalizing symptomatology over the course of BPT treatment sessions. While studies to date have focused on BPT effects on child internalizing symptoms at posttreatment alone, we aim to take this research a step further by tracking trajectories of symptoms session-by-session from pre- to posttreatment. This approach affords the opportunity to explicitly link parent use of specific BPT skills with changes in child internalizing symptoms, which can both (a) shape clinician and client expectations concerning the ebb and flow of internalizing symptoms in the context of BPT and (b) provide clinicians with a better understanding of which BPT skills may be most effective in combating child internalizing symptoms. In addition, this approach informs knowledge of intervention cost-effectiveness (e.g., if BPT reduces internalizing symptoms in just a few sessions, costs savings can be calculated and used as a tool to advocate for greater use of BPT; NIMH, 2015). It is for these reasons that recent reviewers of BPT literature (Gonzalez & Jones, 2016) and government agencies (NIMH, 2015) have called for intensive, session-by-session outcome-tracking.

Building upon the aforementioned literature, this study examines the extent to which completion of one BPT program, HNC, is associated with reductions in child internalizing symptoms in families with low income and 3- to 8-year-old children who have clinically significant problem behavior. Of note, families from lower income backgrounds are more likely to have a child with externalizing and internalizing symptoms, yet less likely to engage in treatment, circumstances attributed in large part to economic and family stressors (see Jones et al., 2013 for a review). Accordingly, understanding the nature of the potential treatment effects of BPT for child internalizing symptoms holds particular promise for this at-risk and underserved group. As such, in the present analyses we (a) measure session-by-session child internalizing symptoms in hopes of identifying how internalizing symptoms change over treatment and (b) examine which BPT skills (Attends, Rewards, Ignoring, Phase II skills [e.g., Clear Instructions, child compliance]) are significantly associated with changes in such child internalizing trajectories. In answering these research questions, we control for other parent (i.e., education, baseline depressive symptoms) and child (i.e., age, gender, race, baseline externalizing symptoms) factors that may account for treatment effectiveness.

Method

Overview

The current secondary analyses use data collected from a study examining the efficacy of the aforementioned HNC (McMahon & Forehand, 2003) BPT program for improving engagement in services among families from low-income backgrounds (for further detail on engagement-related findings, see Jones et al., 2014). Inclusion for the study required that the family income be no more than 200% above the federal poverty level. The family also needed to have a child between 3 and 8 years old who had clinically significant problem behavior (i.e., an early-onset behavior disorder; see Table 1 for greater detail). Families were excluded if the parent had a current substance abuse/dependence diagnosis, psychotic disorder diagnosis, or severe depression/manic episode, or if the child had a disability so severe that BPT would not be an appropriate treatment course. The institutional review board approved all study procedures. Consent was obtained for parental participation and participation of the target children as well.

Table 1.

Eligibility Criteria and Demographics of Sample at Baseline Assessment (n = 62).

Eligibility criteria
Inclusion		Exclusion
1. Income ⩽ 200% of the poverty limit		1. Child had a development disability
2. Child between ages 3 and 8 years		2. Caregiver had current substance use, severe mood, or psychotic disorder diagnosis
3. Child met DSM-IV ODD, CD, or ADHD Diagnosis OR had score on the ECBI Intensity Scale > 131
4. Child and caregiver speak English		3. Current DSS/CPS involvement
Measure	%	M	SD
Child demographics
Age (years)		4.79	1.27
Gender (% male)	49.43
Race
White	65.58
Black or African American	21.86
Multiracial	12.56
Caregiver demographics
Age (Years)		31.94	5.88
Gender (% female)	100
Race
White	68.03
Black or African American	23.49
Multiracial	8.48
Marital status
Single	30.19
Married/engaged/common law	54.71
Divorced/separated	15.09

Note. DSM-IV = Diagnostic and Statistical Manual of Mental Disorders (4th ed.; American Psychiatric Association, 1994); ODD = Oppositional Defiant Disorder; CD = Conduct Disorder; ADHD = Attention-Deficit Hyperactivity Disorder; OR = odds ratio; ECBI = Eyberg Child Behavior Inventory; DSS/CPS = Department of Social Services/Child Protective Service.

Participants

Participants were 62 children (3-8 years old) from families with low income and their primary caregiver (see Table 1). Participants were recruited via advertisements and flyers distributed on public transportation systems, and in nonprofit organizations, community apartment complexes, and local schools and also via referrals from social service providers at local nonprofit and community organizations.

Procedure

Interested families completed a phone screen to ensure that they met eligibility criteria for study participation (see Table 1). If phone screen eligible, families attended a baseline assessment session in which eligibility was confirmed and parents completed a variety of psychosocial measures, including those examined in the current study. After the baseline assessment, families began treatment (as outlined below). Families that completed treatment reported on their continued functioning in a postassessment conducted within 1 week of treatment completion, as well as at 3- and 6-month follow-up assessments. Baseline and postassessments were similar in content. Families were compensated US$50 per assessment, sessions themselves were free, and all study and treatment procedures were approved by the appropriate institutional review board. Child care was provided to nonparticipating siblings as requested.

Treatment

Consistent with Hanf Model BPT programs (see Kaehler et al., 2016), HNC (McMahon & Forehand, 2003) is a two-phase, individual family-focused mastery-based program. Weekly sessions (60 min) are active and include psychoeducation, in vivo skill practice, modeling, and coaching, and ongoing assessment of skill progress and mastery. During Phase I, Differential Attention, caregivers learn Attends (i.e., running commentary of child play), Rewards (i.e., verbal or physical praise), and Ignoring (i.e., removal of verbal or physical attention) in the context of Child’s Game (i.e., child directed play). Caregivers are also asked to practice Child’s Game for 15 min with their child outside of session at home on a daily basis. Child’s Game serves two functions: (a) it provides caregivers with an opportunity to practice the skills learned in session and (b) it promotes daily positive one-on-one time between the caregiver and child. During Phase II, Compliance Training, caregivers learn effective discipline strategies (i.e., Clear Instructions) and consequences procedures (i.e., Time-Out) in the context of “Parent’s Game” (i.e., parent-directed play). Although caregivers do not practice Parent’s Game outside of session, they are instructed to practice and use the Phase II skills to manage child behavior at home. Consistent with the mastery-based nature of the program, advancement to the subsequent skill or phase is dependent on caregivers meeting specific behavioral performance criteria for the prior skills during structured play (i.e., Child’s Game or Parent’s Game), which is observed and coded by therapists during each session (see McMahon & Forehand, 2003, for additional information on the selection of this criteria). Therapists and families check in mid-week each week for a brief telehealth consultation regarding skill use and problem-solving obstacles to use. Treatment length varies based on the rate of caregiver’s skills acquisition, which therapists code in session to determine when mastery of each skill is met. Once caregivers master one skill, they progress to the next skill until they ultimately complete the program. On average, treatment lasts between 8 to 12 sessions (M = 8.14, SD =3.99, range = 1-16 in current sample).

Measures

Primary Study Measures

Behavioral observation of BPT skills

Consistent with standard HNC protocol, behavioral observation of parent use of the HNC skills (i.e., Attends, Rewards, Ignoring, Clear Instructions, and Time-Out) was conducted in the context of each therapy session (i.e., therapist coded skill use to determine parent mastery of skill and progression in program). Therapists began to code parent skills starting in the second session (where Attends, the first Phase I skill is taught). Attends (positive attention in which the parent provides an ongoing verbal description of what the child is doing; e.g., “you are stacking the blocks”), Rewards (positive attention that is provided following the child’s appropriate behavior; e.g., “Great job picking up your toys!”), and Ignoring (i.e., “no look, talk, or touch” in presence of predetermined inappropriate child behavior) were coded in the context of Child’s Game (i.e., child-directed play). Clear Instructions (instructions that gain the child’s attention, are specific, are issued one at a time, and are not phrased as “let’s . . .” statements or questions) and Time-Out (i.e., 3 minutes without verbal or physical attention in response to noncompliance or other not appropriate behavior) were coded in the context of Parent’s Game (i.e., parent-directed play).

All therapists were trained in the HNC Behavioral Observation Coding System (McMahon & Forehand, 2003). Prior to beginning coding, therapists reached agreement on all Child’s and Parent’s Game coding criteria at 80% or higher on prerecorded tapes, compared with expert rankings. In addition, therapist fidelity to coding and treatment procedures was found to be greater than 95% based on weekly review of each therapists’ taped treatment sessions. Importantly, every session conducted by every therapist during treatment was coded for fidelity on a weekly basis by at least one, and sometimes two, licensed clinical psychologists who are considered expert HNC trainers certified to teach and disseminate the HNC program to practitioners nationwide. The two HNC expert trainers who reviewed tapes and provided weekly supervision had over 60 years of combined clinical experience implementing the HNC program with families. Behavioral observations used in the current study were captured from a 3-min observation of parent skills conducted during each session and described below. All means and percentages reported below are derived from parent behaviors demonstrated during these 3-min observations.

Attends and Rewards

Measured by the average number of caregiver Attends (M = 6.14 attends/min, SD = 2.38 attends/min) and Rewards (M = 2.55 rewards/min, SD = 0.95 rewards/min) behaviors observed per minute by the therapist. The caregiver must demonstrate at least 4 Attends and at least 2 Rewards over the 3-min observation to qualify as having “mastered” these skills.

Ignoring

Measured by the percentage of inappropriate child behavior ignored by the caregiver (M = 95.13% of inappropriate behavior ignored, SD = 13.58%). The caregiver must ignore at least 70% of child inappropriate behavior during the 3-min observation to qualify as having “mastered” the skill.

Clear Instructions

Consistent with HNC mastery coding criteria, frequency of instructions and proportion of instructions that were clear were both measured. An instruction was considered to be clear if it demonstrated each of the following characteristics: specific (e.g., “please put the block in the box,” as opposed to “please clean up”); one-at-a-time (e.g., “Put the red block in the box. [Child obeys]. Now put the blue block in the bag,” instead of “put the red block in the box, then the blue block in the bag and then the green block upstairs”); rationale first (e.g., “Its almost time to go to school, so please get in the car,” instead of “Get in the car because it’s almost time for school”); free of “let’s” or “we” statements (e.g., “Please put the block away” instead of “We should put the block away”); and free of questions (e.g., “Please put the block away” instead of “Can you put the block away?”). All therapists had to distinguish between clear and unclear commands at >80% accuracy compared with expert rankings and according to the empirically supported and clinically valid HNC Behavioral Observation Coding System criteria (McMahon & Forehand, 2003). The first indicator used in the current study was the average number of parental Clear Instructions issued per minute (M = 3.19 clear instructions/min, SD = 1.18 clear instructions/min). The second indicator used was the percentage of all parental instructions issued that were considered “clear” (M = 95.92% of parent instructions are clear, SD = 8.84%). The caregiver must provide at least 2 Clear Instructions, and 75% of all caregiver instructions must be considered “clear” over the 3-min observation for a caregiver to qualify as having “mastered” the skill.

Child Compliance

Notably, Time-Out did not occur often enough across sessions to be used as a distinct category of skills. Importantly, the lack of Time-Outs is less likely a result of children demonstrating low levels of disruptive behavior (i.e., clinically significant levels of problem behavior were necessary for study eligibility). Rather, we suspect Time-Outs were infrequent because they were the very last skill taught to parents, who therefore had already mastered Phase I skills and Clear Instructions and utilized those skills to ameliorate most child disruptive behavior. Therefore, in an attempt to measure the effectiveness of all Phase II skills (including Time-Out), we included in the current analyses an indicator of overall Child Compliance during each Phase II session of treatment. We defined Child Compliance as the percentage of all parent Clear Instructions the child complied with within 5 seconds after a command was issued in a given observation (M = 93.45% child compliance, SD = 10.63%). Though not necessarily an indicator of parental mastery of a specific skill, we included child compliance in analyses as an overall indicator of the effectiveness of Phase II skills (including Time-Out) because the more effective parental Clear Instructions and Time-Outs are, the greater percentage of Clear Instructions the child is expected to comply with to avoid going to Time-Out. The child must comply with at least 75% of parental Clear Instructions for the family to qualify as having “mastered” Phase II skills. Therefore, though not specifically a skill for parents to master, Child Compliance was still a skill the family unit as a whole had to master before treatment was completed, and thus an important skill to include in the current analyses.

Child internalizing symptoms

Parents reported on child internalizing symptoms utilizing the seven-item internalizing subscale (e.g., “Feels worthless or inferior,” “Too fearful or anxious”) from the Brief Problem Checklist (BPC; Chorpita et al., 2010). The BPC asked parents to rate symptoms over the past week utilizing a scale from 0 = not true to 2 = very true. The BPC was developed to assess clinical progress over the course of treatment and has demonstrated excellent psychometric properties and validity (Chorpita et al., 2010). The developers of the BPC specifically designed the Internalizing subscale to be used weekly and to be sensitive to small changes in behavior (Chorpita et al., 2010). Authors empirically demonstrated this weekly sensitivity by utilizing random-coefficient trajectory models to estimate week-to-week slope reliabilities (Chorpita et al., 2010). The BPC was found to have higher slope reliabilities (i.e., sensitivity to week-to-week change) than other commonly used measures of child mental health (i.e., the Child Behavior Checklist and Youth Self-Report Scale), leading the authors to conclude that the BPC is an effective tool for ongoing assessment of week-to-week clinical progress in internalizing symptoms over the course of treatment (Chorpita et al., 2010). Therefore, the BPC internalizing scale was utilized in the current study due to the exact match between the time frame (week-to-week) and analytic methodology (trajectory analysis) used in the current study and in the study wherein the BPC was developed.

In the current study, the BPC Internalizing scale was found to demonstrate strong internal consistency across weeks (α = .78). The BPC Internalizing scale was also found to be sensitive to week-to-week changes in internalizing symptoms based on preliminary trajectory analyses, as a linear growth curve model (which assumes there is week-to-week variability in BPC internalizing symptoms) was found to fit the current data significantly better than a random intercept-only model, which assumes that the BPC is not sensitive to week-to-week internalizing symptom variability; χ²(2) = 25.5, p < .01. Therefore, it appears the BPC is a psychometrically sound and appropriately sensitive measure to utilize in the current study. The BPC was administered at baseline and post assessments and between every therapy session via phone (M = 0.36, SD = 0.45). An average of the seven-item subscale was calculated for each time point at which the BPC was administered. Then, these time-specific averages were utilized to estimate trajectories of child internalizing behavior. Although not recruited based on internalizing symptoms, children in the current sample nevertheless demonstrated high risk for experiencing internalizing symptoms at the beginning of treatment. Specifically, at the beginning of treatment 30% of children in the current sample demonstrated scores as high or higher than children who had diagnosed Internalizing Disorders in the BPC standardization sample (Chorpita et al., 2010). Similarly, a full 76% of the current sample demonstrated scores in the “at-risk” range for clinically elevated Internalizing problems (i.e., scored within one standard deviation of children who had diagnosed Internalizing Disorders in the BPC standardization sample; Chorpita et al., 2010). Therefore, it appears the current high-risk sample is appropriate for studying the effects of parent-learned HNC skills on clinically significant trajectories of child internalizing problems.

Covariate Measures

Baseline assessment scores for each of the following constructs were controlled in study analyses.

Child externalizing symptoms

Child baseline externalizing symptoms were measured using the Eyberg Child Behavior Inventory (ECBI; Eyberg & Robinson, 1983). The ECBI is 36-item parent-report inventory that measures common problem behaviors, such as noncompliance, aggression, and whining, in children and adolescents (2-16 years). The ECBI Intensity Scale is utilized in our analyses. The ECBI Intensity Scale measures the frequency with which the child engages in each of 36 behavioral problems on a scale of 1 (never) to 7 (always) by summing the 36 items, with higher scores indicating greater problems. Children rated at or above a score of 131 on the ECBI Intensity Scale are considered to have scores in the clinical range. The ECBI has well-established psychometrics with low-income samples (e.g., Fernandez, Butler, & Eyberg, 2011). In the current study, baseline ECBI Intensity Scale score was M = 157.84, SD = 30.21, α = .84.

Parent depressive symptoms

Parent baseline depressive symptoms were measured using the Beck Depression Inventory-II (BDI-II; Beck, Steer, & Brown, 1996). The BDI-II is a 21-item measure of caregiver self-reported depressive symptoms. Each item contains four statements reflecting varying degrees of symptom severity (range from 0 to 3). Items are summed to obtain a total score, with higher scores indicating more severe depressive symptoms. The BDI-II has been validated with several populations including healthy adults, primary care patients, and people with Major Depressive Disorder (MDD) (Steer, Ball, Ranieri, & Beck, 1999). In the current study, baseline BDI-II score was M = 8.03, SD = 7.73, α = .91.

Demographic variables

Child age, gender, race (0 = White, 1= Black or African American, 2 = Multiracial), and family treatment completion status (0 = Did not complete treatment, 1 = Completed treatment) were also controlled for in the analyses.

Results

Hierarchical linear models (HLM) were estimated using restricted maximum likelihood estimation procedures in the PROC MIXED function in SAS 9.3 to answer research questions. Analyses used data for 62 families who were followed over a total of 613 observed time points. Therefore, these analyses incorporated both baseline parent skills and child internalizing symptoms assessed at the beginning of treatment and parent skills and child internalizing problems demonstrated in each session over the course of treatment. Analyses were conducted within an Intention-to-Treat (ITT) framework (Gupta, 2011), meaning that all families who entered the intervention, including the 14 families who dropped out before treatment completion, were included in all study analyses. Utilizing an ITT framework ensures that treatment effects that emerge from analyses reflect treatment effects in the “real-world” and avoid inflation of effects that may occur if only treatment completers are included in analyses (Gupta, 2011).

To better understand missingness in noncompleter data, we conducted missing data analyses comparing treatment completers to noncompleters on pertinent study variables. Results revealed that families that did not complete treatment were more likely to have children who were boys, t(609) = −2.00, p = .05, and non-White, t(609) = −2.55, p = .01, than families that completed the treatment. There were no significant differences in child age, child baseline ECBI scores, or parent baseline BDI-II scores between treatment completers and noncompleters. With regard to parenting behaviors, noncompleters did not significantly differ from completers in use of Attends, t(609) = 1.82, p = .07; M_{Noncompleters} = 5.59 attends/min, M_Completers = 6.28 attends/min, or Rewards per session, t(609) = 0.37, p = .71; M_{Noncompleters} = 2.48 rewards/min, M_Completers = 2.57 rewards/min. Noncompleters typically terminated treatment before Ignoring, Clear Instructions, and Time-Out were taught. As a result, data are largely missing for noncompleters on these skills and comparisons with completers could not be conducted. With regard to child internalizing behavior, children from noncompleting families did not differ from children from completing families in terms of child BPC internalizing symptoms, t(609) = −1.56, p = .12; M_{Noncompleters} = 0.41, M_Completers = 0.34. Taken together, these results indicate that demographic, parenting, and child behavioral data were missing as a function of an already observed variable, namely, treatment completion status (Curran & Bauer, 2011). Therefore, following convention in the HLM literature, we included treatment completion status as a covariate in all analyses and utilized maximum-likelihood estimation procedures to account for systematic missingness in study variables (Curran & Bauer, 2011). See “Discussion” section for further consideration as to why completers and noncompleters did not differ on some variables.

Modeling Trajectories of Child Internalizing Behavior

One objective of the current study was to characterize the course of child internalizing symptoms across BPT sessions. To answer this question, we estimated a series of models to identify the functional form of child internalizing symptoms over the course of treatment. Following established conventions in the HLM literature (e.g., Curran & Bauer, 2011), we compared three competing models: a random intercept-only model (i.e., differences in intercepts across families with no random slope estimated), a linear growth curve model with a random intercept and linear random slope, and a quadratic model with a random intercept, linear random slope, and quadratic random slope. Results of a likelihood ratio test revealed that the linear growth curve model fit the data significantly better than the random intercept-only model, χ²(2) = 25.5, p < .01, and that addition of the quadratic slope term did not significantly improve model fit, indicating that a linear growth curve model best captured change in child internalizing symptoms. The linear growth curve model had a significant intercept (α = 0.47, p < .01) and significant negative slope (β = −0.02, p < .01). Therefore, on average, caregiver-reported child internalizing behavior at the start of treatment was 0.47 units (i.e., between “Not True” and “Somewhat True” that their child demonstrated a particular internalizing symptom on the BPC in the past week), and child internalizing behavior decreased modestly across treatment sessions at a rate of 0.02 units per session.

Identifying Parenting Skills That Alter Child Internalizing Trajectories

Next, a series of conditional hierarchical linear models were estimated to meet the second objective of the current study, which was to identify the BPT skills most effective in reducing child internalizing symptoms over the course of treatment. The first of these models tested whether any demographic or behavioral control variables predicted changes in the intercept or slope of child internalizing behavior. Child age, gender, race, externalizing behavior, parent depressive symptoms, and family treatment completion status did not significantly predict the intercept or slope. Therefore, in the interest of parsimony, all covariates except treatment completion status were trimmed from the analyses.

The second of these models (see Table 2) tested whether any skills caregivers acquired in each session over the course of treatment predicted changes in the intercept or slope of child internalizing behavior after controlling for a family’s treatment completion status. In this model, caregiver mastery of Attends (as measured by average attends per minute), Rewards (as measured by average rewards per minute), Ignoring (as measured by percentage of inappropriate child behavior ignored), Clear Instructions (as measured by average clear instructions per minute and percentage of parent-issued instructions that are clear), and Child Compliance (as measured by percentage of clear instructions child complies with) were each examined as predictors of child internalizing behavior intercept or slope. As can be seen in Table 2, only caregiver’s Attends (measured during Child’s Game) significantly predicted differences in intercept and slope. Specifically, each one additional attend/min that parents demonstrated at the beginning of treatment was associated with a 0.03-unit increase in child internalizing symptoms at the beginning of treatment. Though, because these measurements occurred contemporaneously, this finding could also be interpreted such that parents of children with higher internalizing symptoms as treatment began also demonstrated greater Attends/min as treatment began. In addition, effects on slope reveal that the linear slope of the trajectory of child internalizing behavior over time decreased by approximately 0.01 units for every additional attend/min that caregivers provided per session.

Table 2.

Effects of Parenting Skills Learned in Treatment on Child Internalizing Trajectories.

Construct	Parameter	β	SE
Predictors of intercept
Parent Attends^a	Attends/min	.03*	.01
Parent Rewards^a	Rewards/min	−.09	.06
Parent Ignoring^a	% of Inappropriate Child Behavior Ignored	.00	.00
Parent Clear Instructions^b	Clear Instructions/min	−.08	.06
Parent Clear Instructions^b	% of Instructions That Are Clear	.00	.01
Child Compliance^b	% of Clear Instructions Child Complies With	.00	.00
Treatment Completion	Treatment Completion	−.27	.24
Predictors of slope
Parent Attends^a	Attends/min	−.01*	.00
Parent Rewards^a	Rewards/min	.02	.02
Parent Ignoring^a	% of Inappropriate Child Behavior Ignored	.00	.00
Parent Clear Instructions^b	Clear Instructions/min	.01	.01
Parent Clear Instructions^b	% of Instructions That Are Clear	.00	.00
Child Compliance^b	% of Clear Instructions Child Complies With	.00	.00
Treatment Completion	Treatment Completion	.08	.05

Assessed during Child’s Game.

Assessed during Parent’s Game.

p = .04.

To better understand the nature of the association between Attends and child internalizing behavior over time, we modeled child internalizing slopes at one standard deviation above the mean, at the mean, and one standard deviation below the mean level of caregiver Attends during Child’s Game, following recommendations in the HLM literature (Curran & Bauer, 2011; Figure 1). Probing these simple slopes (see Figure 1) revealed that when caregivers attended at low levels (approximately 0 attends/min), child internalizing symptoms did not significantly decrease over the course of treatment (slope β = −.03, p = .11). However, if parents attended at medium levels (approximately 3 attends/min), child internalizing symptoms significantly decreased over the course of treatment (slope β = −.06, p < .01). Furthermore, if parents attended at high levels (approximately 6 attends/min), the decrease in child internalizing symptoms over the course of treatment was even more rapid (slope β = −.09, p < .01). Put another way, at low levels of parental attends, average child internalizing symptoms were not expected to decrease to 0 even after 16 treatment sessions, but at medium and high levels of attends, child internalizing symptoms were predicted to decrease to 0 after approximately 10 and 7 sessions, respectively.

Figure 1.

Simple slopes plot depicting child internalizing behavioral trajectories over the course of treatment if parents demonstrate Low (i.e., approximately 0 attends/min), Medium (i.e., approximately 3 attends/min), and High (i.e., approximately 6 attends/min) levels of Attends in Child’s Game over treatment sessions.

Discussion

This study aimed to characterize how child internalizing symptoms changed over the course of treatment, as well as which BPT skills were most effective in altering internalizing symptoms. Results revealed that child internalizing symptoms appeared to decline in a modest, linear fashion across treatment. Furthermore, results revealed that the use of the BPT Attends skill by parents was significantly associated with larger, and more rapid, declines in child internalizing symptoms over the course of treatment. We further consider how these findings contribute to our understanding of BPT’s effects on early-childhood internalizing symptoms below.

It is important to start our discussion of the current results with a qualification. Beyond Attends, no other BPT skills in this study were associated with significant changes in child internalizing symptoms. Perhaps teaching Attends introduces both the parental warmth and support (via the Attends themselves) and structural systems (e.g., Child’s Game habit) that are critical for ameliorating child internalizing problems, particularly in early-onset children and families, such that there is little variance left to be explained by parental use of other skills. An alternative, however, is that our study was underpowered to find additional effects. Indeed, the study sample consisted of 62 participants, which was reduced to 48 participants when Ignoring, Clear Instructions, and Time-Out skills were evaluated (because most of the 14 families that dropped out did so before these skills were taught). Therefore, we strongly caution readers to avoid drawing conclusions about the efficacy of other BPT parenting skills for ameliorating child internalizing symptoms from the nonsignificant findings presented here.

That said, several factors can potentially explain why the use of the Attends skill by parents was significantly associated with decreases in child internalizing behavior over the course of treatment. For example, two primary protective factors theorized to prevent or ameliorate child depression are embedded in the Attends skill taught in BPT: behavioral activation (i.e., mood lift that comes with engaging in pleasant activities) and parental warmth and support (Gonzalez & Jones, 2016). During sessions, caregivers are taught to practice Attends within the context of Child’s Game (i.e., a child-directed interaction; McMahon & Forehand, 2003). Therefore, Child’s Game is essentially a behavioral activation exercise for children with high internalizing symptoms given that it creates a structured context to engage in enjoyable activities (i.e., play) that are motivated by high levels of positive attention from their parent. In addition, Attends themselves are verbalizations of parental warmth and support directed at their child for the purpose of increasing appropriate child behavior (e.g., “Now you are putting those toys away so fast!,” “You are gently playing with the car.”). Thus, Attends may essentially serve as a delivery vehicle for parents to explicitly convey warmth and support toward their child. Indeed, it may be that Attends become protective over the course of treatment (as opposed to at the beginning of treatment) because parental practice of Attends shifts the parent–child relationship toward greater parental warmth and support, and the more rapidly this shift occurs, the more rapidly child internalizing symptoms are ameliorated. To this point, there appears to be a linear dose–response relationship between parental Attends and child internalizing symptoms over the course of treatment. If parents demonstrate an average of 3 attends/min, children’s internalizing symptoms are expected to decrease to minimal levels after 10 sessions. Furthermore, if parents demonstrate an average of 6 attends/min, such symptoms are expected to decrease more quickly (after just seven sessions). Although these findings may seem modest at first, over the course of treatment the effect of parental Attends on child internalizing symptomatology has potentially substantial clinical significance. That is, child internalizing symptomatology at medium and high levels of parental attending declines to minimal levels relatively quickly. These findings highlight the utility of mastery-based BPT programs, which ensure parents are using a given skill (in this case Attends) at a specific level, before progressing to new skills. Such criteria make it much more likely that parents are practicing skills long enough, and at high enough rates, for therapeutic effects to emerge.

With regard to Attends, it is also notable that these techniques seem to offer some level of relief even for families who are unable to complete the treatment program. For example, in our sample, treatment noncompleters dropped out after an average of 2.59 sessions. Yet most of these noncompleters had already mastered the Attends skill (the M_{Noncompleters} = 5.59 Attends/min, above the required number of 4 Attends/min for mastery criteria). Importantly, and as noted in the results, parents who dropped out of treatment showed no significant differences in number of Attends/min compared with their “treatment completing” counterpart families. Furthermore, noncompleter families also demonstrated no significant differences when compared with treatment completers on child internalizing symptoms, as both groups demonstrated decreases in internalizing symptoms over time. Taken together, these results may support prior evidence suggesting that Attends, and the child-directed play interactions Attends are embedded in, may lead to socioemotional improvements in child functioning regardless of whether subsequent skills are learned or not (Kochanska, Kim, Boldt, & Nordling, 2013).

In addition to highlighting the power of Attends, this study informs clinical practice as a first step toward tracking the link between treatment components and symptom improvement that can be replicated and extended with larger samples in future work. Indeed, collecting measures of clinical progress during each session has often been advocated for by prevention scientists but has rarely been implemented in practice (Chorpita et al., 2010; Gonzalez & Jones, 2016). We believe that tracking progress in this way is especially beneficial for providing clinicians with a sense of the course of symptoms they should expect to see across treatment. For instance, if results of the present investigation are replicated, they may provide clinicians working with families with low-income in BPT programs with the knowledge that, in general, internalizing symptoms are expected to decline in a linear fashion across treatment. If clients begin to significantly deviate from this expected linear trajectory, the clinician and family can then try to identify why such deviations are happening, and problem-solve together to provide increased opportunities to practice Attends and other BPT skills. Such treatment tracking and clinician–client collaboration are at the heart of NIMH’s prioritization of personalized behavioral health treatments (NIMH, 2015).

As with all research, study findings must be interpreted in light of limitations. As noted earlier, the study’s small sample size suggests that future investigations with larger samples are needed to replicate and potentially extend results. Second, therapists demonstrated reliability to the Behavioral Observation Coding System prior to their first session and sessions were video-recorded for the purposes of monitoring fidelity to the treatment manual and therapist competence; therapists themselves coded all observations, and reliability checks were not formally conducted on in-session mastery coding. Therefore, the extent to which therapist “drift” from mastery-level coding of skills affected results is unknown. That said, as mentioned previously, every session conducted by every therapist during treatment was coded for fidelity on a weekly basis by at least one, and sometimes two, licensed clinical psychologists who are considered expert HNC trainers and have a combined 60+ years of experience delivering HNC. Though these trainers did not formally recode therapist codes (and therefore inter-observer agreement cannot be calculated), they did review every instance of therapist coding, and either confirmed such coding as valid or brought up concerns about coding which were resolved and corrected during weekly supervision. Corrected codes were used in the present analyses and therapists’ fidelity to HNC treatment was greater than 95%. In addition, therapist drift would suggest greater variability in Attends coding within and between therapists, which would seem to make it more (rather than less) difficult to find patterns such as those seen in the current study for Attends. Moreover, though future studies may benefit from post hoc coding of parent behaviors by independent observers, it is equally important to mention that therapist in vivo coding has been identified as an essential component of mastery-based BPT programs like HNC, which have proven most efficacious for children with clinically significant problem behavior. Third, consistent with the theory inherent in BPT that an explicit focus on child positive behavior via Attends and Rewards will decrease negative behavior (e.g., noncompliance), Time-Out rarely occurred in session. The low frequency of Time-Out can also be attributed to parents effectively using the Phase I skills and the Clear Instruction sequence in Phase II of treatment and thus increasing the likelihood of child compliance. Yet this study cannot provide data regarding the incremental value of Time-Out (versus child compliance) for decreasing child internalizing symptoms. Finally, the present study did not have a no-treatment control group. Therefore, the extent to which increases in parental Attends caused decreases in child internalizing behavior is unknown. It may well be that child internalizing symptoms decreased more rapidly across treatment in the “high” Attends group due to some other unmeasured parent or child characteristic associated with Attends (e.g., parents who attended at higher levels may have talked more in general with their children, making it more likely that these parents taught their children skills to cope with negative emotions).

The present study also possesses several strengths. First, this study focused on families from low-income backgrounds who are often excluded from services research, yet are overrepresented in statistics on psychopathology (Gonzalez & Jones, 2016). In addition, this study used an ITT analysis framework that accounts for dropout in considering treatment efficacy. Third, as explicitly called for by leading prevention and intervention scientists (Chorpita et al., 2011, 2010) and government agencies (NIMH, 2015), this study used session-by-session measures of parent skill acquisition and report of child symptoms, which afforded a unique opportunity to examine more explicit links that may otherwise be lost in more traditional pre–post data analyses and/or studies that examine broader constructs (e.g., warmth/support) more typical in investigations of parent–child relationship. Fourth, this study used one BPT program, HNC (McMahon & Forehand, 2003); however common theory and practice components shared by HNC and other BPT programs, including Parent Child Interaction Therapy (Eyberg & Funderburk, 2011), and Incredible Years (Webster-Stratton, 2008), suggest that findings from this study likely generalize to other BPT programs. Finally, session-by-session tracking of clinical outcomes and subsequent estimation of clinical trajectories clarified potential economic benefits associated with this class of interventions. For instance, as demonstrated in Figure 1, internalizing symptoms were estimated to remit after approximately 20 sessions, 10 sessions, or seven sessions if parents demonstrated low (i.e., 0 attends/min), medium (i.e., 3 attends/min), or high (i.e., 6 attends/min) levels of Attends, respectively. Although the cost of BPT per family varies to some extent across programs, the average cost of HNC is US$82 per session (Honeycutt, Khavjou, Jones, Cuellar, & Forehand, 2015; Jones et al., 2013). As such, if parents demonstrated high levels of Attends, versus medium or low levels of Attends, a total of between US$246 and US$1066 could potentially be saved on treatment costs for a single family.

In conclusion, the present study contributes to the existing literature by demonstrating the usefulness of session-by-session outcome measures to characterize child internalizing trajectories across treatment. The linking of specific behavioral treatment targets (i.e., increasing parental attends from around 0 to around 6 attends per minute), and specific economic outcomes of treatment is made more readily possible by characterizing client progress over time. Importantly, such an approach is at the core of NIMH’s call for cost-effective family-based treatments targeting both early-starting internalizing and externalizing symptom clusters (NIMH, 2015).

Footnotes

Acknowledgements

We gratefully acknowledge the contribution of our study participants.

Authors’ note

W. Andrew Rothenberg is also affiliated with Duke University Center for Child and Family Policy.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Support for this study is provided by an institutional National Research Service Award predoctoral fellowship (WR and MA) provided by the National Institute of Child Health and Human Development (T32-HD07376) and the National Institute of Mental Health (R01-MH100377) (DJJ).

ORCID iD

W. Andrew Rothenberg

Author Biographies

W. Andrew Rothenberg was awarded his PhD in Clinical Psychology from the University of North Carolina at Chapel Hill and is currently a postdoctoral fellow at the Duke University Center for Child and Family Policy. He is passionate about identifying and treating maladaptive intergenerational family processes and child psychopathology. His current research aims to break these deleterious intergenerational cycles in medically underserved communities.

Margaret T. Anton graduated from University of North Carolina Chapel Hill with a degree in Clinical Psychology. She is a currently postdoctoral fellow at the Medical University of South Carolina. Her current research aims to develop new and innovative ways to use digital technology to improve access to the quality of mental health care, particularly for underrepresented populations.

Michelle Gonzalez graduated from the University of North Carolina Chapel Hill with a degree in Clinical Psychology. She is currently a postdoctoral fellow at the United States Department of Veteran Affairs. Her current research interests include understanding best practices in the provision of services to ethnically and socio-economically diverse families, identifying ways in which psychology can inform women’s health issues, and evaluating interventions to treat trauma.

Nicole Lafko Breslend is a research assistant professor at the University of Vermont. Her primary research interests include physiological, social, and familial predictors of problems in the peer group (e.g., peer victimization, rejection).

Rex Forehand, PhD, is a University distinguished professer, Ansbacher professor of Psychology, and director of the Vermont Genetics Network at the University of Vermont. His research interests are parenting, childhood problems, and intervention strategies.

Olga Khavjou is an economist in RTI’s Public Health Economics Program with more than 15 years of experience in the field of health economics. Ms. Khavjou is an expert in cost of illness analyses leading tasks and projects that use econometric modelling techniques to estimate direct and indirect costs of health conditions. She also has extensive experience developing cost collection instruments for many individual- and community-level public health programs and providing training and technical assistance to program staff at the federal, state, and local levels on how to track, collect, report, and analyze program costs.

Deborah J. Jones is a professor of Psychology and Neuroscience at the University of North Carolina at Chapel Hill. Her research examines patterns of risk and resilience within at-risk and underserved families. Most recently, her work examines strategies for enhancing family engagement in evidence-based treatments, as well as strategies for determining if, for whom, and how we may more responsively use evidence-based treatments to reach and impact a greater number of families.

References

American Psychiatric Association. (1994). Diagnostic and statistical manual of mental disorders (4th ed.). Washington, DC: Author.

Armstrong

Lycett

Hiscock

Care

Sciberras

(2015). Longitudinal associations between internalizing and externalizing comorbidities and functional outcomes for children with ADHD. Child Psychiatry and Human Development, 46, 736-748. doi:10.1007/s10578-014-0515-x

Beck

A. T.

Steer

R. A.

Brown

G. K.

(1996). Manual for the Beck Depression Inventory-II. San Antonio, TX: Psychological Corporation.

Burge

Hammen

(1991). Maternal communication: Predictors of outcome at follow-up in a sample of children at high and low risk for depression. Journal of Abnormal Psychology, 100, 174-180. doi:10.1037/0021-843X.100.2.174

Carpenter

A. L.

Puliafico

A. C.

Kurtz

S. S.

Pincus

D. B.

Comer

J. S.

(2014). Extending parent–child interaction therapy for early childhood internalizing problems: New advances for an overlooked population. Clinical Child and Family Psychology Review, 17, 340-356. doi:10.1007/s10567-014-0172-4

Chase

R. M.

Eyberg

S. M.

(2008). Clinical presentation and treatment outcome for children with comorbid externalizing and internalizing symptoms. Journal of Anxiety Disorders, 22, 273-282. doi:10.1016/j.janxdis.2007.03.006

Chorpita

B. F.

Daleiden

E. L.

Ebesutani

Young

Becker

K. D.

Nakamura

B. J.

. . . Starace

(2011). Evidence-based treatments for children and adolescents: An updated review of indicators of efficacy and effectiveness. Clinical Psychology: Science and Practice, 18, 154-172. doi:10.1111/j.1468-2850.2011.01247.x

Chorpita

B. F.

Reise

Weisz

J. R.

Grubbs

Becker

K. D.

Krull

J. L.

(2010). Evaluation of the Brief Problem Checklist: Child and caregiver interviews to measure clinical progress. Journal of Consulting and Clinical Psychology, 78, 526-536. doi:10.1037/a0019602

Cummings

C. M.

Caporino

N. E.

Kendall

P. C.

(2014). Comorbidity of anxiety and depression in children and adolescents: 20 years after. Psychological Bulletin, 140, 816-845. doi:10.1037/a0034733

10.

Curran

P. J.

Bauer

D. J.

(2011). The disaggregation of within-person and between-person effects in longitudinal models of change. Annual Review of Psychology, 62, 583-619. doi:10.1146/annurev.psych.093008.100356

11.

Downey

Coyne

J. C.

(1990). Children of depressed parents: An integrative review. Psychological Bulletin, 108, 50-76. doi:10.1037/0033-2909.108.1.50

12.

Eyberg

S. M.

Funderburk

B. W.

(2011). Parent–child interaction therapy protocol. Gainesville, FL: PCIT International.

13.

Eyberg

S. M.

Nelson

M. M.

Boggs

S. R.

(2008). Evidence-based psychosocial treatments for children and adolescents with disruptive behavior. Journal of Clinical Child & Adolescent Psychology, 37, 215-237. doi:10.1080/15374410701820117

14.

Eyberg

S. M.

Robinson

E. A.

(1983). Conduct problem behavior: Standardization of a behavioral rating scale with adolescents. Journal of Clinical Child Psychology, 12, 347-354.

15.

Fernandez

M. A.

Butler

A. M.

Eyberg

S. M.

(2011). Treatment outcome for low socioeconomic status African American families in parent-child interaction therapy: A pilot study. Child & Family Behavior Therapy, 33, 32-48. doi:10.1080/07317107.2011.545011

16.

Forehand

Jones

D. J.

Parent

(2013). Behavioral parenting interventions for child disruptive behaviors and anxiety: What’s different and what’s the same. Clinical Psychology Review, 33, 133-145. doi:10.1016/j.cpr.2012.10.010

17.

Best

K. M.

Conger

R. D.

Simons

R. L.

(1996). Parenting behaviors and the occurrence and co-occurrence of adolescent depressive symptoms and conduct problems. Developmental Psychology, 32, 717-731. doi:10.1037/0012-1649.32.4.717

18.

Gonzalez

M. A.

Jones

D. J.

(2016). Cascading effects of BPT for child internalizing problems and caregiver depression. Clinical Psychology Review, 50, 11-21.

19.

Gupta

S. K.

(2011). Intention-to-treat concept: A review. Perspectives in Clinical Research, 2, 109-112. doi:10.4103/2229-3485.83221

20.

Honeycutt

A. A.

Khavjou

O. A.

Jones

D. J.

Cuellar

Forehand

R. L.

(2015). Helping the noncompliant child: An assessment of program costs and cost-effectiveness. Journal of Child and Family Studies, 24, 499-504. doi:10.1007/s10826-013-9862-7

21.

Jones

D. J.

Forehand

Cuellar

Kincaid

Parent

Fenton

Goodrum

(2013). Harnessing innovative technologies to advance children’s mental health: Behavioral parent training as an example. Clinical Psychology Review, 33, 241-252. doi:10.1016/j.cpr.2012.11.003

22.

Jones

D.J.

Forehand

Cuellar

Parent

Honeycutt

Khavjou

…Newey

G.A

. (2014). Technology-enhanced program for child disruptive behavior disorders: Development and pilot randomized control trial. Journal of Clinical Child and Adolescent Psychology, 43(1), 88-101. doi:10.1080/15374416.2013.822308

23.

Kaehler

L. A.

Jacobs

Jones

D. J.

(2016). Distilling common history and practice elements to inform dissemination: Hanf-model BPT programs as an example. Clinical Child and Family Psychology Review, 19, 236-258. doi:10.1007/s10567-016-0210-5

24.

Kaminski

J. W.

Claussen

A. H.

(2017). Evidence base update for psychosocial treatments for disruptive behaviors in children. Journal of Clinical Child & Adolescent Psychology, 46, 477-499. doi:10.1080/15374416.2017.1310044

25.

Kochanska

Kim

Boldt

L. J.

Nordling

J. K.

(2013). Promoting toddlers’ positive social-emotional outcomes in low-income families: A play-based experimental study. Journal of Clinical Child & Adolescent Psychology, 42, 700-712.

26.

Levine

D. G.

Ducharme

J. M.

(2013). The effects of a teacher–child play intervention on classroom compliance in young children in child care settings. Journal of Behavioral Education, 22, 50-65. doi:10.1007/s10864-012-9163-z

27.

Luby

J. L.

Belden

A. C.

Tandon

Spitznagel

(2009). Preschool depression: Homotypic continuity and course over 24 months. Archives of General Psychiatry, 66, 897-905. doi:10.1001/archgenpsychiatry.2009.97

28.

McMahon

R. J.

Forehand

R. L.

(2003). Helping the noncompliant child: Family-based treatment for oppositional behavior (2nd ed.). New York, NY: Guilford Press.

29.

Merikangas

K. R.

Nakamura

E. F.

Kessler

(2009). Epidemiology of mental disorders in children and adolescents. Dialogues in Clinical Neuroscience, 11, 7-20.

30.

National Institute of Mental Health. (2015). Priorities for strategy 3.2. Retrieved from http://www.nimh.nih.gov/about/strategic-planning-reports/strategic-research-priorities/srp-objective-3/priorities-for-strategy-32.shtml

31.

O’Connell

M. E.

Boat

Warner

K. E.

(2009). Preventing mental, emotional, and behavioral disorders among young people: Progress and possibilities. Washington, DC: National Academies Press.

32.

Pelham

W. E.

Foster

E. M.

Robb

J. A.

(2007). The economic impact of attention-deficit/hyperactivity disorder in children and adolescents. Journal of Pediatric Psychology, 32, 711-727. doi:10.1093/jpepsy/jsm022

33.

Reinke

W. M.

Eddy

J. M.

Dishion

T. J.

Reid

J. B.

(2012). Joint trajectories of symptoms of disruptive behavior problems and depressive symptoms during early adolescence and adjustment problems during emerging adulthood. Journal of Abnormal Child Psychology, 40, 1123-1136. doi:10.1007/s10802-012-9630-y

34.

Rothenberg

W. A.

Hussong

A. M.

Chassin

(2018). Intergenerational continuity in high-conflict family environments: Investigating a mediating depressive pathway. Developmental Psychology, 54, 385-396. doi:10.1037/dev0000419

35.

Rothenberg

W. A.

Solis

J. M.

Hussong

A. M.

Chassin

(2017). Profiling families in conflict: Multigenerational continuity in conflict predicts deleterious adolescent and adult outcomes. Journal of Family Psychology, 31, 616-628. doi:10.1037/fam0000319

36.

Scott

Knapp

Henderson

Maughan

(2001). Financial cost of social exclusion: Follow up study of antisocial children into adulthood. British Medical Journal, 323, Article 191. doi:10.1136/bmj.323.7306.191

37.

Steer

R. A.

Ball

Ranieri

W. F.

Beck

A. T.

(1999). Dimensions of the Beck Depression Inventory-II in clinically depressed outpatients. Journal of Clinical Psychology, 55, 117-128.

38.

Tronick

E. Z.

Gianino

A. F.

(1986). The transmission of maternal disturbance to the infant. New Directions for Child and Adolescent Development, 34, 5-11. doi:10.1002/cd.23219863403

39.

Webster-Stratton

(2008). The Incredible Years: Parents and Children Series: Leader’s guide: Preschool version of BASIC (ages 3-6 years). Seattle, WA: The Incredible Years.

40.

Webster-Stratton

Herman

K. C.

(2008). The impact of parent behavior-management training on child depressive symptoms. Journal of Counseling Psychology, 55, 473-484. doi:10.1037/a0013664