The effects of the KEEP foster parent training intervention on child externalizing and internalizing problems

Abstract

Children in foster care are at risk for externalizing and internalizing behavior problems. The KEEP foster parent training intervention has been found to effectively reduce parental reports of daily child behavior problems. The aims of this investigation were to (a) examine the effectiveness of the KEEP intervention at reducing behavior problems among children in foster care, as assessed by the Child Behavior Checklist (CBCL), during implementation of the intervention by a community agency using a randomized design and (b) determine whether the intervention is effective at reducing internalizing forms of behavior problems. Foster and kinship families were randomly assigned to either the KEEP intervention or usual services. The KEEP intervention was delivered by a community agency. Child behavior problems were assessed via the CBCL at baseline and at the end of the intervention (4 months after baseline). Data from 310 foster and kinship families with children aged between 5 and 12 years were analyzed using hierarchical linear modeling. The KEEP intervention was effective at reducing the proportion of children in the borderline group for rule-breaking between baseline and follow-up. The intervention was also effective at reducing internalizing problems, as indicated by reductions in internalizing raw scores and the proportion of children in the borderline group over the course of the intervention. These findings suggest the KEEP intervention can provide foster parents with the tools to manage a broad range of behavior problems even as the intervention was delivered by a community agency.

Keywords

Children externalizing problems foster parents internalizing problems intervention

Children in foster care have experienced adverse situations, such as various forms of neglect, family violence, and physical or sexual abuse in their family of origin (Oswald, Heil, & Goldbeck, 2010). Many of these children have also experienced multiple changes in placement. Coupled with separation from their family of origin, it is not surprising that children in foster care exhibit a range of behavioral health issues (Bronsard et al., 2016; Landsverk, Garland, & Leslie, 2002). Externalizing behaviors are especially prevalent in this group (Bronsard et al., 2016; National Survey of Child and Adolescent Well-being Research Group, 2003) and as such are a salient issue for foster parents and caseworkers. Such behaviors have been linked to placement disruptions (Chamberlain et al., 2006; Newton, Litrownik, & Landsverk, 2000), delinquency (Broidy et al., 2003), and high use of mental health services (Erath et al., 2009). Children in foster care have also been found to exhibit higher levels of internalizing problems than the general population (Bronsard et al., 2016). Although internalizing behavior problems may be less salient to caregivers and caseworkers, internalizing problems have deleterious effects on children’s emotional, social, and school functioning (Brendgen, Vitaro, Turgeon, & Poulin, 2002; Rapee, Schniering, & Hudson, 2009; Rapport, Denney, Chung, & Hustace, 2001). Internalizing problems also place individuals at risk for later suicide and substance use disorders (Crum et al., 2008; Marmorstein, 2009).

The KEEP (Keeping Foster Parents Trained and Supported) intervention was designed to provide foster and kinship caregivers with parenting strategies for managing child behavior problems, with the goal of helping reduce foster placement disruptions (Chamberlain, Price, Reid, & Landsverk, 2008). A growing body of evidence indicates that the KEEP intervention is effective at reducing behavior problems in samples of children in foster care, based on parental reports of daily behavior problems (e.g., Chamberlain, Price, Leve, et al., 2008; Price, Roesch, & Walsh, 2012, 2015). There is also evidence that the KEEP intervention is effective at reducing externalizing behavior problems in foster children who have scored in the clinical range for externalizing on the Child Behavior Checklist (CBCL) (Greeno et al., 2016). Furthermore, there is preliminary evidence that the KEEP intervention may be effective at reducing internalizing behavior problems (Greeno et al., 2016), as assessed by the CBCL. Unfortunately, no comparison or control groups were included in these studies. The first objective of the current study was to extend the research on the KEEP intervention by examining whether it is effective at reducing behavior problems, as defined and assessed by the CBCL. The second objective was to determine whether the KEEP intervention is effective at reducing internalizing behavior problems of children in foster and kinship placements.

Based on the principles of Parent Management Training, the KEEP intervention utilizes components of the Treatment Foster Care Oregon model (Chamberlain, Leve, & DeGarmo, 2007). The KEEP program provides foster and kinship caregivers with 16 weeks of training and supervision in behavior management strategies by trained facilitators, as well as support in a group context, within 90-min weekly sessions. The focus of the intervention is to equip caregivers with strategies for creating a family context that will reduce and prevent coercive parent/child interactions and child behavioral problems. Some of the specific strategies taught include use of positive reinforcement, consistent use of nonharsh limit-setting techniques (e.g., time-out), monitoring and supervision, and avoiding parent/child power struggles. The results from an effectiveness trial indicate that the KEEP intervention is effective at (a) reducing parental reports of daily child behavior problems between baseline and follow-up (Chamberlain, Price, Reid, et al., 2008), (b) increasing use of parenting strategies, which mediate the positive effects of the intervention (Chamberlain, Price, Leve, et al., 2008), and (c) increasing the number of positive exits (e.g., reunification with parents) and mitigating the negative effect of multiple past placement changes on future negative placement exits (Price et al., 2008).

It has been proposed that the mental health of children in child welfare systems can be improved through the widespread implementation of evidence-based parenting interventions such as KEEP (Horwitz, Chamberlain, Landsverk, & Mullican, 2010). The Cascading Dissemination Model (see Chamberlain et al., 2008) outlines an optimal process for implementation of such interventions, whereby the management, supervision, and delivery of the intervention are shifted away from the designers of the intervention to independent individuals and organizations. In the current study, the KEEP intervention was managed and delivered by a community agency to a sample of foster and kinship families; it thus represented the final phase of the Cascading Dissemination Model. During this phase, the intervention developer is not involved in the training, supervision, or management of the implementation trial. In this study of the implementation of KEEP, a randomized intent-to-treat (ITT) design was utilized.

Although the behavior management techniques taught and supported by the KEEP intervention have been primarily utilized to address externalizing behavior problems (e.g., outwardly directed and under-controlled behaviors, such as aggression), these strategies may also help reduce internalizing behavior problems (e.g., inwardly directed and over-controlled behaviors, such as depression). They may do this in several ways. First, the KEEP curriculum teaches caregivers strategies for establishing consistent behavioral expectations and consequences (e.g., time-outs). By doing so, caregivers can create a more stable and predictable family environment. Since many children in foster care have experienced inconsistent parenting and chaotic family environments, a more stable and predictable family environment should help lower anxiety and foster emotional regulation. Supporting this contention, Galambos, Barker, and Almeida (2003) found that adolescents whose parents reported using firm behavioral control (e.g., not allowing their child to get away with things) demonstrated lower levels of internalizing behaviors than adolescents whose parents reported employing less behavior control.

The KEEP curriculum also offers instruction and support for parents in the use of nonharsh discipline strategies such as time-outs. Consistent use of nonharsh discipline reduces or prevents the use of harsh discipline, which is associated with heightened levels of internalizing behavior problems (Laskey & Carwright-Hatton, 2009). In addition, parental aversiveness, which is often expressed through discipline, has been found to be associated with internalizing behaviors (McLeod, Weisz, & Wood, 2007; Yap & Jorm, 2015). Many children in foster care have experienced some form of harsh parenting and abuse prior to entering the foster care system and this may have contributed to their internalizing behavior problems (Oswald et al., 2010). Thus, when caregivers implement nonharsh, nonaversive discipline strategies and do so in the manner taught by the KEEP curriculum (e.g., remaining calm), they create predictable and nonharsh patterns of social interactions, which may help ameliorate children’s internalizing symptoms.

Finally, the use of positive reinforcement is a central to the KEEP curriculum. Caregivers are taught to attend to and reinforce positive child behaviors and behavioral progress using informal (e.g., “Thank you for minding”) and formal rewards (e.g., charts). Supporting parents in the use of positive reinforcement for appropriate behavior helps children learn specific prosocial behaviors and facilitates positive interactions between caregiver and child. Parental warmth, defined as a sense of positive regard displayed toward the child (McLeod et al., 2007), has been found to be associated with lower levels of internalizing behavior problems (McLeod et al., 2007; Yap & Jorm, 2015). Various forms of positive reinforcement can be considered as positive regard displayed by the parent toward their children. Thus, increased use of positive reinforcement should contribute to reducing children’s internalizing symptoms.

It is plausible that the parenting strategies taught as part of the KEEP curriculum enable caregivers to create stable and predictable social environments characterized by nonharsh parenting practices and displays of positive regard toward the children in their care. Such social environments should serve to reduce internalizing behavior problems in children. Supporting this assertion are findings from a study by Greeno et al. (2016), who found that family participation in the KEEP intervention resulted in decreases in the severity of externalizing and internalizing problems, as assessed by the CBCL, among children in foster care from baseline to posttest. Unfortunately, no control group was used in this study for comparison.

The first aim of the present study was to examine the effectiveness of the KEEP intervention at reducing child behavior problems, as assessed by the CBCL, among children in foster care, using a randomized ITT design. It was hypothesized that there would be greater reductions in child behavioral problems among participants in the intervention group compared to participants in the control group. The second aim was to determine whether the KEEP intervention would be effective in reducing internalizing behavior problems in children in foster and kinship placements. It was hypothesized that, relative to participants in the control group, there would be greater reductions in internalizing problems among participants in the intervention group.

Method

Participants and recruiting process

For this study, 354 foster and kinship families agreed to participate, understanding that after agreeing to participate they would be randomly assigned to either the intervention or control group. These participants came from a pool of 609 potential families who met eligibility requirements. The participation rate for this study was 58%. This rate is similar to the participation rate for the effectiveness trial for the KEEP intervention, which was 62% (Chamberlain, Price, Reid, et al., 2008). Reasons for declining to participate in this study included scheduling conflicts, being “too busy,” and “not interested.” Among the families who agreed to participate, 15 from the intervention condition either dropped out (n = 8) or did not continue because of scheduling conflicts (n = 7). Four families from the control condition dropped out before completing baseline assessments. Figure 1 provides details of the participant flow over the course of this study. Between baseline and follow-up assessments, retention rates were 82% for the control group and 81% for the intervention group. These rates are comparable to the overall retention rate reported for the original trial of the KEEP intervention (Chamberlain, Price, Leve, et al., 2008).

Figure 1.

Flow of participants in KEEP implementation trial. KEEP = Keeping Foster Parents Trained and Supported.

To identify eligible families, Child Welfare Services data systems were reviewed four times per year. Recruitment for this study was conducted between 2009 and 2013. Eligibility requirements included that the child who was the primary focus of the intervention (Primary Child—PC) was a dependent of Child Welfare Services, between the ages of 5 and 12, had been in the current placement for at least 30 days (to reduce inclusion of children in temporary or emergency placements), and was not considered “medically fragile” (severely physically or mentally challenged). Eligibility also required two or more children in the home and that the family had not received the KEEP intervention in the past.

Potentially eligible families were contacted by phone to confirm eligibility and to assess the parents’ interest in the study. Home visits were scheduled for interested families. During the home visit, parents were provided with an overview of the San Diego State University (SDSU) IRB approved study, a description of the randomization process, and a SDSU IRB approved consent form. If parents agreed to participate and signed the consent form parents were asked to select one of several blank envelopes to discover whether their family would be in either the intervention group or the control group. Once families knew their assignment, the interviewer then assisted parents in identifying the specific segment of the consent form that addressed the procedures to be followed for their particular group (e.g., control group received baseline and follow-up assessments). All participants were provided incentives for completing the baseline and follow-up assessments. In addition, families in the intervention group were given US$15.00 for each intervention session to assist with travel expenses and provided with childcare during each intervention session. In agreement with Child Welfare Services, caregivers in the intervention condition received 8 hr of credit required for state licensing. Caregivers in the control condition participated in continuing education courses (8 hr per year) to maintain licensing. Caregivers in both groups could continue participation in foster and kinship support groups held by the local foster parent association. Families in both groups also received services considered necessary by their caseworker.

One unique aspect of this study was that if the PC left the home during the course of the intervention period, in either the intervention or control group, the child in home who was next closet in age to the PC filled the role of the PC (control: n = 16; intervention: n = 19). The child who took this position could be another child in foster care, a relative (e.g., niece), or a biological or adopted child of the caregiver. This procedure allowed caregivers in the intervention group always to have a child with whom they could concentrate on the acquisition and application of new parenting strategies. This procedure also maintained the PC position in both groups during the intervention phase and provided the opportunity to examine the potential generalizability of the impact of the intervention on additional children within the same home.

Research design

Families were randomized into the intervention group (KEEP) or the control group, using an ITT design which maintains participants in randomized groups throughout the analyses, regardless of whether they completed the intervention or assessments. The ITT design takes into account noncompliance and participant dropout, which regularly occur during the implementation of evidence-based treatments, such as KEEP. The ITT design and analysis thus avoids inflated estimates of efficacy resulting from the removal of noncompliers or dropouts (Gupta, 2011), which is why this design was used in this investigation.

KEEP model description

The KEEP program entails 16 weeks of training with weekly sessions lasting appropriate 90 min each. Caregivers met in groups of 3–10 led by a trained group facilitator. For this study, group facilitators were staff from a nonprofit community agency—Social Advocates for Youth (SAY), San Diego. Group facilitators presented session material within a discussion format that emphasized application to child behavioral challenges faced by caregivers. In addition, group facilitators provided weekly supervision and support in parenting strategies. Parenting groups took place in local community facilities, including churches, recreation centers, or SAY offices. Group formation was depended upon caregiver schedules, locations, and language preferences (English or Spanish). Both English and Spanish versions of the curriculum were provided, depending upon the composition of the parenting group.

An essential component of the KEEP intervention model is training and supervision of group facilitators. Group facilitators had at least an Associate of Arts degree and had prior experience facilitating the KEEP intervention model in a community setting. Group facilitators were originally trained using a variety of techniques, including viewing video recordings of parenting sessions conducted by experienced facilitators, role-playing mock parenting sessions with trainees playing both parent and facilitator roles, and having trainees serve as co-facilitators with an experienced group facilitator in an actual KEEP parenting session. In addition to this initial training, each group facilitator was supervised by an individual with extensive experience (50 plus parenting groups) in the KEEP program. To assess intervention fidelity, the supervisor reviewed each parenting session delivered by each facilitator and completed the Facilitator Adherence Rating (FAR) scale. The FAR is comprised of 14 items that assess group facilitator adherence to three essential elements of the intervention procedures: structure, content, and process. The supervisor rated each item of the FAR on a 1 (low) to 5 (high) scale. For this study, the average FAR rating across facilitators and sessions was 4.47 (SD = .36), indicating high facilitator adherence to the intervention protocol. In addition, the facilitator conducted weekly staff meetings with group facilitators to provide feedback on the FAR ratings and group sessions and to address specific child behaviors that were particularly challenging for caregivers. Clinical consultation was available from a behavioral management specialist associated with the program developer at the Oregon Social Learning Center.

The content of the KEEP curriculum focuses on behavior management strategies previously found to be effective in reducing child behavior problems. These strategies include use of positive reinforcement for appropriate behavior and behavioral progress, use of nonharsh limit setting strategies (e.g., brief time-outs and privilege removal), and parental monitoring and supervision. Additional sessions focused on strategies for managing and monitoring peer relationships, facilitating school success, and avoiding power struggles between parents and children. Group sessions were structured to facilitate group discussion of the application of the KEEP curriculum to the specific child behavior challenges encountered by caregivers. Caregivers also viewed video recordings of application of key parenting strategies (e.g., use of time-out) and were then given the opportunity to role-play those strategies with feedback from the facilitator. To assist with application of the material, caregivers were offered weekly home practice assignments, which could be discussed during weekly follow-up calls by facilitators. When caregivers missed a parenting session, a home visit was scheduled. The majority of parents (85%) received at least one home visit. The average attendance rate across sessions and groups was 67% and the intervention completion rate (including home visits) was 81%.

Data collection procedures

Demographic information and behavioral assessments were collected just prior to baseline and at the end of the intervention (follow-up). For the intervention group, baseline assessments were initiated 2 weeks prior to the start of the intervention and follow-up assessments were completed within the 2 weeks following the end of the intervention program. The pre–post assessment interval range was 18–20 weeks. For the control group, baseline assessments were completed shortly after recruitment, with follow-up assessments 18–20 weeks later. Assessments at both baseline and follow-up were separated into three 45- to 60-min interviews conducted by phone in either English or Spanish (determined by parental preference).

Measures

Family demographics

Information on child, parent, and family characteristics was provided by caregivers during baseline assessments. In each case, caregivers knew the PC for at least a month prior to these assessments.

Child behavior problems

The Child Behavioral Checklist was used to measure externalizing and internalizing problems (CBCL; Achenbach, 1991; Achenbach & Rescorla, 2001) and was completed by the foster/kin caregivers at baseline and termination (i.e., follow-up). In it, the caregivers rated the frequency of 113 child behavioral descriptors using a 3-point scale that ranged from Not True (0), to Somewhat or Sometimes True (1), to Very True or Often True (2). The eight subscales of the CBCL (anxious/depressed, withdrawn/depressed, somatic complaints, social problems, thought problems, attention problems, rule-breaking behavior (delinquency), and aggressive behavior) make up three broadband scales: total behavior problems, internalizing problems, and externalizing problems. T-scores were used in the analyses. For the CBCL subscales, T-scores between 65 and 69 are considered to be in the borderline range and scores >70 are considered to be in the clinical range. For the broadband scales (internalizing, externalizing, and total score), T-scores from 60 to 63 are considered to be in the borderline range and scores >63 are considered to be in the clinical range.

Previous research has found foster parents to be reliable informants of child behavior (Tarren-Sweeney, Hazell, & Carr, 2004). The CBCL is a widely validated and reliable measure, with test–retest reliability ranging from .70 to .95 (Achenbach, 1991). In a nationally representative foster care sample (National Survey of Child and Adolescent Well-being Research Group, 2003), both the internalizing and externalizing broadband scales demonstrated high internal consistency (Cronbach’s α = .90 and .92, respectively). With the current sample, the internal consistency (Cronbach’s α) of each scale at baseline and follow-up assessment, respectively, was as follows: externalizing (broadband)—.93, .92; aggression—.92, .91; rule-breaking—.74, .75; internalizing (broadband)—.87, .86; anxiety/depression—.80, .78; somatic complaints—.71, .64; and withdrawal—.75, .72.

Data analysis procedure

Hierarchical linear modeling (HLM; Raudenbush & Bryk, 2002) was utilized to examine the study hypotheses regarding the KEEP intervention’s effectiveness at improving child behavior problems, as assessed by the CBCL. HLM addresses the nested (or clustered) configuration of the data (i.e., repeated measures [Level-1] nested within children [Level-2]). An ITT approach was included in the analyses to assess the impact of the KEEP intervention from pre- to posttest. For these analyses, the MLR estimation procedure in MPlus was used. This procedure adjusts for nonnormality of the outcome variables and missing data. The main interest in the HLM analyses were the cross-level Group (intervention vs. control) × Time (pre–post intervention) interaction terms, signifying the intervention effects on the broadband and subscale T-scores from the CBCL. Of the 310 participants included in the analyses (see Figure 1), 62 focal children had missing data for the posttest. These participants included children who had changed placement during the intervention period and children from families who did not, for various reasons, complete the termination interview. To examine the impact of the KEEP intervention on child behavior problems as assessed by the CBCL, two types of data were examined. First, changes in CBCL scale T-score values between baseline and follow-up were examined. Next, changes in the proportion of participants above the borderline and clinical cut-off points between baseline and follow-up were examined.

Informed consent

Informed consent was obtained from all individual participants included in the study.

Results

Sample characteristics

The baseline demographics of the sample are listed in Table 1. The sample was composed primarily of female (92%) caregivers. Caregivers represented African American, Caucasian, and Hispanic ethnic groups were a mean age of 45.6, had been foster or kinship parents for an average of 3.5 years, and had an average of 3.5 children per household. The mean age of the children in the homes of these foster parents was 7.56, and about equally divided between males (51.5%) and females (48.5%). In support of the randomization procedures being effective in producing equivalent groups at baseline, there were only two significant differences between the intervention and control groups. There were more African American parents in the intervention group (24%) relative to the control group (10%) and the children in the intervention group were slightly older than children in the control group (7.82 and 7.3, respectively).

Table 1.

Demographic information on foster and kin parents by group.

Demographic information	KEEP intervention group (n = 148)	Control group (n =162)
Parent gender
Female	93%	91%
Male	7%	9%
Mean parent age	45.14 (10.1)	46.03 (10.7)
Parent ethnicity**
African American	24%	10%
Asian/Pacific Islander	0%	3%
Caucasian	32%	41%
Hispanic	41%	41%
Native American	1%	4%
Mixed ethnicity	2%	1%
Parent preferred language (during interviews)
English	65%	63%
Spanish	35%	37%
Mean months foster Parent/caregiver	38.61 (52)	44.74 (64)
Mean number of children in home	3.65 (1.5)	3.42 (1.4)
Relationship to focal child
Kinship caregiver	45%	50%
Nonkinship caregiver	55%	50%
Focal child gender
Female	49%	48%
Male	51%	52%
Mean age of focal child	7.82 (2.4)	7.30 (2.3)*
Focal child ethnicity
African American	24%	12%
Asian/Pacific Islander	2%	1%
Caucasian	11%	18%
Hispanic	45%	51%
Native American	1%	1%
Mixed ethnicity	17%	17%

Note. KEEP = Keeping Foster Parents Trained and Supported.

*p < .05; **p < .01

CBCL differences

Using continuous T-score values for all scales, the HLM analysis showed a statistically significant Group × Time interaction for the internalizing scale (B = −2.25, p = .031) as well as the anxiety/depression subscale (B = −1.73, p = .008). Statistically significant decreases in internalizing and anxiety/depression scores were found for the intervention group relative to the control group. No significant differences were found for the remaining internalized subscales, the externalizing broadband scale, or the externalizing subscales. Descriptive statistics and effect sizes for the target intervention effects are presented in Table 2.

Table 2.

Descriptive statistics and effect sizes for internalizing and externalizing scores at pre- and posttest assessment (n = 310).

	Pretest	Posttest	Cohen’s d	p Value (Group × Time)
Internalizing behavior problems (broadband)				.031
Control	52.43 (11.51)	52.58 (11.89)	.02
Intervention	54.42 (11.71)	51.28 (10.75)	.32
Anxiety/depression				.008
Control	55.98 (7.10)	55.94 (8.18)	.01
Intervention	57.36 (8.18)	54.85 (6.63)	.41
Withdrawn				.105
Control	57.21 (7.93)	57.42 (8.21)	−.03
Intervention	57.72 (8.20)	56.12 (7.07)	.25
Somatic complaints				.233
Control	54.09 (5.34)	54.43 (5.91)	−.06
Intervention	54.88 (6.96)	53.97 (5.56)	.12
Externalizing behavior problems (broadband)				.126
Control	56.27 (12.48)	55.30 (11.18)	.15
Intervention	58.45 (12.82)	54.65 (12.71)	.55
Aggression				.563
Control	58.32 (7.84)	57.91 (7.70)	.10
Intervention	59.78 (8.29)	57.81 (8.25)	.44
Rule-breaking				.392
Control	59.41 (10.35)	57.58 (8.88)	.33
Intervention	61.35 (11.40)	58.06 (9.76)	.53

Note. Standard deviation is given in parentheses.

Similar analyses were subsequently conducted with categorized versions of the CBCL broadband scales and subscales using hierarchical generalized linear models to analyze dichotomous outcomes. Here, T-score cut-offs were used to look at groups of participants that were (a) either above the borderline cut-off value for a given (sub)scale or not, and (b) either above the clinical cut-off value for a given (sub)scale or not. For these dichotomous outcomes, those in the borderline group vs. not showed a statistically significant Group × Time interaction for the internalizing scale (B = −1.27, p = .043; OR = 0.28), withdrawn subscale (B = −1.97, p = .007; OR = 0.14), and rule-breaking subscale (B = −1.63, p = .041, OR = 0.20). A statistically significant decrease in the proportion of individuals in the borderline group was found for the intervention group relative to control for these three outcomes. Frequencies, percentages, and odds ratios are presented in Table 3.

Table 3.

Frequencies/percentages of participants in borderline group for internalizing and externalizing variables at pre- and posttest assessment (n = 310).

	Pretest	Posttest	Odds ratio	p Value (Group × Time)
Internalizing behavior problems (broadband)			1.52	.043
Control	47/29.0%	42/31.8%
Intervention	51/34.5%	30/24.2%
Anxiety/depression			1.40	.215
Control	30/18.5%	19/14.4%
Intervention	31/20.9%	14/11.3%
Withdrawn			1.90	.007
Control	29/17.9%	32/24.2%
Intervention	31/20.9%	16/12.9%
Somatic complaints			2.62	.063
Control	9/5.6%	11/8.3%
Intervention	15/10.1%	7/5.6%
Externalizing behavior problems (broadband)			1.01	.746
Control	71/43.8%	41/31.1%
Intervention	79/53.4%	45/36.3%
Aggression			1.03	.677
Control	48/29.6%	34/25.8%
Intervention	55/37.2%	33/26.6%
Rule-breaking			1.70	.041
Control	38/22.8%	31/24.2%
Intervention	50/33.8%	24/19.4%

No statistically significant Group × Time interactions were found for any scale or subscale using the dichotomous outcomes for clinical cut-off (p > .05 for all; see Table 4 for frequencies, percentages, and odds ratios).

Table 4.

Frequencies/percentages of participants in clinical group for internalizing and externalizing variables at pre- and posttest assessments (n = 310).

	Pretest	Posttest	Odds ratio	p Value (Group × Time)
Internalizing behavior problems (broadband)			1.52	.112
Control	39/24.1%	35/26.5%
Intervention	39/26.4%	23/18.5%
Anxiety/depression			2.10	.270
Control	12/7.4%	11/8.3%
Intervention	16/10.8%	7/5.6%
Withdrawn			2.18	.104
Control	13/8.0%	11/8.3%
Intervention	18/12.2%	7/5.6%
Somatic complaints			1.33	.260
Control	3/1.9%	4/3.0%
Intervention	8/5.4%	2/1.6%
Externalizing behavior problems (broadband)			1.28	.475
Control	50/30.9%	34/25.8%
Intervention	62/41.9%	55/37.2%
Aggression			1.07	.728
Control	29/17.9%	17/12.9%
Intervention	40/27.0%	22/17.7%
Rule-breaking			1.23	.547
Control	25/15.4%	17/12.9%
Intervention	29/19.6%	16/12.9%

Discussion

The primary aims of this study were (a) to examine the effectiveness of the KEEP intervention at reducing child behavior problems, as assessed by the CBCL, among children in foster care, using a randomized design; and (b) to determine whether the KEEP intervention is effective at reducing internalizing behavior problems in children in foster and kinship placements. The results provided support for the KEEP intervention’s usefulness for reducing child behavior problems, as assessed by the CBCL, even as it was being implemented by a community agency. Consistent with past research indicating the KEEP intervention’s effectiveness in addressing externalizing behavior problems, the results revealed a significant reduction in the proportion of children borderline for rule-breaking in the intervention group relative to control between baseline and follow-up, with a reduction from 34% to 19% for the intervention group and a slight increase, from 23% to 24%, in the control group. The intervention was also effective at reducing broadband internalizing behavior problems between baseline and follow-up in the intervention group versus control, in terms of both continuous T-scores and the proportion of children in the borderline group, the latter showing a reduction from 35% to 24% in the intervention group and a slight increase, from 29% to 32%, in the control group. Additional support for the effectiveness of the KEEP intervention in reducing internalizing problems is indicated by significant reductions in T-scores for anxious/depressed problems in the intervention group relative to control from baseline to follow-up and by a significant reduction in the proportion of children borderline for withdrawn behaviors in the intervention group; the proportion decreased from 21% to 13% in the intervention group while increasing from 18% to 24% in the control group.

Thus far, the KEEP foster parent intervention has been found it to be effective at reducing child behavior problems, as assessed by parental reports of daily behavior problems (e.g., Chamberlain, Price, Leve, et al., 2008; Price et al., 2015). There is also evidence that the KEEP intervention is effective at reducing externalizing behavior problems in children with elevated levels of behavior problems, as indicated by scoring in the clinical range on the CBCL (e.g., Greeno et al., 2016). Greeno et al. (2016) also found evidence that the KEEP intervention may be effective at reducing internalizing problems among children in foster care. In that study, university-based interventionists utilized a pretest–posttest design but without random assignment. The results of the current investigation extend the research on the KEEP intervention by demonstrating that even when delivered by a community provider (as opposed to the developers or university-based practitioners), the intervention remains effective at reducing behavior problems. Furthermore, these reductions were evident using a randomized ITT design with a representative sample of foster families and children in foster care. It is important to note that the children in the current study had been in their current foster placement for an average of 13.4 months. Since children with elevated levels of behavior problems are more likely to experience changes of placement, the children in the current sample may not have represented children with severe levels of behavior problems. This possibility is supported by the low proportion of children within the clinical ranges of the CBCL broadband scales for both externalizing and internalizing behavior problems (36% and 25%, respectively; see Table 4) at baseline. Therefore, as might be expected for an intervention delivered by a community agency within a typical nonreferred population of foster and kinship families, the effect sizes, though significant, were small. Nevertheless, participation in the KEEP intervention resulted in some significant changes in child behavior problems among children with relatively normative levels of behavior problems who had been in foster care for over a year.

The results of the current study also expand the prior research on the KEEP intervention by demonstrating the generalizability of the effects of the intervention to internalizing forms of behavior problems. Prior to the current study, there was some preliminary evidence indicating that the KEEP intervention may be effective at reducing internalizing behavior problems. Using the CBCL, Greeno et al. (2016) found decreased levels of internalizing behavior problems from baseline to posttest in a nonrandomized sample of 65 children whose families participated in the KEEP intervention. In the current study, using a randomized ITT design, significant decreases in the T-scores for internalizing behavior problems were found between baseline and follow-up for the intervention group but not the control group. In addition, significant decreases in T-scores were also observed on the anxiety/depression subscale for the intervention group but not the control group. For dichotomous outcomes, a statistically significant decrease in the proportion of individuals in the borderline group versus not was found for the intervention group relative to control for both internalizing and withdrawn behaviors. There were no statistically significant decreases in the proportion of individuals in the clinical group for intervention relative to control for any of the internalizing scales. In sum, significant decreases in the intervention group relative to control were observed for the T-scores of the broadband internalizing scale and the proportion of children in the borderline group for the broadband internalizing problems. However, the findings were less consistent regarding the internalizing subscales, which could be due to the relatively low severity of these specific behavioral symptoms in this sample.

When children enter foster care, they will have often experienced difficult situations and adverse conditions (e.g., neglect and abuse) that place them at increased risk for both externalizing and internalizing behavior problems. Due to the saliency of externalizing behavior problems to foster parents and caseworkers, and the association of these problems with placement disruptions and later adjustment problems in adolescents, intervention and treatment efforts are often focused on lowering externalizing behaviors. However, internalizing problems also warrant attention, given their association with poor emotional, social, and school functioning (Brendgen et al., 2002; Rapee et al., 2009; Rapport et al., 2001) and the development of depression and substance use disorders in adolescence (Crum et al., 2008; Marmorstein, 2009). The findings from this study suggest that within a large and diverse sample of children living in both nonrelative and relative placements, the KEEP foster parent training intervention is effective at reducing internalizing behavior problems, especially at the broadband level.

There are several possible explanations for why the KEEP intervention was effective at reducing internalizing problems in this sample. First, implementing the consistent use of behavioral management strategies may help create a more stable and predictable family environment, which, in turn, contributes to lowering children’s levels of internalizing problems (e.g., Galambos, Barker, & Almeida, 2003). Second, consistent use of nonharsh discipline strategies may help lower and prevent harsh and aversive parent/child interactions. Minimizing harsh/aversive parent/child interactions and increasing the consistent use of nonharsh discipline strategies (e.g., time-outs) should contribute to reductions in internalizing problems. Third, as one of the key components of the KEEP curriculum, positive reinforcement given to the child for behavioral progress and prosocial behavior can be viewed as a form positive regard (i.e., warmth) and parental warmth and positive regard have been consistently linked to lower levels of internalizing behavior problems (McLeod et al., 2007; Yap & Jorm, 2015). One of the strategies taught and supported early on in the KEEP curriculum is the use of both informal (e.g., verbal praise) and formal (e.g., charts) methods to maintain appropriate behavior and teach new behaviors. Charts in particular act as visual reminders for parents to use positive regard when appropriate and prosocial behaviors are displayed. It is important to note that these same processes are not specifically unique to internalizing problems and would likely contribute to the reduction of externalizing problems as well.

Another possible mechanism by which the KEEP intervention may influence levels of internalizing problems is through an overall reduction in externalizing behavior and family conflict. As noted earlier, the KEEP intervention is effective at reducing behavior problems in children in foster care (Chamberlain, Price, Leve, et al., 2008; Greeno et al., 2016; Price et al., 2012, 2015). Moreover, the KEEP intervention has been found to reduce the behavior problems of multiple children in the same home (Price et al., 2015). Because the intervention reduces overall levels of behavior problems within the foster home, it also reduces the potential for coercive or harsh parent/child interactions. In the current study, reductions in the proportion of children displaying high levels of rule-breaking behavior may have contributed to fewer parent/child and sibling conflicts and lower levels of stress for the parents and children. For this reason, children’s levels of internalizing problems may have been reduced as well.

Clearly, further research is needed to identify and understand the mechanisms by which the KEEP intervention, and other similar behavioral-based interventions, impacts internalizing behavior problems. Directions for future research include examining the potential mediating effects of parental use of positive reinforcement, nonharsh disciplinary practices, and consistent limit setting, as well as reductions in levels of family conflict, on reducing internalizing behavior problems. Such research would be enhanced by supplementing parental reports of parenting behaviors and child behavior problems with direct observations of parents’ use of parenting strategies and with child self-reports of internalizing symptoms. Further insight could be gained by using parent, child, and observer ratings of the qualitative aspects of the parent/child relationship, including reciprocal expressions of positive regard, levels of family conflict, and quality of sibling interactions.

As noted in the “Method” section, if the PC left the home for any reason between baseline and follow-up, a substitute child became the PC. Thus, in several instances, follow-up assessments were conducted with a different child from the one who was assessed at baseline. Regardless of these substitutions, there were still greater reductions in externalizing problems (as indicated by a decreased proportion of children in the borderline group for rule-breaking) and internalizing problems (as indicated by decreases in T-scores for internalizing problems and anxious depression, and a decreased proportion of children in the borderline groups for internalizing and withdrawal problems) seen in the intervention group compared to those in the control condition. Thus, these findings provide some support for the potential generalizability of the effects of the KEEP intervention to other children in the same home.

The current study has several limitations. First, the caregivers were not blind to their study condition. After giving consent and then being randomized to a group, the parents knew the condition to which they were assigned. This knowledge may have influenced the parents’ perceptions of their children’s levels of behavior problems between baseline and termination. Parents in the intervention group may have expected positive changes in their child’s behavior over the course of the intervention and therefore rated their children lower on the CBCL scales at follow-up. Conversely, parents in the control group may not have expected any changes in their child’s behavior and their ratings on the CBCL scales at follow-up could have reflected this bias. Second, caregivers were the only informants of the children’s externalizing and internalizing behavior problems. Additional informants, especially children and perhaps teachers, would have provided a more comprehensive assessment of both externalizing problems and internalizing problems. Third, 44 of the original participant families did not complete the baseline interviews and 61 families who completed the baseline interviews did not complete the follow-up interviews.

As noted earlier, children in foster care are at risk for a variety of behavioral health and adjustment issues (e.g., Bronsard et al., 2016; Oswald et al., 2010). Unfortunately, the mental health needs of such children often go unmet (Kerker & Dore, 2006), and the consequences of these unmet needs may include conflictual relationships with peers, difficulties in school, placement disruptions, and risk of psychopathology in adolescence and adulthood. Kerker and Dore (2006) outline a number of barriers to receiving mental health care for children in foster care, including inadequacy of mental health screening, over-burdened caseworkers, and inadequately trained foster and relative caregivers. Given these barriers and challenges, it has been suggested that foster parents serve as therapeutic agents of change within the child welfare system (Chamberlain et al., 2008) and be provided with the training to better identify and manage children’s mental health challenges (e.g., Chamberlain et al., 2008; Horwitz et al., 2010; Kerker & Dore, 2006). Research on the KEEP foster parent training intervention has demonstrated that it is possible to train foster and kinship caregivers to manage children’s daily behavior problems. The findings from the current investigation suggest that the same training may also provide foster caregivers with tools to address a broad range of behavior problems. Thus, implementation of the KEEP intervention or similar evidence-based interventions has the potential to expand the mental health services available within child welfare systems to a wider range of children in foster care, and to do so in a cost-effective manner.

Footnotes

Acknowledgments

The authors thank San Diego County Child Welfare Services Director, Debra Zanders-Willis and Deputy Director, Roseann Myers; Social Advocates for Youth Supervisor, Shannon Throop; lead interventionists, Norma Talamantes, Blanca Mancillas, Moniseha Cole, and Sonia Miramontes; and the foster and relative caregivers who participated.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Ethical approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Support for this research was provided by grant 2R01MH060195-06A1 from the National Institute of Mental Health.

References

Achenbach

T. M.

(1991). Integrative guide for the 1991 CBCL 14–18, YSR, and TRF profiles. Burlington, VT: University of Vermont, Department of Psychiatry.

Achenbach

T. M.

Rescorla

L. A.

(2001). Manual for the ASEBA school-age forms & profiles: An integrated system of multi-informant assessment. Burlington, VT: University of Vermont, Research Center for Children, Youth & Families.

Brendgen

Vitaro

Tergeon

Poulin

(2002). Assessing aggressive and depressed children’s social relations with classmates and friends: A matter of perspective. Journal of Abnormal Child Psychology, 30, 609–624.

Broidy

L. M.

Tremblay

R. E.

Brame

Fergusson

Horwood

J. L.

Laird

… Vitaro

. (2003). Developmental trajectories of childhood disruptive behaviors and adolescent delinquency: A six-site, cross-national study. Developmental Psychology, 39, 222–245.

Bronsard

Alessandrini

Fond

Loundou

Auquier

Tordjman

Boyer

(2016). The prevalence of mental disorders among children and adolescents in the child welfare system: A systematic review and meta-analysis. Medicine, 95, e2622.

Chamberlain

Leve

DeGarmo

D. S.

(2007). Multidimensional treatment foster care for girls in the juvenile justice system: 2-year follow-up of a randomized clinical trial. Journal of Consulting and Clinical Psychology, 75, 187–193.

Chamberlain

Price

J. M.

Leve

Laurent

Landsverk

Reid

J. B.

(2008). Prevention of behavior problems for children in foster care: Outcomes and mediation effects. Prevention Science, 9, 17–27.

Chamberlain

Price

J. M.

Reid

J. B.

Landsverk

(2008). Cascading implementation of a foster and kinship parent intervention. Child Welfare: Journal of Policy, Practice, and Program, 87, 27–48.

Chamberlain

Price

J. M.

Reid

J. B.

Landsverk

Fisher

P. A.

Stoolmiller

(2006). Who disrupts from placement in foster and kinship care? Child Abuse and Neglect, 30, 409–424.

10.

Crum

R. M.

Green

K. M.

Storr

C. L.

Chan

Ialongo

Stuart

E. A.

Anthony

J. C.

(2008). Depressed mood among youth reporting alcohol use: Associations with subsequent alcohol involvement from childhood though young adulthood. Archives of General Psychiatry, 65, 702–712.

11.

Erath

S. A.

Keiley

M. K.

Pettit

G. S.

Lansford

J. E.

Dodge

K. A.

Bates

J. E.

(2009). Behavioral predictors of mental health service utilization in childhood through adolescence. Journal of Developmental and Behavioral Pediatrics, 30, 481–488.

12.

Galambos

N. L.

Barker

E. T.

Almeida

D. M.

(2003). Parents do matter. Trajectories of change in externalizing and internalizing problems in early adolescence. Child Development, 74, 578–594.

13.

Greeno

E. J.

Uretsky

M. C.

Lee

B. R.

Moore

J. E.

Barth

R. P.

Shaw

T. V.

(2016). Replication of the KEEP foster and kinship parent training program for youth with externalizing behaviors. Children and Youth Services Review, 61, 75–82.

14.

Gupta

S. K.

(2011). The intent-to-treat concept: A review. Perspectives in Clinical Research, 2, 109–112.

15.

Horwitz

S. M.

Chamberlain

Landsverk

Mullican

(2010). Improving the mental health of children in child welfare through the implementation of evidence-based parenting interventions. Administration and Policy in Mental Health and Mental Health Services and Research, 37, 27–39.

16.

Kerker

B. D.

Dore

M. M.

(2006). Mental health needs and treatment of foster youth: Barriers and opportunities. American Journal of Orthopsychiatry, 76, 138–147.

17.

Landsverk

Garland

A. F.

Leslie

L. K.

(2002). Mental health services for children reported to child protective services. In Myers

J. E.

Berliner

Briere

Hendrix

T. C.

Jenny

(Eds.), The APSAC handbook on child maltreatment (2nd ed., pp. 487–507). Thousand Oaks, CA: Sage.

18.

Laskey

B. J.

Cartwright-Hatton

(2009). Parental discipline behaviours and beliefs about their child: Associations with child internalizing and mediation relationships. Child: Care, Health and Development, 35, 717–727.

19.

Marmorstein

N. R.

(2009). Longitudinal associations between alcohol problems and depressive symptoms: Early adolescence through early adulthood. Alcoholism: Clinical and Experimental Research, 33, 49–59.

20.

McLeod

B. D.

Weisz

J. R.

Wood

J. J.

(2007). Examining the association between parenting and childhood depression: A meta-analysis. Clinical Psychology Review, 27, 986–1003.

21.

National Survey of Child and Adolescent Well-Being Group. (2003). National survey of child and adolescent well-being (NSCAW one year in foster care wave 1 data analysis report). Washington, DC: U.S. Department of Health and Human Services, Administration for Children and Families.

22.

Newton

R. R.

Litrownik

A. J.

Landsverk

J. A.

(2000). Children and youth in foster care: Disentangling the relationship between problem behaviors and number of placements. Child Abuse and Neglect, 24, 1363–1374.

23.

Oswald

S. H.

Heil

Goldbeck

(2010). History of maltreatment and mental health problems in foster children: A review of the literature. Journal of Pediatric Psychology, 35, 462–472.

24.

Price

J. M.

Chamberlain

Landsverk

Reid

J. B.

Leve

Laurent

(2008). Effects of a foster parent training intervention on placement changes of children in foster care. Child Maltreatment, 13, 64–75.

25.

Price

J. M.

Roesch

Walsh

N. E.

(2012). Effectiveness of the KEEP foster parent intervention during an implementation trial. Children and Youth Services Review, 12, 2487–2494.

26.

Price

J. M.

Roesch

Walsh

N. E.

(2015). Effects of the KEEP foster parent intervention on child and sibling behavior problems and parental stress during a randomized implementation trial. Prevention Science, 16, 685–695.

27.

Rapee

R. M.

Schniering

C. A.

Hudson

J. L.

(2009). Anxiety disorders during childhood and adolescence: Origins and treatment. Annual Review of Clinical Psychology, 5, 311–341.

28.

Rapport

M. D.

Denney

C. B.

Chung

K. M.

Hustace

(2001). Internalizing behavior problems and scholastic achievement in children: Cognitive and behavioral pathways as mediators of outcome. Journal of Clinical Child Psychology, 30, 536–551.

29.

Raudenbush

S. W.

Bryk

A. S.

(2002). Hierarchical linear models: Applications and data analysis methods (2nd ed.). Thousand Oaks, CA: Sage.

30.

Tarren-Sweeney

M. J.

Hazell

P. L.

Carr

V. J.

(2004). Are foster parents reliable informants of children’s behaviour problems? Child: Care, Health and Development, 30, 167–175.

31.

Yap

M. B. H.

Jorm

A. F.

(2015). Parental factors associated with children anxiety, depression, and internalizing problems: A systematic review and meta-analysis. Journal of Affective Disorders, 175, 424–440.