Trajectory of Externalizing Child Behaviors in a KEEP Replication

Abstract

Objective:

The purpose of this study is to examine the correlates of child behavior change over time in a replication of the KEEP intervention.

Method:

The study sample was drawn from the treatment group of the Maryland replication of KEEP (n=65). Change over time was analyzed using multilevel linear mixed modeling.

Results:

Parents’ use of positive reinforcement relative to discipline was associated with the rate of child behavior change among program participants; parents with the lowest initial levels of reinforcement reported the greatest decrease in child problem behaviors. Other participant characteristics were not associated with child behavior change during the study period.

Conclusions:

The results indicate the efficacy of an evidence-based foster parent training program for reducing child problem behaviors and underscore the utility of teaching parents to use more positive responses relative to discipline as a robust path to improved child outcomes.

Keywords

foster parent training conduct problems behavior change linear mixed modeling

According to the most recent wave of data available from the National Survey of Child and Adolescent Well-Being, more than 40% of children in foster care display clinical levels of externalizing behavior problems (Casanueva, Tueller, Smith, Dolan, & Ringeisen, 2014). High frequencies of externalizing behavior problems in children have been linked to poor mental health, academic and employment outcomes for the child, and depression and anxiety among the adults who care for them (Barlow, Coren, & Stewart-Brown, 2002; Burke, Loeber, & Birmaher, 2002; Dretzke et al., 2009; Gross, Shaw, Burwell, & Nagin, 2009; Harden & Whittaker, 2011). Externalizing behaviors in children are also likely to influence the nature of interactions between a caregiver and their child. Increased attention provided by the caregivers may unwittingly reinforce negative child behaviors which, in turn, may escalate harsh parenting (Chamberlain et al., 2008; Patterson, DeBaryshe, & Ramsey, 1990). For children in foster care, worsening behavior can lead to unstable and restrictive placements (Fisher, Stoolmiller, Mannering, Takahashi, & Chamberlain, 2011; James, Landsverk, & Slymen, 2004). To disrupt this cycle, researchers have suggested implementing parent training programs that help caregivers reduce the attention paid to negative behaviors and increase rewards for positive behaviors (Barth et al., 2005).

Background

A sizable field of empirical literature has explored the benefits of parenting interventions. A Cochrane review conducted by Furlong et al. (2013) assessed the evidence for cognitive and cognitive–behavioral-based parenting programs delivered in a group format. The authors identified 13 randomized and quasi-experimental trials of cognitive and cognitive–behavioral-based group parenting programs for primary caregivers of children aged 3 to 12 years with clinical levels of conduct problems (n = 1,078). Furlong et al. (2013) found significant reductions in child conduct problems for treatment group participants. Positive results were consistent whether child behaviors were measured by parents report or independently assessed. The authors concluded that cognitively oriented group-based parenting programs were effective, for parents of youth with significant behavior concerns. This coheres with the findings reported in other meta-analyses of parent management programs (Dretzke et al., 2009; Michelson, Davenport, Dretzke, Barlow, & Day, 2013).

A smaller but growing body of literature has evaluated the effectiveness of parenting interventions delivered to foster and kinship caregivers. A Cochrane review (Turner, MacDonald, & Dennis, 2007) identified just six randomized or quasi-experimental studies of cognitive–behavioral-based parent training programs delivered to foster parents. For foster parents (n = 463) in these studies, the authors identified no overall evidence for the effectiveness of the cognitive–behavioral training programs on child behavior problems, psychological, or interpersonal functioning. Turner, Macdonald, and Dennis (2007) concluded that there was insufficient evidence in this area to guide best practices, highlighting the need for additional studies with larger samples, more precise measures, diverse sampling strategies, and varied program designs.

Subsequent to the Turner et al. (2007) review, increasingly robust evaluations have provided support for the efficacy of group-based foster parent training programs in reducing problem behaviors among children in foster care (Dorsey et al., 2008; Festinger & Baker, 2013). Although the current evidence base for foster parent training programs is uneven, recent studies of group-based foster parent trainings using standardized curricula and robust research designs that measure parent and/or child behavior change have emerged with promising results (Dorsey et al., 2008; Festinger & Baker, 2013). One such program is Keeping Foster and Kinship Parents Supported (KEEP), a 16-week training and support group for foster and kinship caregivers of children between the ages of 4 and 12. In KEEP, caregivers learn to implement research-supported behavior management strategies aimed at reducing the frequency of externalizing problem behaviors exhibited by the children in their care (Chamberlain et al., 2008).

A pilot study of the KEEP model found a significant reduction in child behavior problems among the parents who received the intervention compared to parents who received treatment as usual or an additional stipend with no training (Chamberlain, Moreland, & Reid, 1992). These results were later replicated in a randomized control trial (n = 700) that found significant reductions in the number of externalizing child behavior problems and increased placement stability for the KEEP participants compared with the control group (Chamberlain et al., 2008; Price et al., 2008). In addition, Chamberlain et al. (2008) found that the intervention’s effect on child problem behaviors was partially mediated by parents’ increased use of positive reinforcement relative to discipline. This effect was present among children whose caregivers reported greater than six child problem behaviors in a 24-hr period before starting the intervention (high risk) but not for children below that threshold (low risk). In addition, the children in the high-risk group experienced a greater decrease in problem behaviors compared to children in the low-risk group during the study period. KEEP has since been replicated in multiple geographically diverse settings including Maryland, New York City, the United Kingdom, and Sweden.

In KEEP, parents are taught to reduce the attention paid to negative behaviors and reward positive behaviors in order to reduce the frequency of problem behaviors among the children in their care. On average, caregivers who participate in KEEP report a decline in child problem behaviors, but there are participants for whom this is not true. Hurlburt, Chamberlain, DeGarmo, Zhang, and Price (2010) used growth mixture models to identify four trajectory classes that described the child behavior patterns of KEEP participants over the 16 weeks of treatment. Most children in the sample had a low rate of problem behaviors that remained low throughout the study (69%), about 15% started high and decreased, and 12% started high and stayed high. The remaining 5% started low but saw behaviors increase during the study period. Overall, studies of KEEP have consistently demonstrated a favorable intervention effect on externalizing problem behaviors, placement stability, and permanency outcomes (Buchanan, Chamberlain, Price, & Sprengelmeyer, 2013; Greeno et al., Under Review; Price, Roesch, & Walsh, 2012). Still the findings reported by Hurlburt et al. (2010) highlight the need to better understand the process and correlates of change in parenting interventions, so that caregivers in need of additional supports or alternate programming could be identified and served.

The Present Study

The present study was a replication of the KEEP program delivered in accordance with the original KEEP model with a minor revision, made at the suggestion of the model developers, that the minimum age for inclusion in the study was age 4 instead of age 5 (see Price, Chamberlain, Landsverk, & Reid, 2009 for model description). Previous evaluations of both the KEEP model and foster parent training programs in general have used pre–post data to understand the impact of group participation on child outcomes. The present study provides a unique contribution to the literature by examining the course and correlates of child behavior change among participants in a parenting intervention. The following two research questions were addressed: (1) How do reports of child behavior problems change between pretreatment and 2 months post-intervention for KEEP participants? and (2) Are participant characteristics associated with individual differences in the patterns of child behavior change reported by KEEP participants?

Method

Participants

The study sample was drawn from the Maryland replication of KEEP (Greeno et al., Under Review). The sample consisted of foster and kinship parents providing care for children between the ages of 4 and 12. Children were required to have been in the current placement for a minimum of 1 month and not be considered medically fragile. Seventy-five foster and kinship caregivers, representing 65 households, participated in 9 cohorts of KEEP between 2010 and 2012. Only one caregiver per home was included in the evaluation. All foster and kinship parents who consented to participate in the KEEP replication were included in the analysis (N = 65). Selected demographic characteristics of participants and the target children are presented in Table 1.

Table 1.
Sample Demographics for the Maryland KEEP Evaluation Participants.

Foster/Kin provider Child

$\bar{x}$ /f (SD)/% $\bar{x}$ /f (SD)/%

Age 50 (7.46) 7.5 (2.29)

Gender

Male 1 1% 33 51%

Female 64 99% 32 49%

Ethnicity

Black/African American 33 51% 31 48%

White/Caucasian 21 32% 21 32%

Other 11 17% 11 17%

Missing — — 2 3%

Placement type

Relative/kinship provider 23 35% — —

Licensed state foster provider 42 65% — —

Group attendance 12.5 78% — —

Proportion of positive reinforcement .69 (.08) — —

Initial risk of disruption^a

Low — — 24 37%

High — — 41 63%

CBCL Externalizing Scale score

Clinical — — 25 38%

Subclinical — — 40 62%

Note. N = 65. KEEP = Keeping Foster and Kinship Parents Supported; CBCL = Child Behavior Checklist; PDR = Parent Daily Report.

^aCalculated according to the Chamberlain et al. (2006) standard: low risk = PDR ≤ 6; high risk = PDR > 6.

Measures

Child behaviors were measured using the Parent Daily Report (PDR; Chamberlain & Reid, 1987), a 31-item measure of parent-reported behavior problems (e.g., arguing, complaining, and back talk). Caregivers were asked to recall the past 24 hr and respond yes or no to whether the identified foster or kinship child displayed the behavior. At pretreatment and the 2-month follow-up time points, the PDR score was informed by two administrations within close succession (usually 1 week) and averaged to obtain a stable measure of functioning. The internal consistency of the PDR in this full sample was sufficient (Cronbach’s α = .82 pretreatment and .85 posttest).

Initial risk status was assessed based on prior research by Chamberlain et al. (2006) that identified an increased risk of placement disruptions for children with greater than six behaviors on the PDR. This standard has been used in multiple studies as a moderator of KEEP treatment effect (Chamberlain et al., 2008; Price et al., 2012). A dichotomous variable was created using the pretreatment value of the PDR, so that a child with a pretreatment PDR score less than or equal to 6 was considered low risk (0) and a child with a pretreatment PDR score greater than 6 was considered high risk (1).

The proportion of positive reinforcement is a composite scale developed by the Oregon Social Learning Center (OSLC) to measure parent’s use of behavior management techniques (Chamberlain et al., 2008; Price et al., 2012). Caregivers are asked about the use of specific discipline and positive reinforcement techniques in the previous 24 hours. In addition, caregivers are asked to report their overall use of parent management techniques (positive reinforcement and discipline) on a 7-point Likert-type scale ranging from “3+ times a day” to “less than once a month” to “never.” The item scores are combined to create a scale score that describes the use of positive reinforcement techniques in relation to the total use of parent management techniques. A higher score means more positive reinforcement and less discipline.

Children’s externalizing problems were measured using the Externalizing Scale of the Child Behavior Checklist (CBCL; Achenbach & Rescorla, 2001). The CBCL was completed on the telephone by caregivers once at pretreatment. For the analysis, scores were dichotomized as being in the clinical range compared to the subclinical/normal range. The internal consistency of the CBCL in this sample was sufficient (Cronbach’s α = .91 at pretreatment and .93 at posttest).

Child age in years was measured at the time the parent consented to participate. Attendance rates were calculated as the proportion of group sessions attended in person. Child gender (male or female) and placement type (kinship or nonrelative foster care) were also measured.

Procedures

The present study was approved by the University of Maryland Institutional Review Board. A single-group longitudinal cohort design was used. State child welfare workers contacted foster and kinship families on their caseloads and encouraged them to participate in the program. Facilitators then visited the families in their homes prior to the trainings to answer questions about the program and obtain written consent for participation in the evaluation.

Pretest data on parenting practices and child behaviors were collected approximately 1 week prior to the start of the KEEP training. Additional data on child behaviors were collected weekly during the 16-week intervention and again approximately 2 months after the completion of the KEEP training. Only one caregiver and one child per home were eligible for the study. For consistency, the same foster or kinship parent was asked to answer all questionnaires. Time was measured in weeks (max = 25), with the start date set as 0; pretreatment measures were negative numbers and the weeks during and after treatment commenced were positive numbers. A maximum of 18 time points were analyzed for each youth. Weekly behavior scores were nested within each participant. Out-of-home placement type was defined using administrative data provided by the state child welfare services agency.

Facilitators were trained by consultants from the OSLC over a period of 5 days. Trainees were coached on the KEEP curriculum, group logistics, recruitment, and facilitation. After the initial training period, facilitators received weekly supervision from an OSLC consultant. To ensure fidelity to the model, all of the group sessions were recorded for review by the OSLC consultant. Feedback from the video review was provided during the weekly supervision sessions. Facilitators could be certified to lead groups without supervision after having facilitated three full cycles of the 16-week groups. To ensure fidelity, facilitators continued to submit video recordings of each sessions for intermittent review.

Analysis

Exploratory analysis found sufficient within- and between-subjects variation in PDR scores across time points to justify the use of multilevel statistical methods that can account for the nesting of the dependent variable (PDR scores) within subjects (Singer & Willett, 2003). Therefore, a series of multilevel linear mixed models (growth models) were used to assess the trajectory of externalizing child behaviors (PDR) from pretreatment to follow up and model whether participant characteristics affect the trajectory of child behavior change. All analyses were conducted using SPSS v 21 (IBM, 2012).

All assumptions of multilevel growth models were met. Maas and Hox (2005) suggest using a sample size greater than 50 at the second level of nesting in multilevel models to ensure unbiased estimates of standard errors. The present study exceeded this criteria. Polynomial and quadratic transformations of the weekly PDR scores were tested. The linear model showed superior fit and was used in the analyses. A model-building approach was used as suggested by Singer and Willett (2003). Model fit was assessed by comparing −2 log likelihood (deviance), Akaike’s Information Criterion (AIC), and Schwarz’s Bayesian Information Criterion (BIC) between each successive model.

To gain a better understanding of the differential impact of the intervention, individual trajectories of child behavior were identified (Donaldson, 2008). The data file was split by case ID and a regression was run for each participant using a 50% confidence interval. If the confidence interval contained zero, the slope was classified as flat. If the confidence interval did not contain zero, the regression coefficient was used to determine the direction of the slope. A participant with a positive slope was classified as having increasing frequency of problem behaviors over time. Participants with a negative slope were classified as having a decreasing frequency of problem behaviors over time. Frequencies were calculated to assess the number of participants who had increasing, decreasing, or flat slopes, and bivariate analyses were run to identify correlates of exhibiting a particular trajectory. Finally, a binomial logistic regression assessed whether any participant characteristics were associated with exhibiting decreasing behaviors versus no change or increasing behaviors.

Missing Data

Of the 65 parents who agreed to participate, 56 (86%) completed the intervention. Four parents consented to the study and completed the pretest but never attended the group. Three parents dropped out after attending fewer than three group sessions and two parents dropped out of the study after attending half or more sessions due to issues unrelated to the intervention. Among parents who completed the group, parents attended 78% (12.5 of 16) of the sessions and completed 74% of PDR interviews (a mean of 13.4 of a possible 18 measurement points; SD = 2.94). All participants completed the pretreatment PDR interviews (n = 65) and 43 caregivers completed the follow-up interviews. The proportion of positive reinforcement was missing for only one case in the entire sample.

All participants who consented to take part in the evaluation were included in the analysis. Longitudinal multilevel models are able to handle data in which participants have missing or unevenly spaced observations (Shek & Ma, 2011; Singer & Willett, 2003). Among the nine participants who did not complete the study, five had pretreatment data only. For the four participants with additional data points, there were no clear observable patterns or similarities in the PDR profiles or demographic characteristics that differentiated them from those who completed the group. Sensitivity to missing data was assessed by running analyses that included and excluded dropouts. Model estimates were not affected by the inclusion of subjects who dropped out of the study.

Results

A series of multilevel models were conducted to measure child behavior change (using PDR score) across the 16 weeks of the intervention and a 2-month follow-up. First, the unconditional means model was estimated, which identifies the amount of variation in PDR scores due to within and between participants differences (Table 2). The intra-class correlation of the model was .69, indicating that 69% of the variance in PDR scores was accounted for by differences between participants, justifying the use of a multilevel model (Singer & Willett, 2003).

Table 2.
Summary of Results for the Multilevel Growth Model Fitted to Evaluate the Contributions of Participant Characteristics on the Trajectory of Child Behavior Change^a for KEEP Group Participants.

Parameter Unconditional Means Model Unconditional Growth Model Final Model

Fixed effects

Initial status, $π_{0 i}$

Intercept $γ_{00}$ 5.55* (0.40) 6.39* (0.49) 15.77* (3.64)

Positive parenting ratio $γ_{01}$ −16.78 (5.01)

Initial risk of disruption^b $γ_{02}$ 2.30 (0.80)

Rate of change, $π_{1 i}$

Intercept $γ_{10}$ −0.13* (0.03) −0.78** (0.25)

Positive parenting ratio $γ_{11}$ 0.94* (0.36)

Variance components

Level 1

Within person $σ_{ε}^{2}$ 11.65 (0.62) 10.48 (0.59) 10.56 (0.59)

Level 2

In initial status $σ_{0}^{2}$ 9.2 (1.86) 12.76 (2.71) 5.94 (1.49)

In rate of change $σ_{1}^{2}$ 0.02 (0.01) 0.02 (0.01)

Covariance $σ_{01}$ −0.35 (0.14) −0.18 (0.01)

Goodness of fit

Deviance 4,258.46 4,214.773 4,125.238

AIC 4,264.46 4,226.773 4,143.238

BIC 4,278.42 4,254.705 4,185.020

% Reduction in initial status variance 53.45%

% Reduction in rate of change variance 17.87%

Note. N = 65. KEEP = Keeping Foster and Kinship Parents Supported; AIC = Akaike Information Criterion; BIC = Bayesian Information Criterion; PDR = Parent Daily Report.

^aParent Daily Report.

^bHigh initial risk compared to low initial risk group calculated according to the Chamberlain et al. (2006) standard: low risk = PDR ≤ 6; high risk = PDR > 6.

p < .05; p < .01; p < .001.

Time was included as an independent variable to estimate the amount of change in PDR scores that is due to the passage of time. The mean pretreatment PDR for the sample was 6.39 (SE = .049), which would be considered high risk for placement disruption (Chamberlain et al., 2006). The mean slope for the model with time as the only predictor was −0.13 (SE = .03). At follow-up (i.e., 8 weeks posttreatment), the average KEEP participant would report a decrease of more than three child problem behaviors on the PDR.

Next, we tested the effect of participant characteristics on both pretreatment mean PDR scores and the rate of change. Participant characteristics were tested individually to assess their impact on the pretreatment mean and then with an interaction with time to assess their impact on the mean rate of behavior change (slope). When model fit was improved by inclusion of the item or the associated interaction term, the item was retained in subsequent models. When an item or interaction term was no longer significant after the inclusion of additional items, the nonsignificant item would be removed and a decision to retain the item would be made according to the impact on model fit. Kinship status, child age, and child gender did not improve model fit and were therefore removed from subsequent models. Attendance and CBCL Externalizing Scale scores were significant predictors of pretreatment mean PDR scores and the rate of change in intermediate models, but detracted from model fit when initial risk status was included in the model; attendance and CBCL were, hence, removed from subsequent models.

The final multilevel model for measuring the trajectory of child behavior change included time, initial risk status (high vs. low), proportion of positive reinforcement, and an interaction term for the impact of proportion of positive reinforcement and time as independent variables. The intercept for initial risk status was significant, indicating the mean pretreatment difference between high-risk (pretreatment PDR > 6) versus low-risk (pretreatment PDR ≤ 6) groups was not zero, which was expected. The slope for initial risk was nonsignificant, indicating that the rate of behavior change for participants who started out in the high-risk group was not different from participants who started in the low-risk group. Parents who reported using less discipline and more positive reinforcement at pretreatment reported a significantly lower initial PDR scores (b = −16.78; p < .001). Finally, the interaction between proportion of positive reinforcement and time was significant, indicating that the pretreatment proportion of positive reinforcement affected the rate of change in PDR scores over time (b = .94; p < .05).

An illustration of the impact of the proportion of positive reinforcement on the trajectory of child problem behaviors for children in the high-risk category is presented in Figure 1. Regression lines were calculated and plotted using the value for proportion of positive reinforcement at the mean, 1 SD above the mean, and 1 SD below the mean (Cohen, Cohen, West, & Aiken, 2002). Parents with a lower proportion of positive reinforcement experienced the steepest decrease in the frequency of behavior problems on the PDR over time (b = .065; p < .01). As the 16-week intervention neared its end, the regression lines changed rank order, so that by the 2-month follow-up, parents with the lowest pretreatment proportion of positive reinforcement relative to discipline reported the lowest PDR behavior counts. Approximately 54% of the variance in the mean pretreatment PDR value and 18% of the variance in the PDR rate of change was explained by the inclusion of proportion of positive reinforcement and initial risk status.

Figure 1.
Impact of caregivers use of positive reinforcement relative to discipline at pretreatment on the trajectory of Parent Daily Report (PDR) behavior counts for children with a high initial risk of disruption (PDR > 6). Note. Within 1 SD of mean proportion of positive reinforcement (n = 44); ≥ 1 SD above mean proportion of positive reinforcement (n = 9); ≤ 1 SD below mean proportion of positive reinforcement (n = 11).

To better understand the differential experiences of KEEP group participants, individual trajectories of child behavior change were identified for 60 of the participants (see Table 3). The slope could not be determined for five participants who had less than two recorded PDR scores. More than half of the participants (53%; n = 32) had a negative slope, indicating that the number of behavior problems decreased during the course of the study. About 17% (n = 10) of the participants had positive slopes, indicating an increase in PDR counts over time, and about 30% (n = 18) of the participants experienced no change in their PDR scores during the study period. Bivariate analyses, as well as a binomial logistic regression, were conducted in an attempt to identify factors that could differentiate the participants that improved during the intervention period from those who plateaued or got worse. No participant characteristics or behaviors were significantly associated with belonging to the group with improving behaviors.

Table 3.
Child Behavior Change^a by Initial Risk of Disruption.^b

Initial Risk

Whole Sample Low Risk High Risk

f % f % f %

Whole sample 60 100 22 36 38 63

No change (flat slope) 18 30 8 36 10 26

Increasing PDR (positive slope) 10 17 3 14 7 18

Decreasing PDR (negative slope) 32 53 11 50 21 55

Note. PDR = Parent Daily Report.

^aParent Daily Report.

^bCalculated according to the Chamberlain et al. (2006) standard: low risk = PDR ≤ 6; high risk = PDR > 6.

Discussion and Applications to Social Work

The study aimed to identify the patterns and correlates of behavior change for participants in a research-supported foster parent training program. Participants in the KEEP group reported a significant reduction in child problem behaviors over time; however, some participants reported no change and a small minority reported a worsening of child behaviors during the study period. In the final model placement type, child age, child gender, group attendance, and initial CBCL externalizing score were not significantly associated with the initial PDR score or the change in child problem behaviors reported by participants during the study period. Only the interaction between the proportion of positive parenting and time affected the trajectory of behavior change in this study sample.

Parents who reported using less positive reinforcement and more discipline before attending the KEEP group reported the greatest decreases in child behavior change over the course of the study. This coheres with earlier research that found the proportion of positive reinforcement employed by caregivers to partially mediate the KEEP intervention effect (Chamberlain et al., 2008). Together these findings suggest that KEEP’s greatest strength may be in changing how parents reinforce youth and implement discipline, especially among parents who start with a low proportion of positive reinforcement relative to discipline. The primary drivers of child behavior change in this study are parenting style and time, providing additional support for the theory of change prominent in parenting interventions, and that parents are the key lever for change in child behavior.

Previous studies have found a greater decrease in problem behaviors for children in the high initial risk group (PDR > 6 at pretreatment) compared to children in the low-risk group (Chamberlain et al., 2008; Price et al., 2012). In the current study, behavior change over time was not differentiated by initial risk after controlling for the proportion of positive reinforcement. These findings suggest parent training programs like KEEP may be beneficial even for parents with children who are at lower risk of placement disruption.

Despite findings from an older meta-analysis, Turner et al. (2007) who suggested no evidence for the effectiveness of cognitive–behavioral foster parent training programs, the majority of participants in the current study reported improved child behaviors over time. Almost one third of participants reported no change in child behavior problems during the study period and 17% actually got worse during the course of the intervention. Although the goal of the intervention was to reduce the incidence of child behavior problems, children in foster care tend to exhibit increased problem behaviors over time (Aarons et al., 2010). Thus the maintenance of pretreatment levels of problem behaviors among a large portion of the group participants may be interpreted as programmatic success.

The patterns of child behavior change identified in this sample are similar to those found by Hurlburt et al. (2010); however, the proportion of the sample in each group is quite different. In this study, 63% of the sample began with high PDR scores, compared to about 26% in Hurlburt et al. (2010). Additionally, 36% of the current sample started with low PDR scores that remained low, compared to almost 70% in Hurlburt et al. (2010). The inclusion of more challenging youth in the current study is likely a reflection of the recruitment strategy that focused on the behavior change potential of KEEP for families with very difficult children.

Additional analyses identified no significant predictors that could differentiate the participants who improved during the intervention period from those who plateaued or got worse. More study is needed to better understand whether there are subgroups of participants in research-supported foster parent training programs who need alternate or supplemental training. The results from this study confirm previous findings that participation in group-based foster parent training programs is associated with improved child behaviors and that the proportion of positive reinforcement impacts the rate of change (Chamberlain et al., 2008). Additional research that consider the correlates of change in parenting interventions are needed to understand the differential effectiveness of program participation and improve services for those who are not experiencing the expected decrease in child problem behaviors.

The current analysis is limited by the lack of a control group to provide a counterfactual for participation in the KEEP group. In addition, a larger sample size would decrease the possibility of Type II errors and expand the types of analyses that are possible. Any nonsignificant findings should be confirmed with a larger sample size. Due to sample size at the cohort level, we could not include facilitator or cohort characteristics in the multilevel model, which may have provided important insight into between-group differences. Despite these limitations, the present study provided important insight into the process of change for participants in a research-supported foster parent training program.

In summary, the present study found that on average, participation in the KEEP intervention resulted in decreased frequency of child behaviors problems. In addition, the rate of change was dependent on the parent’s use of positive reinforcement, supporting the findings from the original KEEP trial (Chamberlain et al., 2008). Parents whose proportion of positive reinforcement relative to discipline was low at pretreatment reported the greatest decrease in child behavior problems during the study period. This relationship held even for parents of children in the high initial risk group, strengthening the argument for the effectiveness of research-supported parenting programs in reducing child problem behaviors among diverse samples (Huey & Polo, 2008), in this case, of foster children. Teaching parents to use more positive responses relative to discipline with their children appears to be an important and robust path to improve child outcomes, regardless of pretreatment levels of problem behaviors. Importantly, children of the foster parents who had the least favorable amalgam of parenting strategies before the intervention benefited the most during and after participating in the group. The findings presented here provide additional support for the utility of programs such as KEEP that train foster and kinship parents to manage difficult child behaviors.

	Foster/Kin provider	Child
Age	50	(7.46)	7.5	(2.29)
Gender
Male	1	1%	33	51%
Female	64	99%	32	49%
Ethnicity
Black/African American	33	51%	31	48%
White/Caucasian	21	32%	21	32%
Other	11	17%	11	17%
Missing	—	—	2	3%
Placement type
Relative/kinship provider	23	35%	—	—
Licensed state foster provider	42	65%	—	—
Group attendance	12.5	78%	—	—
Proportion of positive reinforcement	.69	(.08)	—	—
Initial risk of disruption^a
Low	—	—	24	37%
High	—	—	41	63%
CBCL Externalizing Scale score
Clinical	—	—	25	38%
Subclinical	—	—	40	62%

	Parameter	Unconditional Means Model	Unconditional Growth Model	Final Model
Fixed effects
Initial status, $π_{0 i}$
Intercept	$γ_{00}$	5.55*** (0.40)	6.39*** (0.49)	15.77*** (3.64)
Positive parenting ratio	$γ_{01}$			−16.78** (5.01)
Initial risk of disruption^b	$γ_{02}$			2.30** (0.80)
Rate of change, $π_{1 i}$
Intercept	$γ_{10}$		−0.13*** (0.03)	−0.78** (0.25)
Positive parenting ratio	$γ_{11}$			0.94* (0.36)
Variance components
Level 1
Within person	$σ_{ε}^{2}$	11.65 (0.62)	10.48 (0.59)	10.56 (0.59)
Level 2
In initial status	$σ_{0}^{2}$	9.2 (1.86)	12.76 (2.71)	5.94 (1.49)
In rate of change	$σ_{1}^{2}$		0.02 (0.01)	0.02 (0.01)
Covariance	$σ_{01}$		−0.35 (0.14)	−0.18 (0.01)
Goodness of fit
Deviance		4,258.46	4,214.773	4,125.238
AIC		4,264.46	4,226.773	4,143.238
BIC		4,278.42	4,254.705	4,185.020
% Reduction in initial status variance				53.45%
% Reduction in rate of change variance				17.87%

		Initial Risk
Whole sample	60	100	22	36	38	63
No change (flat slope)	18	30	8	36	10	26
Increasing PDR (positive slope)	10	17	3	14	7	18
Decreasing PDR (negative slope)	32	53	11	50	21	55

Footnotes

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors disclosed the reciept of the following financial support for the research, authorship, and/or publication of this article: This research was supported by funding from the Department of Health and Human Services - Administration for Children and Families - Children’s Bureau award HHS-2008-ACF-ACYF-CO-0046 and the University of Maryland School of Social Work Dean’s Research Fund.

References

Aarons

G. A.

James

Monn

A. R.

Raghavan

Wells

R. S.

Leslie

L. K.

(2010). Behavior problems and placement change in a national child welfare sample: A prospective study. Journal of the American Academy of Child & Adolescent Psychiatry, 49, 70–80. Retrieved from https://dx-doi-org.web.bisu.edu.cn/10.1016/j.jaac.2009.09.005

Achenbach

T. M.

Rescorla

L. A.

(2001). Manual for the ASEBA school-age forms & profiles. Burlington, VT: University of Vermont, Research Center for Children, Youth, & Families.

Barth

R. P.

Landsverk

Chamberlain

Reid

J. B.

Rolls

J. A.

Hurlburt

M. S.

… Kohl

P. L.

(2005). Parent-training programs in child welfare services: Planning for a more evidence-based approach to serving biological parents. Research on Social Work Practice, 15, 353–371. Retrieved from http://doi:10.1177/1049731505276321

Barlow

Coren

Stewart-Brown

(2002). Meta-analysis of the effectiveness of parenting programmes in improving maternal psychosocial health. British Journal of General Practice, 52, 223–233.

Buchanan

Chamberlain

Price

J. M.

Sprengelmeyer

(2013). Examining the equivalence of fidelity over two generations of KEEP implementation: A preliminary analysis. Children and Youth Services Review, 35, 188–193. Retrieved from http://doi:10.1016/j.childyouth.2012.10.002

Burke

J. D.

Loeber

Birmaher

(2002). Oppositional defiant disorder and conduct disorder: A review of the past 10 years, part II. Journal of the American Academy of Child & Adolescent Psychiatry, 41, 1275–1293.

Casanueva

Tueller

Smith

Dolan

Ringeisen

(2014). NSCAW II wave 3 tables (OPRE Report #2013-43). Washington, DC: Office of Planning, Research and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services. Retrieved from http://www.acf.hhs.gov/programs/opre/resource/nscaw-ii-wave-3-report-wave-3-tables

Chamberlain

Moreland

Reid

(1992). Enhanced services and stipends for foster parents: Effects on retention rates and outcomes for children. Child Welfare, 5, 387–401.

Chamberlain

Price

Leve

Laurent

Landsverk

Reid

(2008). Prevention of behavior problems for children in foster care: Outcomes and mediation effects. Prevention Science, 9, 17–27. Retrieved from http://doi:10.1007/s11121-007-0080-7

10.

Chamberlain

Price

Reid

Landsverk

Fisher

Stoolmiller

(2006). Who disrupts from placement in foster and kinship care? Child Abuse & Neglect, 30, 409–424. Retrieved from https://dx-doi-org.web.bisu.edu.cn/10.1016/j.chiabu.2005.11.004

11.

Chamberlain

Reid

J. B.

(1987). Parent observation and report of child symptoms. Behavioral Assessment, 9, 97–109.

12.

Cohen

West

S. G.

Aiken

L. S.

(2002). Applied multiple regression/correlation analysis for the behavioral sciences. Mahwah, NJ: Lawrence Erlbaum Associates.

13.

Donaldson

(2008). Patient-reported outcomes and the mandate of measurement. Quality of Life Research, 17, 1303–1313. Retrieved from http://doi:10.1007/s11136-008-9408-4

14.

Dorsey

Farmer

E. M. Z.

Barth

R. P.

Greene

K. M.

Reid

Landsverk

(2008). Current status and evidence base of training for foster and treatment foster parents. Children and Youth Services Review, 30, 1403–1416. Retrieved from http://doi:10.1016/j.childyouth.2008.04.008

15.

Dretzke

Davenport

Frew

Barlow

Stewart-Brown

Bayliss

… Hyde

(2009). The clinical effectiveness of different parenting programmes for children with conduct problems: A systematic review of randomised controlled trials. Child and Adolescent Psychiatry and Mental Health, 3(1). doi:10.1186/1753-2000-3-7

16.

Festinger

Baker

A. J.

(2013). The quality of evaluations of foster parent training: An empirical review. Children and Youth Services Review, 35, 2147–2153. Retrieved from http://doi:10.1016/j.childyouth.2013.10.009

17.

Fisher

A. P.

Stoolmiller

Mannering

A. M.

Takahashi

Chamberlain

(2011). Foster placement disruptions associated with problem behavior: Mitigating a threshold effect. Journal of Consulting and Clinical Psychology, 79, 481–487. Retrieved from http://doi:10.1037/a0024313

18.

Furlong

McGilloway

Bywater

Hutchings

Smith

S. M.

Donnelly

(2013). Cochrane review: Behavioural and cognitive-behavioural group-based parenting programmes for early-onset conduct problems in children aged 3 to 12 years. Evidence-Based Child Health: A Cochrane Review Journal, 8, 318–692. doi:10.1002/ebch.1905

19.

Greeno

J. E.

Uretsky

M. C.

Lee

B. R.

Moore

J. E.

Barth

R. P.

Shaw

T. V

. (2015). Replication of the KEEP foster and kinship parent training program in a population of youth with externalizing behavior problems. Manuscript submitted for publication.

20.

Gross

H. E.

Shaw

D. S.

Burwell

R. A.

Nagin

D. S.

(2009). Transactional processes in child disruptive behavior and maternal depression: A longitudinal study from early childhood to adolescence. Development and Psychopathology, 21, 139. Retrieved from http://doi:10.1017/S0954579409000091

21.

Harden

B. J.

Whittaker

J. V.

(2011). The early home environment and developmental outcomes for young children in the child welfare system. Children & Youth Services Review, 33, 1392–1403.

22.

Huey

S. J.

Jr Polo

A. J.

(2008). Evidence-based psychosocial treatments for ethnic minority youth. Journal of Clinical Child & Adolescent Psychology, 37, 262–301.

23.

Hurlburt

M. S.

Chamberlain

DeGarmo

Zhang

Price

(2010). Advancing prediction of foster placement disruption using brief behavioral screening. Child Abuse & Neglect, 34, 917–926. Retrieved from http://dx.doi:10.1016/j.chiabu.2010.07.003

24.

IBM Corp. (Released 2012). IBM SPSS statistics for windows, version 21.0. Armonk, NY: IBM Corp.

25.

James

Landsverk

Slymen

D. J.

(2004). Placement movement in out-of-home care: Patterns and predictors. Children and Youth Services Review, 26, 185–206. Retrieved from http://dx.doi:10.1016/j.childyouth.2004.01.008

26.

Maas

C. J.

Hox

J. J.

(2005). Sufficient sample sizes for multilevel modeling. Methodology: European Journal of Research Methods for the Behavioral and Social Sciences, 1, 86.

27.

Michelson

Davenport

Dretzke

Barlow

Day

(2013). Do evidence-based interventions work when tested in the “real world?” A systematic review and meta-analysis of parent management training for the treatment of child disruptive behavior. Clinical Child and Family Psychology Review, 16, 18–34. Retrieved from http://doi:10.1007/s10567-013-0128-0

28.

Patterson

G. R.

DeBaryshe

Ramsey

(1990). A developmental perspective on antisocial behavior. American Psychologist, 44, 329–335.

29.

Price

J. M.

Chamberlain

Landsverk

Reid

J. B.

Leve

L. D.

Laurent

(2008). Effects of a foster parent training intervention on placement changes of children in foster care. Child Maltreatment, 13, 64–75.

30.

Price

J. M.

Chamberlain

Landsverk

Reid

(2009). KEEP foster-parent training intervention: Model description and effectiveness. Child & Family Social Work, 14, 233–242.

31.

Price

J. M.

Roesch

Walsh

N. E.

(2012). Effectiveness of the KEEP foster parent intervention during an implementation trial. Children and Youth Services Review, 34, 2487–2494. Retrieved from http://dx.doi:10.1016/j.childyouth.2012.09.010

32.

Singer

J. D.

Willett

J. B.

(2003). Applied longitudinal data analysis: Modeling change and event occurrence. New York, NY: Oxford University Press.

33.

Shek

D. T. L.

C. M. S.

(2011). Longitudinal data analyses using linear mixed models in spss: Concepts, procedures and illustrations. The Scientific World Journal, 11, 42–76. Retrieved from http://doi:10.1100/tsw.2011

34.

Turner

Macdonald

Dennis

J. A.

(2007). Behavioural and cognitive behavioural training interventions for assisting foster carers in the management of difficult behaviour. Cochrane Database of Systematic Reviews, 1. Retrieved from http://onlinelibrary.wiley.com/doi/10.1002/14651858.CD003760.pub3/pdf/standard

	Foster/Kin provider		Child
	$\bar{x}$ /f	(SD)/%	$\bar{x}$ /f	(SD)/%
Age	50	(7.46)	7.5	(2.29)
Gender
Male	1	1%	33	51%
Female	64	99%	32	49%
Ethnicity
Black/African American	33	51%	31	48%
White/Caucasian	21	32%	21	32%
Other	11	17%	11	17%
Missing	—	—	2	3%
Placement type
Relative/kinship provider	23	35%	—	—
Licensed state foster provider	42	65%	—	—
Group attendance	12.5	78%	—	—
Proportion of positive reinforcement	.69	(.08)	—	—
Initial risk of disruption^a
Low	—	—	24	37%
High	—	—	41	63%
CBCL Externalizing Scale score
Clinical	—	—	25	38%
Subclinical	—	—	40	62%

			Initial Risk
	Whole Sample		Low Risk		High Risk
	f	%	f	%	f	%
Whole sample	60	100	22	36	38	63
No change (flat slope)	18	30	8	36	10	26
Increasing PDR (positive slope)	10	17	3	14	7	18
Decreasing PDR (negative slope)	32	53	11	50	21	55