Juvenile Justice Risk Factors and Functional Family Therapy Fidelity on Felony Recidivism

Abstract

Families (n = 5,884) received Functional Family Therapy (FFT) provided as part of court-ordered probation services by 11 community sites throughout Florida. Sites provided home-based FFT to families with male (72%) or female (28%) delinquent youth. Juvenile justice courts referred clients to these services in an effort to redirect them away from incarceration. Clients were Hispanic (18%), Black (41%), and White non-Hispanic (36%), while therapists (female, 79%) were of Hispanic (28%), Black (20%), and White non-Hispanic (50%) ethnic/racial origins. Analyses of clients’ pretreatment recidivism risk and therapist’s caseload of risky clients demonstrated that both individual and treatment site case-mix of client criminal risk levels were associated with higher adjudicated felony recidivism. Furthermore, clinical process indicators suggest that therapists with larger rather than smaller caseloads of high-risk clients provided treatment with greater fidelity. Results suggest that experience in working with challenging clients is critical for achieving fidelity with these cases.

Keywords

fidelity client caseload individual risk family therapy clinical process recidivism

Adolescent delinquency is a significant public health issue with broad implications for individuals, families, and communities (Alexander, Waldron, Robbins, & Neeb, 2013; Drake, Aos, & Miller, 2009; Lipsey, 2009). The costs for detaining youth are staggering. Nationwide estimates for 2005 indicate that the daily detention census in the United States was 93,000 youths, and the daily cost per offender was approximately US$241 (Justice Policy Institute, 2009). The annual costs for detention exceeded US$8.19 billion with US$5.7 billion due to residential placement. The trends for youth involvement in the juvenile justice system, however, appear to be moving in a positive direction. For example, the National Juvenile Court Data Archive indicates that the number of referred youth has declined from 1.8 million to 1.1 million from 1997 to 2013 (Hockenberry & Puzzanchera, 2015). Hockenberry (2016) reported Office of Juvenile Justice and Delinquency Prevention statistics indicating that the number of juveniles in residential settings has declined from a high of 105,000 in 1997 to approximately 54,000 in 2013.

Numerous factors influence treatment with youth in the juvenile justice system, including the expanded use of empirically established, community-based interventions. Functional Family Therapy (FFT; Alexander et al., 2013; Robbins, Alexander, Turner, & Hollimon, 2016) is one evidence-based treatment (EBT) that has been widely disseminated into “real-world” community settings in the past two decades to address adolescent delinquent behavior. Grounded in systemic and cognitive-behavioral theories, the clinical model identifies and addresses dysfunctional family relationship patterns, as well as individual and community risk and protective factors. Treatment aims to increase adaptive behaviors and decrease maladaptive behaviors while accommodating the family’s relational functions. This goal is achieved through intervention techniques with defined therapist goals and skills, using a strong cognitive component that includes systemic skill training in family communication, parenting skills, conflict management, and numerous other skills linked to a variety of syndromes and referral problems. Weekly supervision sessions assist therapists in adapting techniques to the unique problems encountered in each family (Alexander et al., 2013).

Approximately 50,000 families receive home-based FFT services each year (total ~500,000) from 1,600 therapists associated with 350 agencies in 11 countries (Robbins et al., 2016). Evaluations comparing FFT with usual probation services provide an estimated 8.94 benefit-to-cost ratio from the reduction of criminal recidivism (Lee, Aos, & Pennucci, 2015; Taxy, Liberman, Roman, & Downey, 2012). Despite the positive outcomes noted above, not all families benefit from FFT. The present investigation sought to examine this further by testing hypothesized interactions of client risk, site risk, fidelity, and clinical process variables on delinquency outcomes. Because FFT is effective with a very large, highly diverse set of clients and settings, it provides a useful platform to evaluate the potential interactive effects of client risk, therapist fidelity, and case experience on recidivism.

Challenges in Disseminating Evidence-Based Therapies

Numerous barriers exist when transporting EBTs, such as FFT, into community settings (Fixsen, Naoom, Blasé, Friedman, & Wallace, 2005). The process of transporting EBTs is complex and multifaceted, involving the interaction of client and treatment setting characteristics (Henggeler & Schaeffer, 2016; Kazdin, 2008; Schoenwald, Sheidow, & Chapman, 2009). These factors include the type of intervention models (e.g., individual, family, or group), the mix of clients served (e.g., variable client risk and caseload risk associated with reoffending), therapist training and experience, type of community agencies, and the service delivery setting (residential, office-based, or home-based). The results on real-world implementation of EBTs are less powerful than the results observed in controlled evaluations (Schoenwald et al., 2009; Wilson & Hoge, 2013).

Implementation Fidelity and Treatment Outcomes

A fundamental assumption in implementation and dissemination research is that the integrity or fidelity of the intervention is a major determinant of successfully transporting efficacious treatments to community practice settings (Fixsen et al., 2005; Simpson & Flynn, 2007). Fidelity of an intervention reflects both adherence to change principles in the treatment model and competence of service delivery. Researchers have operationalized competence to encompass therapist skill and experience, appropriateness and timing of service delivery, and responsiveness to client behaviors (Hogue et al., 2008). From this perspective, higher model fidelity and therapist competence should produce better outcomes.

Prior research examining family therapy has demonstrated that better outcomes occurred with higher fidelity of service delivery (Robbins et al., 2011). Yet, some family therapy investigations found that fidelity did not directly predict outcomes, but instead the relationship appears to be curvilinear (Hogue et al., 2008) with better outcomes for intermediate fidelity. These investigators suggest that being too lax might result in weak and ineffective interventions, while rigidity (i.e., strict fidelity to the model) may lead to negative outcomes and diminished therapeutic alliance if model guidelines restrict the therapist’s flexibility to address complex problems for some clients. This rigid adherence to the model may be more problematic in community settings when therapists apply EBT guidelines with clients and in settings that differ from those used to develop the model. Regardless of the explanation, these findings indicate that fidelity to outcomes may not be a purely linear one.

Client Risk and Therapist Fidelity

Extensive research with adolescent and adult offenders has demonstrated the importance of client pretreatment risk as a predictor of justice system outcomes (Morgan, Kroner, Mills, Serna, & McDonald, 2013). This research indicates that it is important to match the type and intensity of treatment to the client’s criminogenic risks and needs (Vincent, Guy, & Grisso, 2012). FFT research is consistent with this premise, and it indicates that clinical outcomes depend upon the interactive effects of client risk and therapist integrity in applying the intervention (Sexton & Turner, 2010). Higher fidelity in delivering FFT has a greater positive effect on outcomes for higher compared with lower recidivism risk clients (Turner, Robbins, Rowlands, & Weaver, 2017).

Client Risk, Caseload Risk, Therapist Competence, and Model Adherence

Important factors to consider are the interactions between client risk, caseload risk, and therapist competence, when examining model adherence and outcomes in a community-based implementation. The caseload risk, or extent to which a therapist’s cases comprise riskier clients, may affect clinical adherence and the efficacy of the therapeutic alliance. One perspective is that a high caseload of difficult clients may lead to pessimism and discouragement, resulting in ineffective treatment.

Another more positive view emphasizes the skill development associated with treating challenging cases. Therapists and supervisors with caseloads consisting of a larger mix of risky clients (high-risk caseload) gain more experience in treating these cases. They are more likely to develop skills in working with these families (Sexton & Turner, 2010; Taxy et al., 2012). With experience, therapists become more adept at discriminating among high-risk cases to identify those youth and families who are most at risk of failure. These cases can become a primary focus of case planning and staffing. Increased experience with difficult cases and more supervision may in turn lead to higher rates of fidelity or adherence to the model. Elevated recidivism risk creates increased opportunities for developing skills with the riskiest clients, which in turn may lead therapists to practice with greater fidelity. Hence, higher fidelity may result from, rather than cause, recidivism.

In contrast, therapists who work with a smaller case-mix of risky clients (low-risk caseload) may be relatively successful with most of their low-risk clients, even at low to moderate levels of fidelity, but may have serious difficulties in working with their infrequent high-risk clients. These therapists may fail to recognize the complex risk factors of their clients. They might have less experience or supervisor support in planning interventions to address the needs of their clients. High-risk cases may pull therapists with less experience with these cases “off model” (Sexton & Turner, 2010), which in turn may lead to worse outcomes. Thus, although the overall performance with low-risk caseloads may be good as most clients are at low risk of recidivism, high-risk clients may be at greater risk of failure. An important implication of this reasoning, if substantiated, is that supervisors may need to pay careful attention to the treatment of high-risk clients by therapists with predominately low-risk caseloads. As each felony can cost social institutions US$50,000 with long-term adverse outcomes for victims and perpetrators, enhanced treatment of these juveniles is a major priority (Lee et al., 2015). If the current reasoning is accurate, then high-risk caseloads may be associated with greater therapist competence and adherence, but may also have high recidivism. Low-risk caseloads, on the contrary, may have lower fidelity and lower recidivism. If we ignore client and caseload risk variables, we might inappropriately conclude that intermediate, rather than high therapist adherence, leads to lower recidivism.

Current Study

This secondary data analysis examined hypotheses that client risk and caseload risk interact to influence clinical process indicators (e.g., therapist fidelity, length of service) and felony recidivism rates in a statewide implementation of FFT. The Florida Legislature and Department of Juvenile Justice (FDJJ) spearheaded the statewide Redirection Project to divert youth from incarceration to evidence-based, family-focused community interventions (Baglivio, Jackowski, Greenwald, & Wolff, 2014; Winokur Early, Hand, & Blankenship, 2012).

The research commenced with preliminary analysis designed to test underlying assumptions regarding the association between client risk, as well as caseload risk, and felony recidivism outcomes. Specifically, we sought to confirm prior research (Morgan et al., 2013) documenting the association between client risk and outcomes. We also examined the underlying assumption that the risk and needs assessment instrument used to evaluate clients’ risk to reoffend in Florida, the Positive Achievement Change Tool (PACT), significantly and positively predicts recidivism. The PACT was developed by FDJJ and was largely based upon the Washington State Juvenile Court Assessment, Back On Track! (Barnoski, 1997, 2004). Prior research has documented the predictive validity of the PACT not only with the overall population of youth referred to FDJJ (Baglivio, 2009; Winokur Early, Hand, & Blankenship, 2012) but also across subsamples based on gender, race, and ethnicity (Baglivio & Jackowski, 2013). We sought to confirm these results in the current sample and performed two preliminary analysis to assess whether (a) high-risk clients have higher felony recidivism than low-risk clients, and (b) high-risk caseloads have higher felony recidivism than low-risk caseloads.

Primary Hypotheses

The primary focus of the study was to evaluate the effectiveness of FFT in reducing felony recidivism and the efficiency of case progression while controlling for differences in client risk, caseload risk, therapist experience, and model adherence. We explored the impact of working with a high-risk caseload on fidelity and outcomes. Prior research suggests that the relationship between model adherence and outcomes may be curvilinear, whereby “intermediate adherence, representing a balance between protocol integrity and clinically flexible deviation” results in better outcomes than “high (rigid) adherence or low (lax) adherence” (Hogue et al., 2008, p. 545).

We also explored whether a therapist’s competence in working with risky clients is related to case progression (completing a full treatment dose), and if so, whether this relationship is moderated by a caseload mix of mostly high-risk clients (high-risk caseload). There are conflicting findings in the literature regarding therapist competence. We sought to explore whether the efficacy of FFT in achieving successful outcomes is related to therapist fidelity and caseload mix and whether fidelity is associated with higher rates of treatment completion. As clients were nested within therapists, treatment settings, judicial circuit courts, and geographical regions, we also examined criminal context variables (county incarceration and violent crime rates) to assess their impact on the relationships between client risk, caseload risk, and fidelity.

Within this framework, we tested the following research hypotheses:

Hypothesis 1: Higher client risk and caseload risk are associated (1a) with lower fidelity and (1b) poorer case progress.

Hypothesis 2: (a) When therapists acquire greater experience with high-risk clients, they are more likely to complete a full dose of treatment in high-risk caseload sites, but they are less likely in low-risk caseload sites. (b) Therapists with high-risk caseloads improve their fidelity ratings more rapidly than therapist with low-risk caseloads.

Hypothesis 3: Higher levels of community crime (client risk, caseload risk, county violent crime rates, and county incarceration rates) are positively associated with therapist model adherence (3a) and recidivism rates (3b).

Method

Participants

Participants were families (n = 5,884) who were mandated for treatment by the Florida juvenile courts to FFT treatment to address an offense committed by an adolescent family member. Nearly all of the clients received home-based services, and all were treated by a trained and certified FFT therapist (n = 169) who was associated with one of 11 different community service providers across Florida. The FFT LLC organization provided the training and supervision of therapists in all of these settings.

Gender and Minority Inclusion

Adolescents (male = 72%; female = 28%) and their families participated in this research. They ranged in age from 10.5 to 19.2 years (M = 16.18, SD = 1.38). The diverse sample included non-Hispanic African American or Black (41.1%), non-Hispanic Whites (35.5%), Hispanic Whites (17.8%), or Other racial/ethnic backgrounds (5.8%).

Therapists

Most of the male (23%) and female (77%) therapists were master’s level social workers, mental health counselors, or marriage and family therapists. They were affiliated with one of 11 community service providers that managed referrals, therapist training and supervision, and service delivery. Therapists’ race/ethnicity included African American or Black (20%), Haitian (2%), Hispanic (28%), Other/Unknown (4%), and non-Hispanic White (50%). The therapists acquired extensive case experience with FFT (M = 47, SD = 42, range 3-249) during the study period. Follow-up outcome data were available for clients of 135 therapists.

Design

This secondary data analysis can be viewed as a 2 (Caseload Risk: Low or High) × 4 (Client Risk: Low, Moderate, Moderate-High, and High) factorial design with client gender as a covariate. Clients were referred to one of the 11 community service providers between 2004 and 2013. The implementation at the various sites was staggered over time, permitting an assessment of the effects of case experience that was not fully confounded with calendar time. We categorized clients based on their assessed criminogenic risks and needs, as determined using the standardized and validated FDJJ PACT assessment, which classifies youth into four levels of pretreatment recidivism risk: low, moderate, moderate-high, and high (Winokur Early, Hand, & Blankenship, 2012). We created a dichotomous crime hazard classification (low/high risk) for each of the therapist’s caseloads in the 11 service sites based on the mean scores of all clients on the criminal history subscale of the PACT, adapting the logic of hierarchical linear modeling to address effects of nesting of clients and therapists within settings. Prior studies indicated that male clients had higher recidivism rates than female clients (Baglivio et al., 2014) and we incorporated gender as a covariate.

Data Sources

Data for the current study were made available through the collaboration/cooperation of the FDJJ, the Justice Research Center (JRC), and FFT LLC. Official delinquency indicators were obtained from the FDJJ Juvenile Justice Information System (JJIS), the computerized administrative system that contains records of all juvenile referrals, charges, dates of alleged infractions, detentions, adjudications, dispositions, placement and service history, and risk assessments data. This system conforms to national standards for reporting juvenile justice referrals and dispositions. The JRC used their authorized access to this system to extract all juvenile referral and placement records from both before and after a client’s participation in the program. Unique identifying information was removed from the record and replaced by a coded index so that the JRC could track and link client records over time. The JRC matched JJIS records to official juvenile adult criminal arrest and placement data maintained by the Florida Department of Law Enforcement and the Florida Department of Corrections. These time tagged data enabled us to establish the sequential order of clinical and juvenile justice events, as well as permitted us to evaluate subsequent juvenile and adult recidivism for all study youth.

The PACT risk assessment data for study youth were extracted from the JJIS information system. The PACT is administered to all juveniles referred to FDJJ for a criminal offense or violation of probation. The PACT produces a Criminal History Score and Social History Score upon which risk level classifications are then calculated (low, moderate, moderate-high, and high), and prior research demonstrates that these categories predict the likelihood of juvenile offender recidivism (Baglivio & Jackowski, 2013; Winokur Early, Hand, & Blankenship, 2012). As elaborated upon in more detail below, the PACT was used to operationalize client risk and caseload risk.

The FFT LLC provided access to indicators of FFT clinical processes and therapist fidelity. These measures reflected clinical progress in treatment, as well as therapist adherence to the FFT model and therapist competence. Primary outcomes included measures of felony recidivism involving all youth from the case open date to 1 year after their release from FFT services. Operational definitions for all study measures are outlined below. The findings from the study have the potential to improve the tailoring of interventions to juvenile offenders and to provide better criteria for tracking treatment fidelity.

Study Variables and Procedures

We extracted variables from multiple administrative data sets accessed through the FDJJ or FFT LLC. The FDJJ authorized the JRC staff to see client names and criminal records and to assign a randomly generated case identifier to each data set to link clients across data sets. The researchers received only de-identified data with a random case identifier. The National Institute on Drug Abuse (NIDA) project officer, the FDJJ, and institutional review boards (IRBs) approved study procedures.

Client Risk

All youth referred to FDJJ for a delinquency petition receive a PACT prescreen risk assessment. Probation officers use the PACT in a structured interview with the youths and their family to assess for multiple risk and protective factors. The PACT has extensive psychometric information that it is both a reliable and valid tool to predict the likelihood of further involvement in the juvenile justice system (Baglivio & Jackowski, 2013; Winokur Early, Hand, & Blankenship, 2012). A panel of nationally recognized criminal justice experts developed the original version of the assessment tool (Barnoski, 2004).

The FDJJ has used the PACT to assess recidivism risk for more than 100,000 adolescents in the state of Florida. The Criminal History scale includes 10 items that assess pretreatment criminal behavior. The total score on this scale primarily reflects individual differences in the types of prior offenses (e.g., felony, misdemeanor, weapons, crimes against person), and judicial actions such as confinement orders, or arrest warrants for failure to appear (α = .71). The Social History scale assesses a range of social problems including school behaviors, use of free time, peer relationships, gang involvement, family, mental health problems, prior neglect or abuse, running away from home, and deviant and prosocial attitudes and behaviors (α = .57).

Site Caseload Risk

We used the PACT assessment to classify pretreatment risk factors that may be associated with clinical processes and outcomes. We created an index of the site caseload mix of risky clients using the mean Criminal History Score on the PACT for all clients on the caseload. We dichotomized caseload risk into higher and lower case-mixes of risk depending upon whether they were above or below the mean on this index.

Community Risk

We examined the influence of community-level criminal justice risk using annual Index Crime and incarceration rates for each county in Florida, and we assigned the appropriate regional score to each family. The law enforcement website provided annual Index Crime rates/100,000 residents for each county (Florida Department of Law Enforcement, 2017). The corrections website provided estimates of county incarceration rates (Florida Department of Corrections, 2012). We assigned crime and incarceration rates to each client based upon their county of residence. Information concerning these measures is available in Supplemental Table P1 (Supplemental Material is available in the online version of this article).

Therapist Supervision Procedures and Fidelity Assessment

FFT therapists received standard FFT training and supervision procedures (Alexander et al., 2013). The training included 10 days (five 2-day trainings) of didactic, on-site workshops. Additional training and supervision occurred in 1-hr weekly group consultation calls led by expert FFT LLC consultants over 1 year and ongoing support to an on-site clinical supervisor via telephone calls after the first phase of training. In addition to providing feedback on cases, the consultants (initial phase) and supervisors (after initial phase) rated the therapist’s compliance with administrative procedures (e.g., completing case notes) and the therapist’s fidelity to FFT principles. The supervisor rating scale ranged from 0 to 6, and we dichotomized these ratings into scores above or below the minimally accepted threshold (>3.0).

The supervisor provided these weekly ratings as additional feedback to the therapists, and they recorded these values in the electronic record for each supervision session. We created a sequential order for these weekly ratings starting with the date of the therapist’s first supervision. Using these scores, we calculated a mean fidelity rating for each month that the therapist received supervision. We assigned the mean score to each client who enrolled during the month of the rating. This procedure permitted us to explore the nesting of therapist’s clients. We estimated the internal consistency of supervisor’s ratings (intraclass correlation coefficient [ICC] = .84, α = .94) for each therapist during Months 1 to 3 for each client to reflect the temporal stability of the therapist’s ratings (Turner et al., 2017). Any fidelity ratings analyses only used data after 2008 when the current procedure was adopted.

Dependent Variables

We identified adjudicated recidivism as occurring either during the service episode or during the 1-year period after the service period ended. Recidivism was defined as a subsequent adjudication or adjudication withheld for a law violation, and we classified the offense type as either a felony or any other offense type (misdemeanor, violation of probation due to a new law violation), exclusive of technical violations of probation. The JRC has conducted previous studies (Winokur, Blankenship, & Hand, 2009; Winokur Early, Hand, & Blankenship, 2012) demonstrating that they can link juvenile records in the FDJJ JJIS system to adult arrest and corrections data maintained by the Florida Department of Law Enforecment and Florida Department of Corrections information with 98% matching accuracy. These systems provided a report of date and type of subsequent adjudications, convictions, and any participation in mandated treatment or confinement. The Common Definitions Report (Florida Department of Juvenile Justice, 2011) classifies offenses in terms of severity.

Statistical Considerations

Exploring Nesting Effects

We recognized the possibility of data dependency resulting from the nesting of families within therapists who were nested within sites or community service providers (see Spybrook et al., 2011). Therapists participated in only one of these organizations. We examined these nesting effects and the results are presented in the Supplemental Material (available in the online version of this article).

Analysis Approach

We examined three types of measures: felony recidivism outcomes, clinical indicators reflecting progress in therapy, and therapist fidelity. Due to the large sample sizes, very small effect sizes could be statistically significant without being clinically meaningful. Thus, we primarily focused upon effect size estimates as a basis for making inferences about the effects of therapist level and client level risk factors. We used binary logistic regression procedures for dichotomous dependent variables and factorial analysis of variance for continuous variables.

Stratified Matching Procedure

We adapted the logic of stratified propensity score matching (Guo & Fraser, 2010; Rosenbaum & Rubin, 1983; Turner et al., 2017) to control for the effects of client risk and caseload risk. One matching approach uses propensity scores to create one-to-one matches between cases in two different study samples (e.g., Baglivio et al., 2014). Our matching procedure equates subsets of cases rather than individuals on key variables. We matched based on client risk, caseload risk, therapist case experience, and client gender.

Effect Size Calculations

To facilitate the evaluation of findings, we have reported three different effect size indices based upon Cohen’s (1988) criteria. The first effect size is Cohen’s d = (M₁ – M₂)/SD_pooled which is useful in comparing two groups on a continuous dependent variable (Cohen, 1988; Lipsey, 1990). A second index is η² (eta squared), and it is useful for describing the differences among three or more groups on a continuous measure. The third effect size d_φ is useful for comparing proportions in two groups. The statistic d_φ = (2 × (φ₁ – φ₂)). The index φ = [arcsin(p^1/2)] is computed for dichotomous dependent variables (Lipsey, 1990).

Results

Preliminary Analyses

The preliminary analysis assessed the effects of individual client risk and site caseload mix of risky clients on adjudicated felony recidivism occurring between the case open date and the 1-year period following the release from services (median time = 15 months). We used a 2 (Caseload Risk) × 4 (Client Risk) factorial design with client gender as a covariate; the observed recidivism rates for these combinations are presented in Figure 1. Numerical values and gender details of the analysis are presented in Supplemental Table S1 (available in the online version of this article). The results indicated that both client risk and caseload risk were associated with higher recidivism. As expected, the findings are consistent with prior research linking risk factors to recidivism.

Figure 1:

Percentage of youths with an adjudicated felony recidivism charge between case open date and 1 year following the end of probation supervision (total time ~16 months) by client pretreatment risk category on the PACT and site caseload risk.

Primary Hypotheses

Effects of Client Risk and Caseload Risk on Clinical Process Measures

We reasoned that a therapist’s fidelity and case progress with high-risk clients might depend upon the site’s caseload mix. We recognized that case experience with risky clients could lead to either deteriorating or enhanced clinical performance, and our analyses evaluated these two possibilities. Therapists with a small (low-risk) caseload of very risky clients have little experience in treating these clients and few opportunities for supervision, potentially resulting in low confidence and discouragement. An alternative outcome may occur for supervised therapists with a larger (high-risk) caseload of risky clients. These therapists have more opportunities to gain expertise from treating these clients especially when receiving group supervision and sharing experiences with other therapists.

We examined the hypothesis about therapists either with low- or with high-risk caseloads who are treating clients with higher recidivism risk. We assessed whether increased client risk was associated with lower therapist fidelity to the FFT model and poorer progress in treatment, resulting in the restriction of the treatment dose for clients. To address potential nesting effects, we included the therapist factor as a random effect and site caseload risk and client risk variables as fixed-effect factors in a mixed-effects analysis of variance (ANOVA) with the therapist fidelity score as the dependent variable. The results revealed significant main effects for caseload risk, F(1, 4157) = 9.54, p < .002, client risk, F(3, 4157) = 6.19, p < .001, and the caseload risk by client risk, F(3, 4157) = 3.04, p < .028, interaction.

We conducted pairwise comparisons among the main effect means using a Bonferroni-adjusted confidence level—α/6; p < (.05/6) = .008. The results indicated that the therapists treating the lowest risk clients received significantly higher fidelity scores (M = 3.64, SE = 0.04) than the other three client risk groups (moderate, moderate-high, and high). The comparison of main effect means for the caseload risk variable indicated that therapists working at sites with a high-risk caseload had higher fidelity scores (M = 3.56, SE = 0.03) than those working at sites with a low-risk caseload (M = 3.43, SE = 0.04. See Figure 2 (or Supplemental Table S2, available in the online version of this article).

Figure 2:

The effects of client pretreatment risk and site caseload risk on the mean monthly supervisor fidelity ratings of therapists treating clients in each category.

To evaluate the interaction term, we conducted simple main effect comparisons within each of the caseload risk conditions (low and high). First, we compared the four client risk groups within the high-risk caseload sites, and the results indicated that the four client risk groups were not significantly different from each other. Second, we compared the four client risk groups among the low-risk caseload sites. See Figure 2 (or Supplemental Table S2, available in the online version of this article). These comparisons indicated that the therapists received significantly higher fidelity scores when they were working with their least risky clients than when they worked with their higher risk clients (d_φ = 0.51). The effect size (d_φ = 0.35) among the high-risk caseload sites was of a medium magnitude.

Effects of Risk Factors on Progress During Treatment

The first hypothesis also concerned the therapist progress in achieving interim treatment goals. To assess this element of the hypothesis, we examined the likelihood that clients achieved at least eight sessions, which corresponds to the end of the behavior change phase of FFT, an important interim treatment goal (Alexander et al., 2013). This goal reflects whether therapists are able to engage and retain clients sufficiently to achieve a minimal dose. Hypothesis 1b states that increased client risk and caseload risk are associated with a reduced likelihood of achieving a minimal treatment dose.

We conducted a binary logistic regression analysis using caseload risk, client risk, and gender as independent variables and the dichotomous minimal dose measure as the dependent variable.¹ Within the low-risk caseload sites, the differences in achieving a minimum dose between their lowest risk (85%) and their highest risk (63%) clients resulted in a relative large effect size (d_φ= 0.51; see Figure 3). Within high-risk caseload sites, the difference in the rate of achieving a minimal dose for the lowest individual risk clients (86%) versus the highest individual risk clients (72%) reflected a moderate effect size (d_φ = 0.35; see Figure 3). We also computed an effect size (d_φ = 0.19), which was in the small range, for the difference in rates among the highest risk clients between the low- and high-risk caseload sites. In summary, these results indicated that therapists in low-risk versus the high-risk caseload sites had more difficulty with clinical processes, including achieving a minimal dose treatment with their high-risk clients.

Figure 3:

The effects of client pretreatment risk and site caseload risk on the percentage of clients receiving a minimal dose of therapy (eight sessions).

Effects of Risk Factors and Therapist Experience on Case Completion

The second hypothesis stated that therapists with high-risk caseloads and more case experience working with risky clients are more likely to complete a full dose of treatment (12 or more sessions) than are therapists with a low-risk caseload and less experience with high-risk clients. For this analysis, we determined how many cases each therapist had treated prior to the current case. We categorized the prior case experience into three groups (0-20, 21-40, and 41-60). We also pooled the low- and moderate-risk groups together (LR Client) and the moderate-high and high-risk youth (HR Client) together to create larger samples sizes. The therapists required approximately 6, 12, and 18 months to acquire these sets of cases.

We conducted a binary logistic regression analysis using caseload risk, the three-group case experience measure, client binary risk (low/moderate, moderate-high/high), and gender as independent variables. The dichotomous dependent variable indicated whether the client received a full dose (0 = No, 1 = Yes). The results indicated that the Caseload Risk, p < .007, Exp(B) = 1.40, Client Risk, p < .001, Exp(B) = 0.76, and Therapist Case Experience, p < .001, Exp(B) = 1.15, were statistically significant (see Figure 4 and Supplemental Table S4, available in the online version of this article). The gender effect was not significant. To test Hypothesis 2, we compared the full dose index within the high-risk caseload sites, and the results indicated that the early cases (0-20: 47%) did not differ from the middle cases (21-40: 47%, d_φ = −0.058), but did differ from the late cases (41-60: 65%, d_φ = 0.38). To test the second hypothesis further, we compared the full dose index within the low-risk caseload sites, and the results indicated that the early cases (0-20: 49%) did not differ from middle cases (21-40: 46%,d_φ = −0.07) or from late cases (41-60: 52%, d_φ = 0.06).

Figure 4:

Percentage of clients who received a full scheduled dose of treatment (12 sessions) by therapist’s prior case experience and by sites with either a low-risk caseload (LR Site [gray, above]) or a high-risk caseload (HR Site [black, above]) and the client’s pretreatment risk.

We also examined Caseload Risk × Binary Client Risk interaction among late cases (41-60). The results (see Figure 4) indicated that the Low-Risk Caseload-High-Risk Client group (35%) had significantly lower completion rates than the Low-Risk Caseload-Low-Risk Client group (50%, d_φ = 0.31), the High-Risk Caseload-Low-Risk Client group (60%, d_φ = 0.50), or the High-Risk Caseload-High-Risk Client group (55%, d_φ = 0.41). In summary, the findings are consistent with Hypothesis 2a but not 2b. Thus, the findings for high-risk client indicated that case experience in high-risk caseloads are linked to better clinical outcomes but worse outcomes in low-risk caseload sites.

Exploratory Analysis of the Relationship of Case Experience and Fidelity

We examined whether therapists would improve in model fidelity with experience, especially when accompanied by expert supervision, if their case-mix included a higher proportion of challenging cases. We identified therapists’ fidelity ratings and their case experience within sites that had a larger or smaller caseload of clients with risk-related criminal histories (recall Figure 1). We assessed whether early case experience was associated with changes in their supervisor’s fidelity ratings of the therapist. We combined pairs of cases (e.g., 1-2, 3-4) to produce more stable estimates. All clients began treatment after the FFT LLC adopted the current supervision and ratings procedures in 2008 (Robbins et al., 2016).

We determined the percentage of clients whose therapists had fidelity ratings above the clinical threshold during the second month of their treatment. This time usually corresponds to the behavior change stage of treatment. We defined acceptable fidelity ratings as a supervisor score of 3 or higher. To make these comparisons, we computed an effect size (d_φ) based upon Cohen’s (1988) procedure for comparing proportions. The analyses for early cases indicated that therapists with larger caseloads of challenging cases had fewer acceptable ratings (medium-large effect sizes) than therapists with less challenging cases (1-2:d_φ = 0.23; 3-4: d_φ = 0.52; 5-6: d_φ = 0.27). However, an examination of later cases (i.e., 9-14) produced the opposite pattern. Therapists with a larger caseload mix of risky clients improved more than therapists with a smaller risky case-mix or low-risk caseload so that, over time, they had more ratings (medium-large effect sizes) above the clinical threshold (9-10: d_φ = −0.33; 11-12: d_φ = −0.52; d_φ = −0.37).

After the 15th case, therapists working with low-risk caseloads improved so that a small effect size difference occurred between the case-mix sites, being negligible by the 24th case. We extended these comparisons through the first 80 cases of each therapist, which required approximately 2 years. We pooled the scores into groups of 10 clients (e.g., 1-10). The proportion above acceptable fidelity thresholds improved with case experience (1-10 = 24%; 11-20 = 33%; 21-30 = 42%; 31-40 = 53%; 41-50 = 60%; 51-60 = 67%; 61-70 = 82%; 71-80 = 82%). These findings suggest that most therapists improve over a 2-year period.

We conducted an exploratory analyses to address a possible confound concerning the effects of therapist case experience. We recognized the possibility that the changes within a site caseload risk level might result from changes in the way that clients were assigned to therapists as they gained more experience. Specifically, we anticipated that the sites might withhold their more challenging cases from their newest therapists and assign these cases to their more experienced therapists. Thus, experienced therapists might be assigned to more difficult cases.

To examine this possibility, we conducted an exploratory analysis to compare the case-mix of lower risk (low, moderate) and higher risk (moderate-high, high) clients by the therapist’s case experience in blocks of 10 clients. We summarized results in Figure 6 (and Supplemental Table S6, available in the online version of this article), and the findings indicated that the high-risk caseload sites have a greater percentage of the high-risk clients through each block of cases for their first 50 cases. The effect sizes for the difference in caseload risk ranged from 0.31 to 0.50. Thus, case-mix difference between low-risk and high-risk caseloads persists throughout the therapist’s case experience in their first year.

In summary, sites with a low-risk caseload had lower fidelity scores, especially when treating their high-risk clients. High-risk caseload sites had higher fidelity scores, and these ratings were not significantly different by client risk level. These findings are consistent with the premise that sites containing more of the challenging high-risk cases were associated with greater model compliance as reflected in supervisor ratings. Any differences due to client risk and therapist experience were not due to shifting assignments of risky clients to therapists.

Criminal History Context and Therapist Fidelity

We examined the hypothesis that therapists working in higher crime environments would be more faithful to FFT, which would be reflected in higher supervisor fidelity ratings. We selected four measures to reflect the criminal context for clients. The first was the client’s own Criminal History Score from the PACT, the second was the therapist’s case-mix of risky clients, the third was a measure of the violent crime rate for the county in which the clients received services, and the fourth was the incarceration rate per capita in the client’s county (see Supplemental Table P1). We extracted these last two variables from official Florida criminal records and imputed the county scores to each client in that county. We obtained the county-specific rates for 2010 (the midpoint of the study period).

We regressed these four criminal context scores on fidelity scores after 2008-2009 fiscal year, when a new ratings approach was adopted. The overall model was statistically significant, adjusted R² = .11, F(4, 2943) = 23.02, p < .001. Caseload risk variable, β = 0.14, t(2943) = 7.01, p <.001, county incarceration rate, β = 0.14, t(2943) = 7.01, p < .001, and county violent crime rate, β = 0.14, t(2943) = 7.01, p <.001, were associated with higher fidelity scores. The individual Client Risk score based upon the PACT Criminal History score was not significantly associated with therapist fidelity, β = −0.02, t(2943) = 1.05,p < .294. In summary, these findings are consistent with the premise that a higher criminal context is associated with greater adherence to the clinical intervention.

Effects of Risk Factors and Clinical Process Measures on Felony Recidivism

The last hypotheses extend the previous ones to examine the joint predictive effects of four criminal context measures, client gender, and two clinical process measures on the dichotomous Total Felony Recidivism Index. This recidivism measure indicated whether any adjudicated felony offenses occurred between the case open date until the end of court supervision (0 = None, 1 = Any). We estimated separate binary logistic regression models within either the low- or high-risk caseload sites. Three other criminal context variables included the Client Risk (PACT) categories, the client’s county incarceration rate, and the county violent crime rate. The clinical process measures were therapists’ case experience and supervisor’s fidelity rating.

The results indicated that recidivism was elevated for clients with higher PACT Criminal History scores both in the low-risk (p < .001) and the high-risk (p < .005) caseload sites. Males had higher recidivism rates for both the low-risk (p < .001) and the high-risk (p < .001) caseload sites. Higher therapist fidelity ratings (p < .028) and more case experience (p < .030) were associated with lower recidivism in the low-risk caseload sites.² These variables did not predict recidivism for the high-risk caseload sites (ps > .55). Neither the county violent crime rate nor the incarceration rate was statistically significant for site caseload risk (ps > .10).

A summary of findings for low-risk caseload sites indicates that higher client risk and lower fidelity were associated with worse recidivism. Low fidelity was also associated with higher client risk and poorer progress in treatment. The fidelity gap between lower and higher risk clients widens as therapists gain case experience. High-risk clients have less chance for favorable clinical and recidivism outcomes as clinicians gain experience in low caseload sites.

Discussion

This secondary data analysis study examined hypotheses concerning the effects of individual client and community risk as well as therapist’s caseload risk and experience on clinical processes, therapist competence, and treatment outcomes of a large-scale, statewide implementation of FFT. The present findings demonstrate the importance of having a complex, multifaceted framework for understanding how quality of treatment interacts with therapist and youth/family factors to facilitate successful outcomes. For example, the results of the preliminary analyses indicated that both therapist caseload mix of risky clients and individual client risk levels were associated with higher recidivism (see Figure 1). These effects were also associated with differences in therapist model fidelity (Hypotheses 1a). Therapists with a smaller case-mix of risky clients (low-risk caseload) had lower fidelity ratings than those working with a larger number of risky clients on their caseload (Figure 2). Therapists with a low-risk caseload had worse fidelity ratings for their highest versus their lowest risk clients. Fidelity was not differentially associated with client risk when therapists had a larger caseload mix of risky clients. A key pair of findings was that therapists treating higher risk clients had both better fidelity and more recidivism. These findings are inconsistent with the premise that higher fidelity necessarily leads to better outcomes (Fixsen et al., 2005; Simpson & Flynn, 2007).

The present findings address two alternative hypotheses concerning the relationship of fidelity and outcomes. A first explanation is that a curvilinear relationship exists between fidelity and outcomes. That is, very high or low fidelity is associated with worse outcomes. A possible explanation for curvilinear effects is that poor fidelity leads to ineffective interventions, while very high fidelity may lead to overly rigid implementation of models. Both processes may contribute to higher recidivism (Hogue et al., 2008). This explanation posits that intermediate fidelity may be optimal because therapists are flexible in applying their model.

A second explanation is that high fidelity is the result and not the cause of high recidivism risk. The findings suggest that therapist’s caseload mix, case experience, and youth/family risk may interact with treatment quality to affect recidivism. For example, there were significant differences in patterns of progress in treatment by caseload mix. When therapists had a smaller case-mix of high-risk clients, they were less likely to successfully engage and retain clients (especially their highest risk clients). They did not make sufficient progress to achieve a minimal dose (Figure 3) or a full dose of treatment with their higher risk clients.

In contrast, therapists with a larger caseload mix of risky clients had better fidelity and treatment progress with their highest risk clients than therapists with fewer risky clients (Figures 2 and 3). Moreover, as shown in Figure 4, therapists with a high-risk caseload also got better at completing treatment with their high-risk clients as they gained more experience, as evidenced by the percentage of families who received 12 or more sessions. Furthermore, supervisors provided more favorable fidelity ratings to therapists with a larger caseload mix of risky clients (Figure 5). We recognized that therapists’ case experience and caseload mix might be confounded so that therapists with greater experience may receive cases that are more challenging. As presented in Figure 6 therapist caseload mix did not change with experience.

Figure 5:

Percentage of therapists’ cases that were above acceptable fidelity threshold by case experience at the sites with a low-risk caseload or high-risk caseload.

Figure 6:

Percentage of clients with pretreatment moderate-high or high PACT risk categories by site caseload (low risk or high risk) and the sequential order of the therapist’s cases.

Does Higher Risk Promote Higher Fidelity?

The present findings provide evidence that in certain contexts higher fidelity is associated with worse juvenile justice outcomes. We hypothesized that high-risk families lead therapists to adhere more closely to model guidelines in an attempt to mitigate the risk factors associated with their clients. From this perspective, high fidelity is a result rather than a cause of high recidivism. Therapist’s experience with a larger mix of risky clients and neighborhoods may heighten therapists’ awareness to issues across all levels of risk, including being able to identify those cases that are at highest risk of having a negative outcome. This sensitivity may be particularly likely for home-based treatments where therapists see first-hand the family and neighborhood risks that are often associated with negative client outcomes. Experience and ongoing assessment in these settings may increase the therapists’, supervisors’, and team’s ability to identify the highest risk cases and to devote time and resources to plan and deliver interventions that match the level of risk. Thus, the high perceived risk for recidivism may increase model fidelity to mitigate the negative effect of these risk factors. High fidelity may result from high risk of recidivism rather than the reverse.

As noted, the current findings provide support for the perspective that higher community and client risk levels are associated with higher therapist fidelity of implementation. Regression analyses addressing Hypothesis 3 indicated that higher community risk of incarceration, county violent crime rates, and caseload risk were associated with higher fidelity. These results are consistent with extensive research (Risk, Need, Reactivity or R-N-R) demonstrating the importance of client risk as a predictor of criminal justice outcomes (Morgan et al., 2013). This research also indicates that it is important to match the type or intensity of treatment to the client’s risk of recidivism as well as their potential responsivity to treatment.

Clinical process indices for the sites with a larger case-mix of risky clients (high-risk caseload) indicated that they were more likely to retain clients in treatment and to provide a full dose of treatment for their highest risk clients. These clinical process indicators were consistent with supervisor ratings of the therapist’s fidelity. The enhanced model fidelity did not completely offset the negative influence emanating from community and client risk factors.

Limitations of the Present Study

The present analysis was restricted to a single clinical model in the evaluation of the interactive effects of client recidivism risk, therapist caseload risk, and case experience on felony recidivism. Within the FFT model, higher risk clients had better fidelity and worse criminal outcomes. This association may suggest that high fidelity contributes to poor outcomes. However, previous research concerning FFT indicates that the largest effect size differences between FFT and usual care occurred with high-risk clients who received high fidelity treatment (Sexton & Turner, 2010; Turner et al., 2017). In other words, the negative effects of risk and fidelity were stronger in usual care models as compared with FFT. Without random assignment, it is not possible to rule out the possibility that some unmeasured confounding variable was responsible for the outcomes assessed. This analysis relied heavily on administrative data sets. While we used a variety of quality control procedures, we could not match approximately 15% of the clinical process measures to the juvenile justice records.

Clinical Implications

The present findings have important implications for clinical supervision. Therapists with smaller caseloads of risky clients may need additional training and supervision to work effectively with higher risk clients. Although they have fewer of these clients in their practice, they appear to have greater difficulty in engaging them in a full course of treatment. Therapists with a larger caseload of risky clients have more urgency to consult their supervisor and greater opportunities to implement lessons learned with these clients.

These findings are immediately relevant to family-based approaches that are being widely implemented in community settings. The reality of real-world practice is that therapists have limited time to devote to supervision and planning. The current results provide guidance to supervisors about how to be strategic in selecting which cases to discuss in supervision. For FFT LLC, an organization that oversees the training and implementation of FFT to more than 1,600 therapists in 350 settings annually (Robbins et al., 2016), the current results suggest how outcomes can be improved by using client and community risk to drive consultation.

While the present findings reflect relatively small effect sizes, the costs associated with each felony recidivism failure are substantial (US$40,000-US$50,000; Taxy et al., 2012; Winokur Early, Hand, Blankenship, & Chapman, 2012). The cumulative savings associated with each avoided offense provide a potential pool of resources to improve clinical services. It is possible that the consultation process can be fine-tuned to include ongoing feedback about client risk and caseload risk to accelerate therapist development and improve fidelity for the highest risk cases.

Supplemental Material

CJB813184_Supplemental_Material – Supplemental material for Juvenile Justice Risk Factors and Functional Family Therapy Fidelity on Felony Recidivism

Supplemental material, CJB813184_Supplemental_Material for Juvenile Justice Risk Factors and Functional Family Therapy Fidelity on Felony Recidivism by Charles W. Turner, Michael S. Robbins, Kristin Winokur Early, Julia L. Blankenship and Lisa R. Weaver in Criminal Justice and Behavior

Footnotes

Authors’ Note:

This research is supported in part by National Institute on Drug Abuse (NIDA) Grant R01 DA036622-01A1 to Charles W. Turner, Principal Investigator. The authors are indebted to the Florida Department of Juvenile Justice, Helen Midouhas, and Kim Mason from the Functional Family Therapy (FFT) LLC, and Greg Hand from the Justice Research Center, who assisted in obtaining data files for this research. Evidence Based Associates provided administrative support for the Florida Redirection Project. We are also indebted to staff at the Florida Department of Juvenile Justice for their support and collaboration on this project. The opinions presented in this article are those of the authors and not necessarily the views of the Florida Department of Juvenile Justice. Michael Robbins receives compensation from Functional Family Therapy LLC, an organization dedicated to disseminating FFT into community agencies.

Notes

Supplemental Material

Supplemental Material is available in the online version of this article at .

Charles W. Turner, PhD, received his doctorate in Social Psychology from the University of Wisconsin–Madison, he was a faculty member at the University of Utah Psychology Department for 35 years, and he is now a senior scientist at the Oregon Research Institute. He has participated in 20 federally funded projects as a lead methodologist, and his research has focused primarily upon the evaluation of prevention and treatment outcomes for children and adolescents with behavior problems. He has been involved for 40 years in process and outcome research on Functional Family Therapy.

Michael S. Robbins, PhD, received his degree in clinical psychology from the University of Utah and he is now a senior scientist at the Oregon Research Institute as well as being the clinical and research director of FFT LLC. He has extensive experience conducting clinical research on family therapy with drug-using delinquent adolescents and their families, and he has directly overseen the training of hundreds of family therapists, both nationally and internationally. He is recognized as a leader in the areas of process and outcome research in adolescent drug abuse treatment, and he has led numerous federally funded research studies that involve the examination of in-session processes as well as clinical outcomes.

Kristin Winokur Early, PhD, received her doctorate from the College of Criminology and Criminal Justice at Florida State University, and she is now a professor of criminology in the graduate program at Purdue University Global and the research director of the Justice Research Center in Tallahassee, Florida. Her research focuses on evaluating the effectiveness of juvenile justice interventions on youth and family outcomes. She was appointed by Governor Rick Scott to serve on Florida’s Juvenile Justice and Delinquency Prevention (JJDP) State Advisory Group (SAG), serving on the Executive Committee and as the Chair of the Grants and Contracts Committee.

Julia L. Blankenship, MSW, received her master in social work from Florida State University and she is now the president of the Justice Research Center in Tallahassee, Florida. She is an expert in the analyses of large administrative data sets, and her current research focuses on assessing outcomes of at-risk youth and youth involved with the juvenile justice system. She has been the Principal Investigator and coinvestigator on numerous studies, resulting in reports and articles on program, process, and outcome research.

Lisa R. Weaver, BA, received her bachelor’s degree from Rutgers University in political science and she is currently a data analyst at the Oregon Research Institute (ORI). She has participated as a research analyst and data manager on large, complex data sets associated with several National Institutes of Health (NIH) grants at ORI that examine treatments for families and youth with serious problem behaviors. She also has extensive experience in working with children and youth with special needs.

References

Alexander

J. F.

Waldron

H. B.

Robbins

M. S.

Neeb

A. A.

(2013). Functional family therapy for adolescent behavior problems. Washington, DC: American Psychological Association.

Baglivio

M. T.

(2009). The assessment of risk to recidivate among a juvenile offending population. Journal of Criminal Justice, 37, 596-607.

Baglivio

M. T.

Jackowski

(2013). Examining the validity of a juvenile offending risk assessment instrument across gender and race/ethnicity. Youth Violence and Juvenile Justice, 11, 26-43. doi:10.1177/1541204012440107

Baglivio

M. T.

Jackowski

Greenwald

M. A.

Wolff

K. T.

(2014). Comparison of multisystemic therapy and functional family therapy effectiveness: A multiyear statewide propensity score matching analysis of juvenile offenders. Criminal Justice and Behavior, 41, 1033-1056. doi:10.1177/0093854814543272

Barnoski

(1997). Standards for improving research effectiveness in adult and juvenile justice. Olympia: Washington State Institute for Public Policy.

Barnoski

(2004, March) Assessing risk for re-offense: Validating the Washington State Juvenile Court Assessment. Retrieved from http://www.wsipp.wa.gov/ReportFile/868/Wsipp

Cohen

(1988). Statistical power analysis for the behavioral sciences. Hillsdale, NJ: Lawrence Erlbaum.

Drake

E. K.

Aos

Miller

M. G.

(2009). Evidence-based public policy options to reduce crime and criminal justice costs: Implications in Washington state. Victims & Offenders, 4, 170-196. doi:10.1080/15564880802612615

Fixsen

D. L.

Naoom

S. F.

Blasé

K. A.

Friedman

R. M.

Wallace

(2005). Implementation research: A synthesis of the literature (Report No. 231). Tampa: Louis de la Parte Florida Mental Health Institute, University of South Florida.

10.

Florida Department of Corrections. (2012, January 10). Florida county detention facilities’ average inmate population. Retrieved from http://www.dc.state.fl.us/pub/jails/

11.

Florida Department of Juvenile Justice. (2011, July 7). Common definitions report: Process for computing outputs and outcomes for the annual comprehensive accountability report. Retrieved from http://www.djj.state.fl.us/research/reports/reports-and-data/static-research-reports/comprehensive-accountability-report/common-definitions

12.

Florida Department of Law Enforcement. (2017, January 4). Annual uniform crime reports: County and municipal offense data. Retrieved from http://www.fdle.state.fl.us/FSAC/Crime-Trends/Total-Index-Crimes

13.

Guo

Fraser

M. W.

(2010). Propensity score analysis: Statistical methods and applications. Thousand Oaks, CA: SAGE.

14.

Henggeler

S. W.

Schaeffer

C. M.

(2016). Multisystemic therapy®: Clinical overview, outcomes, and implementation research. Family Process, 55, 514-528. doi:10.1111/famp.12232

15.

Hockenberry

(2016). Juveniles in residential placement, 2013 (Report No. NCJ 249507). Retrieved from https://www.ojjdp.gov/pubs/249507.pdf

16.

Hockenberry

Puzzanchera

(2015). Juvenile court statistics 2013. Retrieved from https://www.ojjdp.gov/ojstatbb/njcda/pdf/jcs2013.pdf

17.

Hogue

Henderson

C. E.

Dauber

Barajas

P. C.

Fried

Liddle

H. A.

(2008). Treatment adherence, competence, and outcome in individual and family therapy for adolescent behavior problems. Journal of Consulting and Clinical Psychology, 76, 544-555. doi:10.1037/0022-006X.76.4.544

18.

Justice Policy Institute. (2009). The costs of confinement: Why good juvenile justice policies make good fiscal sense. Washington, DC: Author.

19.

Kazdin

A. E.

(2008). Evidence-based treatment and practice: New opportunities to bridge clinical research and practice, enhance the knowledge base, and improve patient care. American Psychologist, 63, 146-159. doi:10.1037/0003-066X.63.3.146

20.

Lee

Aos

Pennucci

(2015). What works and what does not? Benefit-cost findings from WSIPP (Report No. 15-02-4101). Olympia: Washington State Institute for Public Policy.

21.

Lipsey

M. W.

(1990). Design sensitivity: Statistical power for experimental research. Newbury Park, CA: SAGE.

22.

Lipsey

M. W.

(2009). The primary factors that characterize effective interventions with juvenile offenders: A meta-analytic overview. Victims & Offenders, 4, 124-147.

23.

Morgan

R. D.

Kroner

D. G.

Mills

J. F.

Serna

McDonald

(2013). Dynamic risk assessment: A validation study. Journal of Criminal Justice, 41, 115-124. doi:10.1016/j.jcrimjus.2012.11.004

24.

Robbins

M. S.

Alexander

J. F.

Turner

C. W.

Hollimon

(2016). Evolution of functional family therapy as an evidence-based practice for adolescents with disruptive behavior problems. Family Process, 55, 543-557. doi:10.1111/famp.12230

25.

Robbins

M. S.

Feaster

D. J.

Horigian

V. E.

Puccinelli

Henderson

C. E.

Szapocznik

(2011). Therapist adherence in brief strategic family therapy for adolescent drug abusers. Journal of Consulting and Clinical Psychology, 79, 43-53.

26.

Rosenbaum

P. R.

Rubin

D. B.

(1983). The central role of the propensity score in observational studies for causal effects. Biometrika, 70, 41-55.

27.

Schoenwald

S. K.

Sheidow

A. J.

Chapman

J. E.

(2009). Clinical supervision in treatment transport: Effects on adherence and outcomes. Journal of Consulting and Clinical Psychology, 77, 410-421. doi:10.1037/a0013788

28.

Sexton

Turner

C. W.

(2010). The effectiveness of functional family therapy for youth with behavioral problems in a community practice setting. Journal of Family Psychology, 24, 339-348. doi:10.1037/a0019406

29.

Simpson

D. D.

Flynn

P. M.

(2007). Moving innovations into treatment: A stage-based approach to program change. Journal of Substance Abuse Treatment, 33, 111-120. doi:10.1016/j.jsat.2006.12.023

30.

Spybrook

Bloom

Congdon

Hill

Martinez

Raudenbush

S. W.

(2011). Optimal design plus empirical evidence: Documentation for the “optimal design” software (Version 3.0). Retrieved from http://wtgrantfoundation.org/resource/optimal-design-with-empirical-information-od

31.

Taxy

Liberman

Roman

J. K.

Downey

P. M.

(2012). The costs and benefits of functional family therapy for Washington, DC. Washington, DC: Urban Institute. Retrieved from http://www.urban.org/research/publication/costs-and-benefits-functional-family-therapy-washington-dc

32.

Turner

C. W.

Robbins

M. S.

Rowlands

Weaver

L. R.

(2017). Summary of comparison between FFT-CW® and usual care sample from administration for children’s services. Child Abuse & Neglect, 69, 85-95. doi:10.1016/j.chiabu.2017.04.005

33.

Vincent

G. M.

Guy

L. S.

Grisso

(2012). Risk assessment in juvenile justice: A guidebook for implementation: Models for change. Chicago, IL: MacArthur Foundation.

34.

Wilson

H. A.

Hoge

R. D.

(2013). The effect of youth diversion programs on recidivism: A meta-analytic review. Criminal Justice and Behavior, 40, 497-518. doi:10.1177/0093854812451089

35.

Winokur

Blankenship

Hand

. (2009). The 2009 Florida Department of Juvenile Justice Program Accountability Measures (PAM) report. Tallahassee, FL: Justice Research Center. Retrieved from http://www.thejrc.com/publications.asp

36.

Winokur Early

Hand

Blankenship

. (2012). Validity and reliability of the Florida Positive Achievement Change Tool (PACT) Risk and Needs Assessment instrument: A three-phase evaluation (validation study, factor analysis, inter-rater reliability). Tallahassee, FL: Justice Research Center. Retrieved from http://www.djj.state.fl.us/docs/probation-policy-memos/jrc-comprehensive-pact-validity-and-reliability-study-report-2012.pdf

37.

Winokur Early

Hand

Blankenship

Chapman

. (2012). Redirection continues to save money and reduce recidivism. Tallahassee, FL: Justice Research Center. Retrieved from http://www.evidencebasedassociates.com/wp-content/uploads/2016/09/JRCEval_12.pdf

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.37 MB