Abstract
Purpose:
This overview of reviews analyses the existing evidence base of functional family therapy (FFT), which is a manualized, family-based intervention for youth with behavioral problems and their families. FFT has been implemented among youth aged 10–18 at risk of, or presenting with, behavioral problems such as delinquency, violence, substance abuse, sexual perpetration, and truancy.
Method:
A multipronged search was conducted across 15 databases, 10 websites, and expert contacts in February 2018.
Results:
The search yielded 159 hits of which 31 were included and critically appraised. Included reviews were published between 1986 and 2018 and the number of included studies ranged from 1 to 18 (including 20–5,344 participants). Main effects of the intervention on core outcomes (recidivism and substance abuse) were modest and out-of-home placement was not reported. Secondary outcomes were also modest but generally positive.
Conclusions:
Findings demonstrate that overall quality of reviews is low, which makes any certainties about FFT inconclusive; this overview provides a concise, valid, and methodologically sound synthesis of the research into FFT, which requires more rigorous investigation.
Functional family therapy (FFT) is a manualized family-based intervention program for youth with behavioral problems and their families. Stemming from a systems approach to family therapy, FFT is a short-term intervention, consisting of approximately 30 hr of treatment. The intervention is designed to address family dysfunction by recognizing and modifying maladaptive family communication patterns, training family members to negotiate effectively, set clear rules and boundaries about privileges and responsibilities, and finally to generalize changes to community contexts and relationships. FFT has been used in the treatment of behavior-disordered youth and families for over four decades, but there are questions concerning the reliability and consistency of its evidence base. An overview of reviews or “umbrella review” gathers together the current research in one comprehensive and systematic document (Becker & Oxman, 2008; Higgins & Green, 2011). Unlike a systematic review, overviews are not necessarily limited in scope or focus, as they are not restrained to the primary study as the unit of analysis but are based on the systematic review; in this way, an overview is able to provide a broader type of information regarding a specific intervention addressing different outcomes and applications. An overview is an excellent means of addressing the following concerns: (1) use of the same intervention in the treatment of different problems or populations, (2) differing outcomes from the same intervention within the same issue or population, and finally (3) to determine whether there are any adverse effects resulting from the same intervention (Ballard & Montgomery, 2017). It is thus possible to shed light on the strengths and limitations of current research and reporting and allows for practitioners to easily and comprehensively make decisions based on the best evidence available. This overview addresses all three of these objectives regarding FFT and in particular to assess for risk of harm (Pieper, Antoine, Neugebauer, & Eikermann, 2014; Pieper, Beuchter, Jerinic, & Eikermann, 2012).
Description of Behavioral Problems in Youth Aged 10–18
Behavioral problems refer to a wide array of psychiatric disorders and psychosocial problems, both internalizing and externalizing, including oppositional defiant disorder, attention deficit hyperactivity disorder, and substance abuse disorders. About one third of children with childhood behavior problems develop conduct disorder (CD; American Psychiatric Association, 2013). This group of disorders affects not only the youth and family but extends to have a negative impact on communities and society at large. In 2010, the Center for Disease Control and Prevention the United States reported 3.5% of children aged 3–17 were diagnosed with behavioral problems and 4.7% with illicit drug use disorder in the past year (Costello, Erkanli, & Angold, 2006). For the individual, these disorders are associated with an increase in both internalizing and externalizing behaviors and symptoms. Internalizing problems are associated with a lack of control or ability to recognize and cope with emotions, social isolation or withdrawal, and liability (McCulloch, Wiggins, Joshi, & Sachdev, 2000). Externalizing behaviors are likely to bring the attention of authority or parental figures, as they are overt in nature and may be hostile, aggressive, or destructive. Several studies have found that there is comorbidity between internalizing and externalizing behavior problems (Bornstein, Hahn, & Haynes, 2010; McWey, Cui, & Pazdera, 2010) which may lead to increased chance of self-harm or suicide. In addition to these symptoms, there is likely a higher rate of failure or underachievement in the academic arena and unemployment later in life (Petterutti, Walsh, & Valesquez, 2009).
Within the family setting, attention and care may be focused on the identified client to the detriment of siblings or the family as a whole. The balance within the family can be disturbed, possibly leading to divorce, abandonment, higher rates of alcohol and substance use, untreated mental health issues, and other maladaptive behaviors. Abuse, neglect, interpersonal violence, and sexual misconduct are possible results of family dysfunction, which is highlighted by the fact that in the United States, a person under the age of 18 commits 1 in the 12 domestic violence offenses that come to the attention of law enforcement (http://www.ojjdp.gov/ojstatbb; Prinz, 1988. Increased violence and delinquency contribute to growing unemployment rates and illegal activities such as substance use, underage alcohol consumption, and the sale of narcotics (Petterutti et al., 2009). These activities may lead to imprisonment that will in turn affect a youth’s ability to contribute to the long-term economic productivity of the community and the nation as a whole. The United States spends approximately 5.7 billion dollars per year on the incarceration of minors (http://www.ojjdp.gov/ojstatbb).
Existing Responses to the Problem
A number of family-based interventions have been developed to prevent and/or treat behavioral problems among children and youth. Multisystemic therapy has gained favor in the court system as a mandated treatment for offending or high-risk youth and their families (Henggeler, Schoenwald, Borduin, Rowland, & Cunningham, 2009; Carr, 2000), as have the cognitive–behavioral approaches of aggression replacement training (Goldstein, Barry, & John, 1998), systemic family therapy, and moral reconation therapy (Little & Robinson, 1988); among these, FFT is one of the oldest and best known. Since its development in 1969, FFT has grown to over 270 active sites; while the majority of these are located within the United States, FFT has also been implemented in Belgium, the Netherlands, Sweden, Norway, The United Kingdom, Ireland, and New Zealand (Alexander, Waldron, Robbins, & Neeb, 2013). It was chosen for this overview because of its longevity and increasing uptake.
Description of the Intervention and Theory of Change
FFT reflects a core set of theoretical principles in which behavior is seen as a representation of the family relational system, that is, as indicative of the communication, patterns, and purposes of the family. Its developers describe the overarching goals of FFT as follows: to change the maladaptive behaviors of youth and families, especially those who at the outset may not be motivated or may not believe they can change; to reduce the personal, societal, and economic consequences of disruptive behavior disorders and to do so with less cost, in terms of time and money, than many other treatments currently available (Alexander, Pugh, Parsons, & Sexton, 2002). FFT uses a manualized approach and includes reframing, interrupting of negativity or blame, redirection of focus, interpretations of patterns of maladaptive behavior with links to emotions, a deepening understanding of actions, and communication training with focus on positive communication. It incorporates theories of information processing, social cognition, and the psychology of emotion.
FFT is a short-term (90-day), intensive, and comprehensive program designed for the treatment of behavioral misconduct in youth aged 10–18. It is a treatment modality rooted in family systems theory and cognitive–behavioral theory that can be delivered in clinical settings, school settings, or in the home. It is intended to address youth and families with a wide array of presenting problems including but not limited to criminal behavior, truancy, running away, sexual misconduct, substance abuse, risk of out-of-home placement, and as a reentry program for youth returning to the community following release from institutional settings. The program requires between 8 and 30 hr of direct service to youth and their families typically over an average of 12 home visits in 90 days. Over more than three decades, its designers have modified the original concept in efforts to increase effectiveness and uptake by service providers. According to the literature presented by FFT Inc., these changes have been made based on a “solid foundation of evidence” (Alexander et al., 2013).
Therapists are expected to adjust FFT to family members’ capacities and to the specific problems they face. The focus of an FFT therapist should not be on one specific member of the family rather the therapists attempt to build an equal and balanced alliance with each member of the group. This technique serves to minimize blame and fosters an environment in which each family member feels understood and vital to the process.
Overview and Essential Implementation Elements of FFT
Core Components
The intervention consists of five major components and may include pretreatment activities: engagement in change, motivation to change, relational and interpersonal assessment, behavior change and generalization across behavioral domains, and multiple systems. These goals are accomplished in five ordered phases, the implementation of which is dependent upon the earlier phase; each has an assessment and intervention component directed at specific goals. The founders and advocates of FFT attribute its effectiveness to the careful sequencing of techniques, helped by the continuous assessment and intervention processes (Alexander et al., 2013). Details of the five implementation phases are as follows:
Phase 1: Engagement, wherein workers focus on forming a productive and balanced relationship with the entire family, which involves “maximizing factors which enhance the perception that positive change might occur (intervention credibility), and minimizing factors (e.g., poor program image, difficult location, insensitive referral) that might signify insensitivity and/or inappropriate resources” (Alexander et al., 2013, p. 11). This phase also requires a high level of engagement from the therapist including availability, contact with as many family members as possible, cultural sensitivity, and “matching” to the family unit. Therapists must maintain strength-based relational focus in this as well as all subsequent phases.
Phase 2: Motivation attempts to minimize hopelessness and to create a positive context. To develop or enhance family members motivation, workers identify and quickly begin to modify the pattern of changeable intrafamily risk factors, especially negativity, hopelessness, and blaming; “and initiate and/or strengthen intra-familial protective factors that can mitigate the effect of risk factors that cannot be changed” (Alexander et al., 2013, p. 12). Therapists focus on the relationships and interrupt blaming sequences while encouraging positive themes and reframing negative behaviors.
Phase 3: Relational Assessments are performed in order to understand and analyze information relating to the relational process of the family, as well as to plan for the following behavior change and generalization phases. Therapists continue to redirect the focus from individual problems to a relational perspective. This will aid in understanding negative or destructive behaviors in a new, more positive manner.
Phase 4: Behavior change is used to “develop long term behavior change patterns that are culturally appropriate, context sensitive, and individualized to the unique characteristics of each family member” (Alexander et al., 2013, p. 13). Workers focus on cognitive, interactive, and emotional issues; emphasize positive communication and parenting skills; and provide concrete resources that “guide and symbolize specific changes in behavior” (Alexander et al., 2013, p. 13). This phase aims to reduce intrafamilial risk factors and enhance protective factors. In this, the most directive phase of treatment, therapists may assign tasks, use technical aids, teaching, role-playing, training, conflict resolution skills, or other means for developing necessary skills.
Phase 5: Generalization is the final goal of treatment and focuses on the broadening of behavior change to other settings and social systems. This involves referrals, mobilizing community support systems, and modifying deteriorated family–system relationships with schools, probation officers, or other systems. “Generalization activities involve knowing the community, developing and maintaining contacts, initiating clinical linkages, creating relapse prevention plans and helping the family develop independence” (Alexander et al., 2013, p. 15).
After the conclusion of treatment, families are sometimes seen in “booster sessions”; these can be requested by the family or as a refresher if there is a relapse of maladaptive behaviors. A relapse may be signified by objective outcomes such as arrest or truancy or by family reporting a desire for assistance. These sessions are designed to reinforce the skills learned in the behavior change phase of treatment and to remind the family of their ability to achieve a higher level of functioning.
Underpinning all five phases is the concept of “obtainable change”; this refers to the setting of realistic and achievable goals. These goals are not merely those set by the therapist, authorities, probation officers, or other treatment providers but agreed upon and discussed within the scope of treatment. The youth and family are integral to the decision of which goals to pursue in treatment.
Assessment focuses on the functional nature of problems within the family rather than on a diagnosis and is described as a “continuous, multilevel, multidimensional, and multimethod process that includes individual, family, behavioral, and contextual factors” (Alexander et al., 2013, p. 22). The relational assessment examines not only the connectedness or relative autonomy of family members but also the hierarchical standing of each member in relation to the others. It focuses on why or how the family continues in its promotion and maintenance of problematic sequences, chains of behavior, events, and interactions; assessment will also identify key risk and protective factors for the family.
How the Intervention May Work
The model is suggested to be useful for complex and multidimensional problems because of its flexible structure and cultural sensitivity (Alexander et al., 2013). The attention to relational patterns and clear focus on family systems is thought to be at the core of the intervention’s success. As many FFT therapists provide services in the home, it is particularly suited to those clients who may be initially unwilling, unable, or otherwise unlikely to attend sessions in a clinical setting. Within FFT, there is a very strong stance against blaming and negativity. Neither individuals nor families are to be seen or treated as “bad” or wrong. Often the families treated by an FFT therapist may be described as resistant to treatment, but FFT attempts to overcome this through its use of the techniques discussed above. When there is a perceived failure or lack of improvement, it is the therapist who must identify ways to alter her approach and treatment of the family, not vice versa. FFT therapists attempt to provide a therapeutic experience unlike that to which most families are accustomed: An experience that focuses on alliance and fostering a cooperative working relationship. Families should feel engaged and aware rather than belittled or blamed for their problems.
Method
Objectives
To describe, assess, and evaluate the existing reviews of research on the effectiveness of FFT as used for the treatment of youth aged 10–18 presenting with behavioral problems. This overview is protocol driven, with a detailed research plan presented to a reviewing body at Oxford University Centre for evidence-based intervention, at which time a priori methods and outcomes were established and approved.
Search strategy for identification of reviews
Electronic searches were made of relevant bibliographic databases, government reports, and professional websites. Reference lists of articles were examined and experts contacted to search for the so-called gray literature. Searches for ongoing reviews or protocols were conducted including the Prospero database. There were no publication, geographic, or language restrictions. Searches covered the following sources up to February 5, 2018:
Biomedical sciences databases: MEDLINE, EMBASE, PsycINFO, Cochrane Library; Prospero.
Social sciences and general references databases: ASSIA (1969–), Dissertation Abstracts International (1969–), ERIC (1969), InfoTrac (1969–), ScienceDirect (1969–), Sociological Abstracts (1969–), Social Work Abstracts (1969–), Web of Knowledge/Web of Science (1969–), Social Sciences Citation Index (1969–).
Government policy sources: U.S. Department of Health and Human Services, U.S. National Institutes of Health, U.S. Centers for Disease Control, Office of Juvenile Justice and Delinquency Prevention, U.S. Government Printing Office, U.K. Home Office, The Ministry of Health, Public Safety Canada, and Australian Institute of Criminology.
Personal contacts: First author made contact with FFT developers, practitioners, and independent researchers to identify unpublished reviews.
Cross-referencing of bibliographies: References in located reviews were checked to identify additional sources.
Search terms: Search terms for MEDLINE (modified as necessary for other databases) were as follows: FFT (either as index treatment or as comparison arm) and (evaluation$ or outcome$ effect$ or review$ or analysis).
These terms were selected by considering previous reviews and via a library specialist at University of Oxford.
Criteria for considering reviews for this overview
Reviews that focused on randomized controlled trials (RCTs) and quasi-experimental designs which met the Cochrane Effective Practice and Organization of Care Review Group criteria were included in this overview. FFT must be reported, evaluated, and described in sufficient detail to enable analysis. Reviews had to clearly report on and evaluate at least one primary study of a specifically licensed FFT program or site. The reviews evaluated a specifically licensed FFT program or site and included the core elements needed for a program to qualify.
The review must have inclusion criteria for studies which compared a licensed FFT program to another service, alternative treatment, or no treatment. No treatment includes simple probation, which in the United States varies from state to state but is broadly defined by the Juvenile Law Center as any disposition involving the supervision of a delinquent youth in the community rather than in a secure confinement facility; commonly, it is comprised of approximately 85% checking in or monitoring and 15% education or mentoring services with possible elements of tracking devices, house arrest, or community service (https://jlc.org). Reviews may include any of the primary studies of FFT. Book sections, memos, government documents, and presentations were included. For complete screening forms, contact author.
Types of outcome measures reported
The three primary goals for both providers and recipients of FFT are reduction in recidivism rates, fewer removals from the home, and reduction of substance use or abuse (Alexander et al., 2013). Acceptable objective outcome measures include police and court records, out-of-home placement or the termination of parental rights, drug screening tests, and school reports of truancy, suspensions, or expulsions. Secondary outcomes are subjective and include parent and teacher reports of disruptive, acting out or unmanageable behaviors, self-reported delinquency, drug use, internalizing symptoms, peer relations, and family conflict levels. Family dysfunction, while a significant concern, is seen in FFT as residing at the core of more serious offending and high-risk behaviors and is not an outcome. FFT proposes in their training and dissemination materials that follow-up periods should last anywhere from immediately after the conclusion of treatment to 5 years postintervention, and the three primary outcomes of FFT have been used as measurement of the intervention since its original development (Alexander et al., 2013).
Primary objective outcomes
Recidivism defined as rearrest, incarceration, or reoffending. Recidivism reports relied upon court records, judicial determinations, probation reports, and conviction rates; self-reported or otherwise nonadjudicated criminal activity is not included in recidivism rates.
Substance use determined by standardized drug screening performed by agency, court, or probationary staff.
Placement out of the home or entrance into care in a group-home, foster care, juvenile detention, or secure treatment facility.
Secondary outcomes
School attendance, as reported and documented in official school records.
Parental reports of externalizing behaviors of youth and family functioning as included in the integrated reported system contained in the FFT model.
Self-reported internalizing and externalizing delinquent behavior ideally collected through standardized instruments such as the Child Behavior Checklist, Behavioral Questionnaire for Children (BASC-2; Achenbach, 1991). Data drawn from self-reports authored by FFT Inc. included in the intervention manual were also eligible for inclusion (http://www.fftllc.com).
Reviews without at least one primary FFT study or without control comparisons were excluded, as were discussions of adaptations of FFT and programs similar but not explicitly stated to be an FFT program.
Assessment of methodological quality
The Assessment of Multiple Systematic Reviews (AMSTAR) checklist (Shea et al., 2007) for the evaluation of systematic reviews was selected, but there did not exist sufficient information contained within many reviews to adequately complete this assessment. Instead, reviews were assessed to determine the design of included studies, whether there existed any method of critical appraisal, and whether or not the authors had a connection to FFT Inc. that may result in a degree of bias.
Quality of evidence in included reviews
Quality of included reviews was assessed by whether or not included studies were randomized, length of follow-up, heterogeneity, imprecision, and whether trials were adequately powered. A number of the reviews include multiple trials that are not RCT, which places them at a lower level of quality. Additionally, evidence may be downgraded due to the fact follow-up periods are variable and often do not exceed 6 months posttreatment. Furthermore, a number of studies included in the reviewing literature contain sample sizes that are fewer than 100 participants, which leads to imprecision of evidence. The importance and impact of heterogeneity among participants within studies and between reviews is considered. Close attention was paid to whether there were enough common elements between reviews that allowed them to be discussed within this work. The issue of heterogeneity and review quality extends from the trials themselves to the type of included review, paper, or report.
Data extraction and management
A data collection form was piloted and all data coded and put into an extraction sheet. All data were independently double extracted and any disagreements resolved by a third party. For additional information regarding data collection, contact first author.
Results
Results of the Search
The search strategy, capturing titles, abstracts, and key words yielded a total of 159 references, considered as possible reviews or meta-analyses, with 31 duplicates and two registered protocols of reviews currently in progress. After screening abstracts, 31 were excluded as not being reviews of primary research and 91 identified for full-text retrieval. After examining the texts, 60 were discarded for ineligibility (identified as primary research, training manuals, not containing information relating specifically to FFT, or did not report results for FFT); 6 meta-analyses and 25 narrative reviews, totaling 31 reviews (n = 31), met all eligibility criteria and were evaluated for overall quality, scientific rigor, quality of evidence, and potential bias. See the PRISMA chart (Figure 1).
PRISMA flow diagram.
Included Reviews
There were 31 reviews or meta-analyses, chapters, or policy briefs which met inclusion criteria. All reviews met the qualifier of discussing only those programs that are licensed and clearly stated to be FFT. Programs that may contain similar elements to FFT but were not licensed as such were not included. Nor were reviews only discussing adaptations of FFT. All included reviews must cite and discuss at least one primary study of FFT in enough depth to justify inclusion in this overview. See Table 2 for list of excluded reviews.
Research has suggested that there is a distinct possibility that reviews conducted or written by investigators who have an allegiance to the program models they are investigating produce significantly more positive results than those conducted by investigators without such allegiance (Eisner, 2009; Shadish, Cook, & Campbell, 2002). The reported effect sizes of prevention and intervention trials have been shown to be noticeably larger when program developers are involved as opposed to those conducted by independent researchers (Eisner, 2009). Alternatively, it may be important to note that positive effects of an intervention during a study conducted by “developers—as evaluators” may be attributed to either (a) the possibility that the quality of implementation is of a higher standard when it is delivered by a developer of the intervention or (b) disparity in outcomes between studies delivered by developers versus independent researchers that can be attributed to systemic bias relating to an ongoing conflict of interest by a champion of the program and possibly related to financial factors. The methods of this overview consider the presence of reviewer bias and study design, by comparing results of reviews published by FFT program developers and stakeholders to those obtained by others.
Design of included reviews
The trial design of included studies is not stated in the majority of reviews, and only 16 of the 31 utilize any standardized or peer-reviewed critical appraisal method (see Table 1).
Design of Included Studies.
Note. RCT = randomized controlled trial; FFT = functional family therapy; CoI = conflict of interest.
Participants in included reviews
Only 10 reviews reference the age of trial participants and sample inclusion criteria, and only 5 contain information regarding race or gender of study participants. Contextual demographic information is often missing from the reviews.
In the meta-analyses, the authors briefly describe the inclusion criteria for the participants in the trials. Excepting Farrington (2003), the meta-analyses include youth aged 11–18, with a diagnosed substance use or abuse disorder. While age and sex of participants is included, there is no further information concerning core elements that may directly impact the outcomes of the intervention.
Primary Outcomes Reported
Recidivism as an outcome measure is reported in 16 of the reviews, which includes 11 studies containing a total of 1,427 unique participants. Ten reviews include a percentage rate for reoffending ranging from 11% to 67% recidivism in the FFT groups versus 36% to 93% in comparison and control groups, although definitions of recidivism vary widely across reviews. Sexton, Ridley, and Kleiner (2004), Sexton (2004b) and Dumas (1989) report “significant reductions” in recidivism and “less than half” the recidivism rate of the comparison or control group, without any specific data being provided. Diamond, Serrano, Dickey, and Sonis (1996), Huey and Henggeler (2001), and Chamberlain (1995) state that FFT results in reduced recidivism but provide no clear data nor specify the type and severity of crimes. Across the reviews, authors do not clarify which definition of recidivism is being used. Only four (n = 2,629) state what reports or records were accessed within the studies to gain this information: Alexander and Sexton (2002), Gordon (1995), Henggeler and Sheidow (2003), and Vaughn and Howard (2004).
Substance abuse is only reported in 10/31 reviews; six (n = 1,846) of these are meta-analyses focusing on cannabis or substance abuse. The included meta-analyses are the only reviews that report effect sizes within them. The effect sizes reported range from a Hedge’s g of −1.18 to 0.21, with three meta-analyses reporting nonsignificant effect sizes (Baldwin, Christian, Berkeljon, & Shadish, 2012; Bender, Tripodi, Sarteschi, & Vaughn, 2011; Vaughn & Howard, 2004). Of the remaining four, Hogue and Liddle (2009; n = 349) state that there was “an improvement in drug use” (p. 18); Vaughn and Howard (2004; n = 249) report a 50% reduction in the drug severity index; and Waldron and Turner (2008; n = 341) state that the treatment group “showed significant reductions in substance use at post and/or follow-up” (p. 248).
Out-of-home placements are not reported as an outcome of any study despite claims that FFT is an effective tool in the reduction of out-of-home placement for children and youth who are at risk of removal (Alexander et al., 2013).
Secondary Outcomes Reported
School attendance is not discussed or reported in any review of FFT, nor is there any reported collection of data concerning school attendance. This information is included neither in the included reviews nor in the registered protocols of FFT reviews still in progress. Thus, no judgment can be reached regarding whether FFT will result in better (or worse) school attendance and truancy.
Parent reports of family functions, interactions, and youth-related conduct issues were not routinely included, with no mention of clinical services system reports, no questionnaires, and no other data collection tool utilized in any of the included reviews. It is not possible to determine whether it is the trials themselves or the author reporting them that did not collect this information. The FFT model has embedded within it evaluations and assessments relating to family functions, and it is striking that these data are not included in the reviews of the research.
Self-reported internalizing and externalizing delinquent behaviors are also absent from reviews. Again, FFT has a method of collecting these data, but it was either not collected or not reported. Additionally, reviews did not contain the use of, or reference to, other measurement scales such as the BASC-2 or other peer-reviewed depression or behavioral scales. Information relating to self-reported criminal activity, depressive symptoms, family conflict, school-related behaviors, and other behavioral- or conduct-related issues are not present.
Other Outcomes Reported
Earlier reviews of FFT, which reference the initial studies such as Alexander and Parsons (1973; n = 20); Klein, Alexander, and Parsons (1977; n = 86); and Barton, Alexander, Waldron, Turner, and Warburton (1985; n = 27), report findings that are neither objective nor easily quantifiable. These outcomes are described as “reductions in conflict level,” a “longer duration of silence,” and “decreased defensiveness.” These outcomes, while appearing to be beneficial, are open to interpretation and do not appear to be based on valid, objective, or peer-reviewed measures.
Quality of Evidence in Included Reviews
Reviews, book chapters, and reports written by the creators of FFT or by individuals who are now or have been employed by FFT, Inc. may be at a high risk for bias. Only 5 of the 31 included reviews contain summary of findings tables with risk of bias assessments and only 16 include the use of a peer-reviewed critical appraisal method. These reviews generally report only the positive outcomes or measures while ignoring other less favorable outcomes. See Table 2 for complete listing of reviews and trial design of included studies.
Table of Excluded Studies.
Note. FFT = functional family therapy; CoI = conflict of interest .
Effect of Intervention
When examined as a whole, the reviews describe effects of FFT in inconsistent terms. Importantly, no review reports a directly harmful or detrimental outcome. A number of reviews, however, report no significant differences between groups in any of the outcome domains including family functioning or decreased internalizing and externalizing behaviors (Austin, Macgowan, & Wagner, 2005; Baldwin et al., 2012; Tarolla, Wagner, Rabinowitz, & Tubman, 2002; Tolan, 1986). Recidivism is measured by different means across studies, and the reviewing authors do not specify which measure or definition of recidivism they are using as an outcome measure. This makes any assumptions or inferences regarding recidivism extremely difficult to substantiate. The recidivism rates reported range from 11% to 67%, and Farrington (2003) draws attention to the fact that although low recidivism rates were reported, there were definite methodological problems with attrition and variable follow-up periods. The lowest recidivism rate was found in Gordon (1995), in which authors cite a quasi-experimental trial they had conducted in rural Ohio with court-mandated youth (n = 27). The highest rates of recidivism (60% or above) are reported by Huey and Henggeler (2001; n = 189); Mulvey, Arthur, and Reppucci (1993; n = 40); and Gordon (1995; n = 106) and refer to a trial conducted by Barton and colleagues in 1985 (Barton, Alexander, Waldron, Turner, & Warburton, 1985). Specifics of this trial are not included in the reviews.
Discussion
Summary of Main Results
This overview aimed to gather and assess the preexisting reviews of FFT and to present information regarding the model in a single comprehensive document. The narrative reviews of FFT did not suffice as a reliable source of information regarding the model and left a great deal of room for conjecture and uncertainty. The three primary outcome measures promoted by FFT Inc. have not been adequately addressed in the existing literature. This overview sheds light on this issue and highlights the importance of increasing involvement of those not directly linked to the development or dissemination of FFT Inc. Even with the evidence currently available, there is no indication that a course of FFT will not cause detrimental effects, or harm. Without this certainty, it seems unwise to advocate for the intervention without hesitation.
Recidivism is a complex outcome which is open to interpretation. It can refer to any number of outcomes including contact with the court system, rearrest, conviction, and self-reported criminality. A higher standard of proof is employed when obtaining a conviction versus an arrest. To measure recidivism, only as a subsequent conviction greatly skews the outcomes in favor of a lower recidivism percentage rate. Differing definitions of recidivism may lead to any reported rate, across the included reviews, becoming less meaningful as a result. The picture of evidence that emerges from studies employing completely different standards of measurement will be diverse; without identifying and defining these differences, we cannot reach valid conclusions. According to the National Institute of Justice and the Office of Juvenile Justice Prevention, a status offense such as truancy or probation violation can be counted as recidivism but should not be measured by the same metric as a felonious arrest. An arrest or conviction requires a far greater standard of suspicion and proof, whereas status offenses are most often reported by probation officers and do not contain antisocial or disruptive elements. These inconsistencies and lack of designation often leads to an overarching lack of clarity in the conclusions (https://www.nij.gov; Measuring Recidivism. (n.d.). Considering many reviews did not report this outcome, it becomes even more difficult to reach conclusions relating to this primary outcome, which is unfortunate since the intervention was in part specifically designed for this purpose.
Farrington (2003), a meta-analysis with a focus on recidivism rather than substance abuse, concludes that the two trials (Alexander & Parsons, 1973; Gordon, 1995) included do lead to positive results and a significant effect. Despite a better standard of reporting and analysis than many other reviews, the sample size in these two trials was 52 and 86, respectively (Farrington, 2003).
While FFT is considered to be a method of reducing out-of-home placements, no review reported on this outcome. It is perhaps possible that out-of-home placements increased in duration or frequency. Without clearly stating this result, it is unwarranted to assert that FFT reduces these occurrences as claimed in FFT site and training literature (Alexander et al., 2013; http://www.fftllc.com). For this outcome to go unreported and undocumented is troubling at least and possibly a significant failing of the current research and trials of FFT. Similarly, reduction in substance abuse is one of the primary outcomes as stated by FFT, and gathering objective data relating to this outcome would do a great deal in terms of strengthening the evidence around the claim that a course of FFT will lessen incidents of substance misuse.
School attendance as an outcome is also not sufficiently addressed in the literature, and it is striking that such an objective measure was not reported in the reviews. It is impossible to know whether these data were simply not collected, thus being a reporting error, or whether the reviewers chose not to include it in their results, because they judged it to be insignificant, they did not have access to it, or for any other reasons.
FFT does include in its design multiple methods for the reporting and rating of internalizing symptoms, but these records are not included by the reviewing authors. Additionally, there are additional standardized methods of measuring these symptoms that are either not employed by trialists or not reported on the results.
The current evidence suggests reviews may not include all available research and pertinent contextual information in analyses, resulting in an incomplete or skewed picture of the evidence. It is impossible, when examining the reviews of FFT, to establish whether the trials referenced are well designed, executed, and significant as the information necessary to make such judgments is too often absent from the review. Reviews are vulnerable to biases and “perspective” that may influence the reader to lean in one direction or another. Reviews do not examine any possible bias of referenced trials or include a risk of bias assessment, though one mentions that “most trials were performed by the developers of the model.” Reviews must have a sound methodological basis for reaching any conclusions, an approach that is outlined and supported by both the Cochrane Grading of Recommendations Assessment, Development and Evaluation guidelines and AMSTAR (Higgins & Green, 2011; Shea et al., 2007).
Reporting guidelines that have been widely adopted were not employed. Consolidated Standards of Reporting Trials (CONSORT) has been developed, designed, and is being improved upon to assist in the consistent and standardized reporting of trials (Motgomery et al., 2018; Schulz, Altman, & Moher, 2010). Still, it would behoove researchers to adopt these guidelines due to the fact that, were trials more completely and rigorously reported, the standardized reviews of them would in turn be of a higher quality. If CONSORT guidelines had been utilized in the reporting of the FFT trials, current available research would be of greater value and substance.
Reviews of FFT often do not report or discuss any elements of implementation or contextual concerns, staff characteristics, or adherence within the trials. These issues are known to have a significant impact on the outcomes and efficacy of interventions (Montgomery et al., 2013a), and this information could be invaluable when determining the strengths and weaknesses of the model.
When describing sample inclusion criteria, 10 reviews (Alexander et al., 2002; Austin et al., 2005; Breunlin, Breunlin, Kearns, & Russell, 1988; Cottrell & Boston, 2002; Dumas, 1989; Gordon, 1995; Henggeler & Sheidow, 2003; Huey & Henggeler, 2001; Sexton, 2009; Waldron & Turner, 2008) refer to study participants, but the criteria are not elucidated well and include phrases such as “light to severe offenses,” “habitually truant,” “soft” juvenile offenders, and “serious juvenile offenders.” There is no definition of these terms and no explanation of what level of behavior is present. There is a great deal of room for interpretation. Waldron and Turner (2008) report outcomes for youth diagnosed with “heavy alcohol abuse”; Dumas (1989) reports the participants were court referred; Austin, Macgowan, and Wagner (2005) discuss a sample that required a substance abuse diagnosis; and Breunlin, Breunlin, Kearns, and Russell (1988) report on a study with status offenders. These are definable categories which are far more objective and useful when examining the quality of design and methodology in the included reviews.
Contextual information such as socioeconomic status, number of family members, criminal backgrounds, and education has been shown to have an impact on effectiveness and outcomes of FFT. Thus, it is problematic that this information is not included by the review authors. Making well-informed decisions relating to the use of FFT for specific populations based on evidence is difficult if and when this information is not readily available or accessible. A high-quality systematic review of the evidence may help in remedying this situation, but for the purposes of this overview, such information was not readily available in the extant literature.
Outcomes and the Theory of Change
The 31 reviews of FFT included do not report consistently on the core outcome measures of FFT, and therefore, it becomes difficult to determine whether it is in fact an effective method of treating conduct and behavioral problems in youth. There is no clear linkage between the outcomes FFT Inc. promotes treatment for and the research outcomes. Of the six main outcomes described by the developers, substance abuse is reported in 7 of the 31 reviews and recidivism in only slightly over half of them (n = 16). This does not allow for any strong conclusions to be reached regarding the treatment’s efficacy. But once more, it is not possible to ascertain whether this is due to possible design or implementation failure in the trials or to a lack of consistency and breadth in reporting.
Trials examining FFT have been completed almost exclusively by the developers of the model or by those who otherwise benefit from its use and uptake and have a significant stake in the success of the model. Even in instances of independent reviewing authors, the trials contained in these reviews were performed by developers of FFT. This may influence the outcomes in a number of ways. When a trialist is particularly familiar and involved with the intervention being researched, it may lead to increased fidelity and adherence to the original model design (Elliott & Mihalic, 2004). However, it has been indicated that studies conducted by investigators who have an allegiance to the program models they are investigating produce significantly more positive results than those conducted by investigators without such allegiance (Eisner, 2009; Shadish et al., 2002; Oxman & Guyatt, 1991).
There are many persuasive arguments relating to the need for standardized methods relating to the completion, evaluation, and reporting of trials, and these demands become increasingly important and difficult when examining a complex intervention. Fidelity must also be examined not just as adherence to a manual or program guideline but fidelity to the underlying mechanisms themselves. Implementation concerns are not adequately addressed in the existing reviews; FFT is largely believed to be a very adaptable model, a belief that is not demonstrated in the research. It is unknown whether the trials themselves do not examine implementation or whether the reviewing authors did not gather this information. Both implementation and fidelity are major concerns when assessing the outcomes and overall effectiveness of a complex intervention. These issues cannot be adequately rated within the scope of this overview; the reviews included in this trial did not contain within them consistent or detailed information regarding duration of treatment, skill, and training of practitioners; context; and adherence to the model or attendance of either practitioners or recipients of the intervention. Without this information, it is not possible to ascertain what concerns and impact implementation may have on the outcomes. Such implementation data should have been addressed and reported in the original reviews; where there is no discussion of this, it was noted as a possible demonstration of failure on the part of either the reviewers or those running the trials to heed the importance of implementation fidelity (Montgomery et al., 2013b).
Overall Completeness and Quality of Evidence
The existing reviews do not offer sufficient evidence on which to base decisions concerning FFT. Of the 43 available randomized, quasi-experimental, and observational trials across 31 reviews which include a total of 4,607 unique participants fewer than half are identified, included, or discussed in the reviews. In many instances, the reviewers fail to specify populations served, participant characteristics, or outcomes measured in the trial. The reviews of FFT do not provide information such as whether or not a protocol existed or was registered, search strategy for identification of studies, methods used, or outcomes measured. This creates difficulties in the gathering of substantial information concerning the quality or completeness of many of the reviews.
Agreements and Disagreements With Other Studies or Reviews
Prior reviews of FFT present a picture that is incomplete, but largely positive. The reviews date back to the mid-1970s and contain varying degrees of detail, thoroughness, and complexity. Previously published reviews state with minimal hesitation that FFT is an effective and beneficial method of treatment for behavioral problems, CD, and substance abuse issues. It appears that this is a conclusion arrived at without the use of consistent or sound methodology. This overview employed rigorous methods of identification and evaluation of the existing reviews. Additionally, tables included in this overview were completed not through information provided in the reviews but by use of the original primary studies in an effort to gain a better picture of the evidence. The sound and comprehensive search strategy, methodological process of evaluation, and analysis methods arrive at a different conclusion; one that presents significant areas for further examination and research, as well as shedding light on some of the questions regarding efficacy of FFT across different populations and highlighting the need for implementation analysis.
Potential Biases in the Overview Process
It is impossible to eliminate all potential for bias, the authors have made every attempt to locate and evaluate all the relevant reviews, as well as double coding data and having a third party resolve any disagreements. There is also a risk of bias stemming from the absence of reviews still in progress; currently, there is a registered literature search and meta-analysis which is being funded by FFT Southwark with an unknown prospective date of completion. A systematic review of FFT protocol is currently being undertaken and is registered with the Cochrane Collaboration and when completed will be adherent to Cochrane guidelines.
This overview gathered and assessed the preexisting reviews of FFT and presented this information in a comprehensive and rigorous manner. It was determined that the existing reviews of FFT did not suffice as a reliable source for this information and left a great deal of room for conjecture and uncertainty. The primary outcome measures promoted by FFT Inc. have not been adequately addressed in the existing reviews.
Strengths and Limitations
Overviews aim to provide a comprehensive picture of the existing evidence for a particular intervention, by assessing and examining the existing reviews. This overview differs from existing reviews in that it meets the standards set out in the Cochrane handbook guidelines for rigor and quality. The methods used in searching, data extraction, and synthesis are clearly detailed, possible biases are reported, and there is a higher level of transparency than is present in the existing literature whether in reviews or overviews. Double coding was employed throughout the overview, and authors were contacted to confirm or clarify data when necessary.
Conclusion
This overview outlines the uncertain effects of FFT across different populations, and the existing reviews illustrate the fact that certainties regarding FFT are based on a tenuous research base. The initial research, effectiveness, and efficacy trials were held in Salt Lake City, UT. While there are some trials of FFT conducted by independent researchers (n = 17), the majority of research continues to be conducted by the developers and disseminators of FFT (n = 27). Additionally, many of the reviews currently available are authored by the developers of the model. While this is to some degree expected, it is demonstrative of a need for independent researchers to conduct well-designed and well-reported trials and subsequent reviews of FFT.
Given these factors, it may not be advisable to continue adopting FFT without reexamining and testing the effects. It is difficult to determine the quality of any evidence regarding the efficacy and impact of FFT and it is necessary that this situation be remedied. Trials and replications should be performed and reported with greater stringency around guidelines and quality control. FFT has not yet been adequately proven effective across culturally disparate and varied populations. There is a need for well-powered trials that include all primary outcome measures of FFT this would measure out-of-home placements, and internalizing and externalizing behaviors; utilize a consistent measure of recidivism; and evaluate cost-effectiveness. It appears that in nearly 40 years of existence, there continue to be a number of unanswered questions regarding effectiveness and implementation. Continued research and investigation with a focus on implementation with a high standard of design will do a great deal to address these concerns.
Footnotes
Acknowledgements
Authors would like to recognize the contributions of Dr. Jane Dennis, University of Bristol; Julia H. Littell, Bryn Mawr College; Elaine Kidney, University of Birmingham; and Library Sciences, University of Oxford.
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors received no financial support for the research, authorship, and/or publication of this article.