Abstract
Psychotherapies for children and adolescents have been tested in hundreds of randomized controlled trials across five decades, and many of these youth therapies have now been classified as empirically supported treatments (ESTs). A burgeoning movement is underway to implement these ESTs in clinical practice settings, but questions arise as to whether the treatments are ready for practice and whether they will improve outcomes for clinically referred youths. Our data show ESTs to be more effective than usual care, on average, but only modestly so, and there are troubling exceptions. One reason may be that the design of most ESTs (e.g., single-disorder focus, linear session sequence) does not fit the characteristics of referred youths or clinical practice very well. Indeed, youth psychotherapy research has not focused much on the clients or contexts of actual clinical care. An alternative empirical approach, the deployment-focused model, proposes developing and testing interventions with the clients, clinicians, and contexts for which they are ultimately intended. Recent application of the model highlights its potential for stimulating robust treatments that are effective in clinical practice.
Efforts to help children and adolescents (herein collectively referred to as “youth”) are as old as parenthood. However, specific strategies for helping have evolved dramatically over time and have been influenced by ancient moral teachings (e.g., spare the rod, spoil the child), classical philosophy (e.g., using discourse to explore and restructure thoughts and behavior), medicine, and other healing traditions. The approach called youth psychotherapy is often dated to Sigmund Freud’s consultation with the father of a very anxious “Little Hans” and Freud’s psychoanalysis of his own daughter. Youth psychotherapy was later shaped by psychology’s grand theories, humanism, behaviorism (e.g., Jones, 1924), cognitive and cognitive-behavioral methods (e.g., Meichenbaum & Goodman, 1971), and diverse alternative approaches. By the turn of this century, Kazdin (2000) identified 551 different named therapies used with children and adolescents.
Empirical Tests of Youth Psychotherapy
Youth treatments began to be tested empirically in the mid-1900s. Initial quasi-experimental studies of vaguely described treatments were followed by increasingly rigorous randomized controlled trials (RCTs) testing better documented, manual-guided psychotherapies. With this shift, concerns that youth psychotherapy had no effect (Eysenck, 1952; Levitt, 1963) gave way to evidence showing therapy outperforming various control groups. Eventually, meta-analyses (e.g., Weisz, McCarty, & Valeri, 2006; Weisz, Weiss, Han, Granger, & Morton, 1995) showed respectable mean effects across hundreds of RCTs—effects within the range found for adult psychotherapy (see Fig. 1). Task forces have now applied scientific criteria to the accumulating RCTs to identify evidence-based or empirically supported treatments (ESTs) for youth (e.g., Silverman & Hinshaw, 2008). The criteria for EST status differ somewhat across various task forces and review groups, but most require multiple supportive RCTs, ideally conducted by independent research teams. Identifying ESTs is now a growth industry: The National Registry of Evidence-Based Programs and Practices (http://www.nrepp.samhsa.gov/ViewAll.aspx; accessed 10/05/2013) lists 306 “evidence-based” interventions—192 for children and adolescents. A national movement is now underway to implement these interventions in everyday treatment settings.
Mean effect sizes found in two broad-based meta-analyses of adult psychotherapy effects (the two bars at the left: Shapiro & Shapiro, 1982; Smith & Glass, 1977), four broad-based meta-analyses of youth psychotherapy effects (the four middle bars: Casey & Berman, 1985; Kazdin et al., 1990; Weisz et al., 1987; Weisz, Weiss, Han, Granger, & Morton, 1995), and two meta-analyses of RCTs comparing evidence-based youth psychotherapies to usual clinical care (the two bars at the right: Weisz, Jensen-Doss, & Hawley, 2006; Weisz, Kuppens, et al., 2013). The full bar for Kazdin et al. (1990) shows the mean effect size for treatment versus inert control group comparisons; the dashed line shows the mean for treatment versus active control group comparisons. The full bar for Weisz et al. (1995) shows the mean effect size when unweighted least squares analyses were conducted; the dashed line shows the mean for weighted least squares analyses. (Reprinted with permission, John R. Weisz)
As we enter the implementation era, two critical questions arise for clinical science. First, do the ESTs actually improve youth outcomes more than current practices do? If not, then the case for implementing ESTs may not be so clear. Second, are the ESTs designed in ways that fit the clinical care contexts where implementation will occur? If not, the ESTs may be difficult to implement properly where they are most needed (i.e., where clinically referred young people are actually treated). These two questions pivot on a core issue for our field: Is clinical science in its current form producing interventions that are robust enough to succeed in the real-world contexts where most youth mental health care takes place? My students, colleagues, and I address these questions in much of our work.
ESTs Versus Usual Care
Our RCTs and meta-analyses have shown that classifying a treatment as an EST is no guarantee that it will outperform the status quo. In fact, benefit tends to drop markedly when treatments leave the secure base of their university or laboratory settings and are tested against usual clinical care. Our meta-analyses of RCTs pitting ESTs against usual care (Weisz, Jensen-Doss, & Hawley, 2006; Weisz, Kuppens, et al., 2013) show highly variable outcomes, numerous studies in which ESTs do not outperform (and in some cases actually underperform) usual care, and markedly lower mean effect sizes than studies using mainly waitlist and experimenter-constructed control groups (see Fig. 1). In fact, the mean effect sizes in our EST versus usual care meta-analyses reflect a probability of only .58 (vs. chance at .50) that a randomly selected youth treated with an EST would be better off after treatment than a randomly selected youth treated with usual care. Importantly, our most recent meta-analysis (Weisz, Kuppens, et al., 2013) showed that ESTs did not significantly outperform usual care among studies using clinically referred youths or youths impaired enough to meet criteria for a formal diagnosis. These two groups are arguably prime targets for EST implementation.
These findings may be a logical consequence of the research that has produced the youth ESTs; that research is not very representative of real-world clinical practice (Weisz, Jensen-Doss, & Hawley, 2005). Although the studies are generally high in internal validity (good news), most of them are low in external validity (bad news, if effective implementation in clinical care is one’s goal). In a recent examination of 461 youth RCTs (with 1,160 treatment and control groups), spanning the 1960s through the most recent decade, we found that only 2.1% of all the groups were described by authors as involving clinically referred clients treated by practitioners in practice settings (Weisz, Ng, & Bearman, in press).
This body of relatively unrepresentative research has produced primarily (a) treatments for single disorders, or homogenous clusters, even though most clinically referred youths have comorbidity, multiple co-occurring problems, and diverse treatment needs; (b) linear treatments—with therapist manuals presenting session contents in a prescribed order (e.g., psychoeducation first, then relaxation training)—even though everyday treatment is packed with unexpected events and shifts in treatment needs; and (c) treatments that work well when implemented by researchers’ own students or employees and under conditions designed by researchers, even though real-world implementation definitely will not be done that way. As we have seen, these ESTs, which do not seem to fit the everyday clinical care context well, also fail to show either consistent or substantial effects in that context when compared to usual care (Weisz, Jensen-Doss, & Hawley, 2006, Weisz, Kuppens, et al., 2013).
The problem with ESTs that do not fit well or perform well in practice settings is not just a concern for clinical science but a challenge for our colleagues in the clinical practice community as well. The risk is that practitioners may not have access to those EBTs that will most benefit their practice and most enhance outcomes for the girls and boys they treat. Clearly, clinical scientists and clinical practitioners have a shared interest in the development of treatments that fit well and perform effectively within the ecosystem of real-world clinical care.
Building More Robust Treatments
To build robust treatments that are potent in everyday clinical care, we have proposed a deployment-focused model (Weisz, 2004; Weisz & Gray, 2008). The model calls for interventions to be developed and tested with the kinds of clients and therapists, and in the kinds of settings, for which the interventions are ultimately intended and to be routinely tested against usual care. Such research can and should meet the same rigorous methodological standards as traditional RCTs, but it can arguably generate more externally valid evidence on intervention effects, moderation, mediation, and mechanisms of change than would research in clinically unrepresentative contexts. The deployment-focused model reflects this notion: ESTs that have been built for, adapted to, and tested within clinical care contexts should be more effective in those contexts than ESTs that have been designed and tested in less clinically representative ways.
Our application of this model over the years has taught us a lot about the conditions within which ESTs must be robust to succeed (Weisz, Ugueto, Cheron, & Herren, 2013). Our most recent effort, in collaboration with wise colleagues in the Research Network on Youth Mental Health, led to the creation of a transdiagnostic protocol designed to address youth comorbidity (anxiety, depression, and misconduct) and ongoing shifts in treatment needs (Chorpita & Weisz, 2009). An RCT showed this treatment approach to be markedly more effective than usual care and more effective than traditional single-disorder treatments (Weisz et al., 2012). We expect to continue applying the deployment-focused model and experimenting with treatment innovations; in the process, we hope to engage an ever-larger cohort of clinical scientists building vigorous, practice-ready treatments and a rigorous evidence base on their impact.
Footnotes
Acknowledgements
I am grateful to so many wise colleagues, postdoctoral fellows, and graduate students for their extensive contributions to this work; to Lea Petrovic for her valuable help with this manuscript; and to the children, families, clinicians, clinic administrators, and government program and policy leaders who have participated in and supported our research and enriched our thinking.
Declaration of Conflicting Interests
I receive royalties for some of the published works cited in this article.
Funding
Some of the research reported here was supported by grants from the National Institute of Mental Health (MH57347, MH068806, MH085963), the Norlien Foundation, the MacArthur Foundation, Casey Family Programs, and the Annie E. Casey Foundation.