Efforts to Bridge the Gap Between Research and Practice in Social Work

Implementation Science

In response to this disappointment, a rich field of implementation science has blossomed, which is beginning to identify factors at various organizational levels that help to explain why some organizations have been more successful than others in implementing ESTs with adequate fidelity and good outcomes. By identifying these factors, implementation science can offer recommendations for bridging the gap between research and practice (at least with respect to implementing ESTs with fidelity; Damschroder et al., 2009; Embry & Biglan, 2008; Glisson & Schoenwald, 2005; Weisz, Ugueto, Herren, Afienko, & Rutt, 2011). Some of these promising recommendations were presented by speakers at this conference.

Despite the value of these recommendations, however, their successful implementation will have to overcome some negative attitudes about research among not only rank-and-file practitioners and administrators, but also among some of our profession’s esteemed leaders. For example, in 1998—as president of Society for Social Work and Research (SSWR)—I attended a National Association of Social Workers (NASW) Summit Meeting that was held with the purpose of uniting the diverse elements of the social work profession and identifying what issues and priorities social work organizations could agree upon. And yet—despite that unification purpose—the Summit’s keynote speaker decried the excess amount and excessive influence of research in social work education. He added that he is “insulted” by the notion that we need to research the outcome of our efforts to provide care. Hearing that, I left that summit less than sanguine about the prospects of ever bridging the gap between research and practice (Rubin, 1999). However, that summit meeting was 15 years ago. Let’s hope that much has changed since then—especially in light of the growth of the evidence-based practice movement.

Common Elements and Common Factors

One promising change is the current effort to begin to identify common elements that are shared by various ESTs. The aim of identifying those common elements is to make the implementation of ESTs more acceptable, less complex, and less costly to practitioners and agencies. Instead of expecting practitioners to rigidly adhere to an entire manualized EST, identifying the core essential and indispensible elements of the EST as well as its adaptable elements might give practitioners more flexibility to make the EST fit their organization and clientele and also reduce the costs of practitioner training (Galinsky, Fraser, Day, & Rothman, 2013; Sundell, Fwrrer-Wreder, & Fraser, 2012).

However, there is a big difference between adapting a specific EST for a specific setting or population, and trying to identify a broad array of common elements shared by various ESTs. Taking the broader approach to identifying common elements can be risky. For example, consider the difference between the common elements identified by Chorpita, Becker, and Daleiden (2007) and those identified by Bender and Bright (2011).

Chorpita et al. (2007) searched for common elements in the ESTs for “depression in girls between ages 10 and 12” (p. 648). Their search yielded the 23 common elements displayed in Figure 1, in order of the relative frequency with which each appeared in the randomized clinical trial (RTC) literature for that target population (p. 468).

OPEN IN VIEWER

Figure 1.

Common elements identified by Chorpita et al. (2007) for the treatment of depression in girls between ages 10 and 12.

In contrast to the approach taken by Chorpita et al. (2007), which limited their search to depressed girls between ages 10 and 12, Bender and Bright (2011) reviewed RCTs for a target population specified in much broader terms: “disruptive behavior and traumatic stress among adolescent girls.” After reviewing a database of 430 RCTs of mental health interventions for youth (again, notice the broadness of their search boundaries), they recommended that instead of training practitioners in any particular manualized EST for reducing disruptive behavior and traumatic stress among adolescent girls, they be trained in the eight common elements shown in Figure 2.

OPEN IN VIEWER

Figure 2.

Common elements identified by Bender and Bright (2011) for the treatment of disruptive behavior and traumatic stress among adolescent girls.

In the eight common elements derived by Bender and Bright, there is no mention of exposure therapy, which is known to be one of the ESTs with the most empirical support for traumatic stress. Also absent from their list is cognitive processing therapy, which also has a strong evidence base in trauma treatment, especially when coupled with exposure therapy (Rubin & Springer, 2009). It should also be noted that the primary techniques or elements from these treatments (e.g., exposure to the traumatic event, changing distressing cognitions) were also not a part of the list.

Therefore, I recommend caution when employing the common elements approach, especially if the common elements are derived from casting a very wide net to find a broad and diverse database of interventions for a diverse range of target populations. The striking difference between the list of eight common elements derived by Bender and Bright and the 23 derived by Chorpita et al. speaks both to the promise of the common elements approach and to its risks.

Several distinctions are important to note here. One is that the common elements approach can be characterized as falling somewhere within a continuum from very narrow to very broad. As displayed in Figure 3, at the narrowest end of the continuum the approach can be limited to a single EST. In this case, the “common” elements are those that are core parts of the intervention that need to be kept but that can be adapted to fit different target settings or populations. A less narrow approach might involve identifying common elements across a small set of ESTs that target the same problem. A broader approach would identify common elements that target different problems, but would not cast too wide a net regarding the range of problems being targeted. At the broadest end of the continuum would be approaches like the one taken by Bender and Bright, in which a very wide net is cast to identify common elements across a diverse range of ESTs and target problems. I suggest that the riskiness of the comment elements approach increases as we move from left to right—from the narrower to the broader end of the continuum displayed in Figure 3. In fact, while extolling the promise of the common elements approach, Barth et al. (2013) caution that “although the common elements approach allows for … flexibility … following a treatment manual as prescribed may still be the best option when appropriate or available” (p. 110).

OPEN IN VIEWER

Figure 3.

Level of risk of ineffectiveness or harm using a common elements approach.

Transferability of RCTs to Real-World Practice Conditions

Despite the progress being made in implementation science, there is evidence that even when ESTs are implemented with excellent fidelity in real-world agencies they only modestly outperform treatment as usual (Weisz et al., 2011). When considering possible reasons for the disappointing degree to which ESTs are being implemented successfully in the real world of practice, we may need to consider some of the problems in the research supporting the ESTs—especially the degree to which RCTs are assessing ESTs that have limited transferability to real world practice conditions.

One important issue involves differences in clientele, particularly regarding the underrepresentation of comorbid and minority participants in many RCTs. Regarding minorities, for example, despite the strong evidence base for trauma-focused cognitive behavioral therapy (TFCBT) with traumatized children, there is a dearth of evidence regarding whether it is effective with Latino and Latina children. The underrepresentation of comorbid and minority participants in many RCTs is especially problematic for social work practice because so many social work clients are minorities or have multiple comorbid problems. Thus, it is conceivable that even when ESTs are implemented the same way that they were implemented in the RCTs supporting them, they may not be as effective with clients who differ from the RCT clients. Consequently, one implication for further bridging the gap is the need for outcome studies with the kinds of clients that social workers are most likely to serve.

This need is especially applicable to the issue of comorbidity. For example, many clients with PTSD also have a substance use disorder. In addition to knowing whether ESTs like exposure therapy are effective with PTSD, and in addition to finding out if they are effective with clients with such comorbidity, we need to learn about what interventions for substance abuse they need to be combined with, and in what sequence. The need for such studies is particularly serious for the large numbers of military service members now returning from Iraq and Afghanistan, whose PTSD is often comorbid not only with substance use disorders but with other critical problems.

Treatment Fidelity

These issues also have implications for concerns about treatment fidelity when implementing ESTs. Maybe the need to implement ESTs in exactly the same way that they were implemented in RCTs is being overrated. The evidence-based practice process emphasizes the importance of practitioners integrating the research evidence with their knowledge of idiosyncratic client attributes. Therefore, in addition to worrying about whether practitioners are following treatment manuals meticulously when providing ESTs, we should be equally concerned about the need for them to modify the manualized approach to better fit those clients whose comorbidity or other attributes make them different than most clients who have participated in the research providing the empirical support. Likewise, practitioners should consider the flexibility of existing treatment protocols of ESTs when deciding whether to adopt a specific EST and whether and how to adapt it.

For the same reason, another implication for bridging the gap is the need to do more effectiveness studies assessing the outcomes of interventions that are currently empirically supported when they are provided in the context of real-world practice settings where the training and supervision in the intervention may have been less than ideal or where the realities of the practice setting—such as level of practitioner experience or skill, or caseload sizes—might be less desirable than the ideal conditions in the RCTs that provided the empirical support for the intervention.

In addition, these designs should include variables derived from implementation science to try to ferret out the processes and organizational variables that lead to the successful implementation of the interventions that are found to be effective in these settings. Likewise, as mentioned earlier, they should assess what combinations of common elements are most associated with treatment success in settings in which one or more ESTs have been adapted. Social work should be at the forefront of this kind of research, given our emphasis on applied research in real practice settings.

Value of Descriptive Outcome Studies

To facilitate this kind of research, we need to recognize how difficult (and rare) it is for social work researchers to use rigorous RCT designs in assessing outcome in real-world social work practice settings. For those researchers who do not opt for the route of seeking major funding for RCT studies, there is a lower hanging fruit that I believe would be of considerable value.

For the same reason that practitioners do not need baseline phases when monitoring client progress with ESTs, less rigorous and primarily descriptive outcome studies—even some without control groups—can make an important contribution to the knowledge base about those interventions when they are provided in real-world social work practice settings. Since prior studies have already provided causal inferences about the probabilistic effectiveness of those interventions, simple preexperimental studies can descriptively examine the level of outcome attained in real-world settings.

For example, if meta-analyses have shown that on average a particular intervention or program reduces recidivism 30% in the control group versus 15% in the intervention group, then an agency adopting and/or adapting that intervention can see if its recidivism rate drops to a similar degree. Likewise, if meta-analyses have shown that an intervention tends to improve scores on an outcome measure by an average of x points, then an agency can assess the average degree of improvement among its clients who have received that intervention.

Such studies would have immediate value to the agency conducting them in that they would show whether the adopted intervention is as effective in their setting and with their clients as they hoped it would be and whether they might need to tweak it or replace it. Thus, such studies would reduce the gap between research and practice. Moreover, the studies could be aggregated to reflect the external validity and real-world effectiveness of the interventions whose original empirical support was obtained under more ideal practice conditions and with more homogeneous samples, which, by the way, might also help bridge the gap between research and practice. Suppose a large number of these descriptive studies can be completed and reported coupled with descriptions of the way the adapted interventions were tweaked, the common elements employed, the various organizational and practitioner attributes characterizing the practice setting, and the intervention science recommendations that were implemented to foster their fidelity. This literature could provide a basis for an ongoing inductive process—using mixed qualitative and quantitative methods—for developing and then testing hypotheses about the conditions under which ESTs or their common elements can be made to be more feasible to implement and more effective in real-world practice settings. Such designs would also be less expensive and more feasible, as they do not require randomization and related implementation procedures, and require fewer participants to test hypotheses.

To clarify, I am not recommending only studies without control groups. Instead, I am recommending that social work researchers use the best designs that are feasible in real-world practice settings, and that even if a descriptive study without a control group is the only feasible option, then that study can make an important contribution, too, as long as it is assessing the adaption of an EST.

Incentives

But what will incentivize social work researchers—who typically are faculty members whose tenure and promotion require that they do more prestigious research—such as RCTs—to do the hard work of persuading agencies to undertake outcome studies of adapted ESTs and then to do the hard work of carrying out those studies themselves without significant funding? One suggestion I have is for the editors of prestigious social work journals, like Research on Social Work Practice, to promote the submission of such studies.

I recognize that many will disagree with this recommendation, arguing that journals should only publish studies with greater internal validity and that advance more generalizable knowledge. But I would counter that our journals already publish many methodologically weak outcome studies with no more internal validity or generalizable knowledge than the ones I am suggesting (Rubin & Parrish, 2007). Moreover, those studies by and large are not assessing the outcome of already supported ESTs when they are implemented in real-world agency settings. So, please don’t get me wrong. I am NOT recommending that journals promote the submission of outcome studies with limited internal validity in general—just those that assess the outcomes of already supported ESTs when they are implemented in real-world agency settings.

I would also counter that generalizable knowledge can be advanced inductively. As I noted earlier, the published outcome studies that I am recommending could provide a basis for using mixed qualitative and quantitative methods to develop hypotheses about the conditions under which ESTs or their common elements can be made to be more feasible to implement and more effective in real-world practice settings.

Will favorable publication prospects be enough of an incentive for social work researchers? Maybe not—especially in those Research I universities that are putting so much pressure on faculty members to obtain lucrative research grants. But maybe it will be a sufficient incentive in those institutions where such grants are NOT such an important factor for faculty advancement. And perhaps even in institutions where they are an important factor faculty can educate administrators about the value of such studies and advocate that they be given more weight in tenure and promotion criteria—not only in terms of their value to the knowledge base but also to their value as a service to community agencies.

Again, don’t get me wrong; I am not disparaging the notion of encouraging faculty to obtain major research funding and rewarding them for it. Indeed, earlier I mentioned that producing well-funded research supplying the empirical support for various research-based treatments helps pave the way for more practitioners to be guided by research. I am merely suggesting that perhaps obtaining major funding should not be the only route to tenure and promotion and that the kinds of studies that I am recommending—if done well—should be encouraged and rewarded as well.

Consider, for example, the following hypothetical scenario. An administrator in a relatively small agency serving primarily traumatized Latino youths invites a junior social work faculty member seeking tenure in a Research I university to conduct an outcome study in the agency to assess the extent to which an adapted version of TFCBT is achieving comparable outcomes in that agency as in the various RCTs that have supported it with non-Latino youths. The agency can offer a sample size of at most 40 for the study. Agency procedures—as well as administrator and practitioner demands—make an RCT infeasible and require other less than pristine methodological attributes. Despite the inescapable methodological limitations, however, it is possible to design a study that would have enough value to make its tentative findings somewhat credible and useful to the agency.

If the faculty member agrees to do the study only if they get a federal grant, the chances are slim that the study will get done. One reason is that the researcher would probably be aware of the difficulty of getting a federal grant approved in a setting that would require having a small sample size and other less than pristine methodological attributes. Another reason is that the time required to write and submit the grant proposal and probably then rewrite and resubmit it might not be acceptable to the administrator. Moreover, even were the study to obtain funding, by the time the researcher would be able to implement the study the administrator may have left the agency or may have moved on to some other priority. Who can blame the junior faculty member for not wanting to invest so much in a potentially futile pursuit?

If instead of pursuing a big grant, the researcher agreed to do the study under limited methodological conditions that were acceptable to the agency, they could help bridge the gap between research and practice in two ways: (1) By doing research of immediate value to the agency and (2) by publishing the study and thus contributing to an accumulating literature identifying the conditions under which adaptations of TFCBT are and are not effective with Latino youth.

Although I recognize the value of social work faculty obtaining major research funding, I want to pose the following provocative question about it: When such funding becomes the main or only route to promotion or tenure, to what extent might pressure to obtain it impede alternative efforts to bridge the gap between practice and research? That is, are there some faculty researchers who are less successful in obtaining major funding now investing years of work submitting and then resubmitting ambitious grants instead of pursuing the kinds of studies that I am recommending?

For example, an assistant professor in a Research 1 University recently commented in an e-mail to me as follows:

I know personally I’ve been steered away from research focusing on social service agency priorities/needs by some mentors because I’ve been told it’s not federally fundable and any results I could “only” hope to publish in social work journals.… We’ve had special efforts to train social workers to be NIH savvy, or build our research infrastructure, but we are being pushed to compete in arenas generally dominated by other disciplines.

In light of this phenomenon, I wonder if we can better bridge the gap by giving faculty researchers another option to pursue promotion and tenure, such as by conducting studies that help practitioners and administrators assess whether an adopted EST is as effective in their setting as they hoped it would be and what factors helped improve or impede its implementation and outcome.

Don’t get me wrong. As I have already stated, pursuing major funding is a good thing—especially if one succeeds in getting funded, as opposed to devoting years toward developing and rewriting and resubmitting grant applications that often never get funded. I’m just suggesting that the kinds of studies that I am recommending for bridging the gap should be equally important in the pursuit of tenure and promotion. I should also clarify that I am not recommending program evaluation studies in general. I am only recommending studies that assess the effectiveness of ESTs that are applied in real-world practice settings.

Conclusion

In conclusion, Figure 4 lists the key recommendations that I have made in this address. What are the chances that my provocative recommendations for bridging the gap will be implemented in the future? It’s hard to say, after all, as Yogi Berra astutely noted, it’s hard to make predictions, especially about the future. However, I hope that they have at least provided some useful ideas for discussion over the next two days.

OPEN IN VIEWER

Figure 4.

Summary of main recommendations.