Abstract
This article describes the experiences of an agency administrator who developed a meaningful and effective collaboration with university researchers to address the needs of her client population. The initial agency–university collaboration process and its benefits are described as well as the efforts required and challenges faced when adopting and implementing motivational interviewing as a primary approach for case management services in a large, urban homeless services program. Finally, additional lessons learned from an administrator’s perspective are provided.
Keywords
One promising strategy to bridge the research–practice gap is to increase collaboration between academic researchers and local community social service practitioners. While there are multiple challenges and considerations in achieving feasible and mutually beneficial agency–university collaboration (Burgio, 2010; Israel, Schulz, Parker, & Becker, 1998; Kerner, Rimer, & Emmons, 2005), agencies that aspire to implement empirically supported interventions or programs may find this avenue quite fruitful (Dulmus & Cristalli, 2012; Miller, Krusky, Franzen, Cochran, & Zimmerman, 2012). Academic faculty are often looking for opportunities to collaborate with administrators who truly value research and are flexible and open to incorporating new programs and evaluation procedures, while agencies can benefit by increasing the efficiency and effectiveness of their services, subsequently enhancing their funding portfolio. Similarly, academic faculty are able to provide many supports which agency administrators and practitioners are not able to provide support due to lack of time, expertise, or resources. For example, academic researchers are often able to assist with comprehensive literature reviews, design evaluation or research studies, and provide training to student researchers to collect data, analyze the data, and write and disseminate the results in reports or the professional literature.
While an agency–community partnership may not be feasible for every organization or agency, given the sheer ratio of faculty to agencies in most localities, successful agency–university partnerships often yield many benefits including satisfaction on both sides knowing that a program or intervention is truly helping clients (the goal of both researchers and practitioners) and capitalization on their collaboration as a way to leverage both effective community services and research funding.
As Allen Rubin suggested in his keynote address (2015), one major barrier the academy must address in order to increase university–agency collaboration and to bridge the research–practice gap is that of better incentivizing newer social work faculty to allocate limited time on the tenure clock to engage in research that may be valuable for the purpose of bridging the research–practice gap but not necessarily fundable. As research funding is a large consideration for tenure at many of the larger, research-focused universities, newer faculty members must examine the risks of these collaborations, especially if the possibility of future funding within a particularly small agency is very low. This is an issue that deserves broader discussion. However, there are many tenured faculty (and perhaps untenured) who have engaged in fruitful university–agency collaborations that have led to improved services, integration of research in real practice settings, and mutually beneficial service or research funding. This article provides the story of one of these successful collaborations—one that started with the lead author, an administrator at SEARCH Homeless Services in Houston, TX, approaching a widely known researcher named Carlo DiClemente.
This article will provide a chronological account of this story using concepts from the National Implementation Research Network’s Implementation Frameworks (Bertram, Blase, & Fixsen, 2015; Metz et al., 2015), beginning with a description of SEARCH homeless prior to this collaboration and ending with a description of how motivational interviewing (MI), an empirically supported intervention, was adopted and implemented. While the implementation frameworks, and the related stages of implementation, did not exist at the time that these events took place, they do provide a helpful context for organizing the story. This story is told in the first person, as it is the lead author’s story.
In the Beginning
In the mid-90s, SEARCH Homeless Services was a fairly young agency, having only provided services to persons who were homeless for about 7 years. Services included a low-demand drop-in center and street outreach, transitional housing, general equivalency diploma (GED) preparation, job training, employment services, and child care. More than half of those seeking services were experiencing problems related to alcohol or drugs, primarily crack cocaine. SEARCH, like many other homeless agencies in Houston at the time, regularly drug tested clients as a condition of program participation. If a client tested positive for drugs, they were given the following three options: (a) go to treatment; (b) attend and document ninety 12-step meetings in 90 days; and (c) be kicked out of the agency’s programs. As such, the climate was one of mistrust. The staff did not trust the clients to tell them the truth. The clients felt they could not be honest with staff or they might get kicked out of the valuable, needed services, such as housing, GED classes, or employment services.
Program staff was well aware that clients were using substances, as they would daily observe evidence of clients coming to class high, including bloodshot or glassy eyes, jittery body movements, difficulty concentrating, and so on. The 12-step meetings were held daily on-site to reduce substance use during group, yet people continued to use drugs. On the plus side, clients continued to seek ways to improve their lives through the services offered, even while continuing to use. However, as an agency, we were not successful in establishing a helpful therapeutic relationship with them or helping them with their substance abuse and dependence. It was out of desperation that we turned to an expert outside of the agency.
Blind Trust
I had known and been somewhat familiar with Carlo DiClemente’s research with substance use, especially since he was developing something radically different from what was being offered in the treatment programs and 12-step groups. He was a researcher, university professor, and well regarded in the community. In 1995, I approached him about the struggles we were having in helping people who were homeless and using substances. He suggested that I meet with his research team at the University of Houston (UH). We began meeting, and he promptly took a position at the University of Maryland–Baltimore County. That could have been the end of the collaboration, but Dr. Mary Velasquez and the UH team were willing to continue with this project.
Exploration
Almost 20 years later, one combs the research literature before deciding which empirically supported practice(s) to implement. In the mid-90s, that was not the path our agency took, nor did it even occur to us to do so. Consulting with experts was our route. Essentially, we trusted Drs. DiClemente’s and Velasquez’s knowledge of the substance use field and regarded them as expert consultants. As we moved into the adoption process, staff became involved in a step-by-step, very slow learning and implementation process. This process was not well thought out, nor intentionally designed, but developed more organically as the sometimes loose and more formal collaboration continued year after year. The changes to the agency were not necessarily welcome by all program staff. Some staff had personally gotten sober through alcoholics anonymous or narcotics anonymous and were quite unhappy with the direction in which the organization had taken in the past. In contrast, other staff and administrators, including the licensed chemical-dependency counselor, were onboard with the changes. We began slowly changing from a confrontational, punitive approach to one that made more sense to most of the staff. A large part of this was our collaboration with the university research team.
Our first project with the Dr. Velasquez and the UH team was the implementation of a survey of participants in our longer term programs, such as housing, adult education, and employment. The survey not only gathered information about the perceived relationship that clients had with staff but also planted seeds about the possibility of a different kind of relationship devoid of negative consequences from speaking honestly about current substance use with program staff.
The second step in this collaboration focused on individuals accessing the low-demand services provided in the drop-in center or resource center. Substance use, psychological distress, and readiness to change were assessed in 100 participants using standardized questionnaires. We identified the following high proportions of high-risk substance use: (a) 80% reported alcohol use, with 65% drinking at high-risk levels and (b) 60% had a history of drug use, with 82% at high-risk levels (Velasquez, Crouch, von Sternberg, & Grosdanis, 2000). While the majority of respondents felt that they drank and/or used drugs “too much,” most were not ready to change this high-risk behavior. Our collaboration with the UH research team led us to understand and apply the Transtheoretical Model (TTM) of intentional behavior change to recognize where clients were in the process of change and to guide our program (Prochaska, DiClemente, & Norcross, 1992). The TTM is an integrative biopsychosocial model that conceptualizes the process of how people change their behavior. The model is often referred to as the stages of change. However, application of the TTM goes beyond assessing the stage of change. Specific experiential or behavioral processes are associated with each stage and are the drivers that help one progress through the various stages. Based on this model, most of our clients were in the precontemplation or contemplation stages of change—not ready or getting ready to change. Previous staff demands that clients go to treatment or 12 steps were possibly useful for someone in action, but not for the majority of those in earlier stages of change for their substance use. Consistent with the tenets of the TTM, we selected and adopted MI as an empirically supported intervention that was, at the time, being compared to other empirically supported treatments in a large National Institute on Alcohol Abuse and Alcoholism treatment matching study by DiClemente and colleagues (Project MATCH Research Group, 1993).
Installation
In the early years, we focused on how adoption of these models could change the organizational culture and the ways in which staff actually interacted with clients. We began with a 1-day workshop in both the TTM and MI with Dr. Velasquez. Together we had already begun, in the late 1980s, developing workbooks, organized by the TTM and stage of change, to guide staff in their individual case-management sessions and substance use groups with clients. There were a number of processes within each stage that, theoretically, the client needed to work through in order to be ready to move into the next stage of change. The workbook provided, at a minimum, one session or exercise per process of change. The early stages focused on experiential processes and primarily utilized MI strategies. The later stages, focusing on behavioral processes, used many cognitive behavioral strategies. After developing the workbooks over several years, we realized the content might be useful to others in the substance use treatment field. We contacted some publishers and Guilford Press was very enthusiastic about publishing a treatment manual that was both science based and practical. In fact, they asked us not to contact any other publishers as they were certain of its contribution to the field. Following a revision, Guilford published Group Treatment for Substance Abuse: A Stages of Change Therapy Manual (Velasquez, Gaddy-Maurer, Crouch, & DiClemente, 2001).
Implementation
As an administrator, my intent was for a line staff caseworker or program manager to become the champion that would drive the implementation effort forward at SEARCH. Unfortunately, that never happened and money was never dedicated to training. The learning process and implementation moved forward by fits and starts.
Training
After Dr. Velasquez’s initial training and workbook development, we used a blended approach for many years. We purchased books on MI and started a book club. Staff watched the MI Professional Video Training Series in small groups—slowly, dissecting the sessions, trying to understand what was actually happening in MI counseling sessions.
Trying to get some traction, we went out on a limb and wrote a grant to a large state foundation, proposing to use the TTM and MI in conjunction with wraparound services in permanent supportive housing to persons who were disabled and homeless. We received this funding, and this 3-year project funded intermediate and advanced training for the project staff with some limited coaching. The outcomes were significant in several areas and encouraged SEARCH to focus more resources on developing skills in the stages of change and MI with staff.
Over time, it became clear that no other champion was going to show up within our agency and that I would have to take on the role more fully. This meant (a) becoming knowledgeable and competent in both models and (b) beginning to train and coach. In 2008, I began providing MI training in-house to all SEARCH staff with a Licensed Chemical Dependency Counselor in the agency who had quite a bit of training and skill in MI. We used Substance Abuse and Mental Health Services Administration (SAMHSA)’s free TIP 35 Enhancing Motivation for Change in Substance Abuse Treatment, which included a manual and an accompanying in-service training curriculum. We also used DiClemente’s 2-hr video of the TTM (2004) as a part of the training. All program staff were required to read the TIP manual and view the TTM video prior to attending MI training. During this grant-funded period, I began using MI in sessions with SEARCH clients and received coaching in MI. I then applied to attend the (highly competitive) Motivational Interviewing Network of Trainers’ train-the-trainer (TT) workshop and was accepted. This grant had given us the money to build an infrastructure to deliver MI within our agency and monitor its fidelity.
Since this time, we continued providing in-house skill building workshops once or twice a year to all new SEARCH program staff and those in need of a refresher. We have also opened up these trainings to other social service agencies in the community and started ongoing monthly Brown Bags where we view segments from a variety of MI training videos and follow up with discussions. Additionally, some staff receive coaching, if the public grant that covers their salary will pay for the training.
Feedback and Coaching
Traditional training in the TTM and MI can help improve staff attitude, knowledge, and confidence (Walters, Matson, Baer, & Ziedonis, 2005), but does not adequately equip providers to deliver evidence-based practices (Carroll, Martino, & Rounsaville, 2010; Martino, 2010; Miller, Yahne, Moyers, Martinez, & Pirritano, 2004). There are many components to the knowledge base and skills needed to deliver both MI and the TTM competently, which have more recently added supervision to typical MI workshops using expert-led and TT strategies instead of a self-study approach (Martino et al., 2011). Miller, Yahne, Moyers, Martinez, and Pirritano (2004) found that counselors who received mail feedback, phone-based coaching, or both after attending a workshop on MI gained more proficiency than others who attended a workshop with no supervisory follow-up. Notably, the counselors who received the most intensive level of supervisory input—both feedback and coaching—had clients who exhibited significantly larger improvements in motivation for change. It is important to note that practitioner self-evaluation has been found to be misleading, with many practitioners overestimating their effectiveness (Martino, 2010). As such, feedback and coaching, as well as outcome assessment, are essential components to improve social services.
Given these new developments, in 2009 we capitalized on an opportunity to further develop our training infrastructure through a 5-year SAMHSA Services in Supportive Housing grant. This grant required the use of “evidence-based practices” to fidelity and supported the use of coaching and booster sessions for our case managers in addition to the initial 2-day small group staff training. Training workshops focused on the 12 steps to learning and skill building for MI, beginning with MI spirit and Open questions, affirmation, reflective listening, and summary reflections (OARS)—both based on the client-centered therapy of Carl Rogers. Once this foundational competency is met, there is a “conceptual leap” that the staff has to make in order to strategically craft responses to the client’s statements, integrating all of these various theories, in order to help people be successful in their change efforts.
Staff performance on skill in utilization and measurements of fidelity of the TTM and MI was primarily evaluated through the use of audio recordings of case management or substance use counseling sessions with residents in the permanent housing settings. Staff were asked to upload two sessions per month of digital recordings of their client interviews (with the client’s approval) to a secure SharePoint website. As the expert trainer, I listened for a range of strategies, beginning with identification of a target behavior and assessing the stage of change. I used the Motivational Interviewing Treatment Integrity (MITI) skill coding system and rated staff on their frequency of use and proficiency with MI as well as the infusion of other counseling strategies (Moyers, Martin, Manuel, Hendrickson, & Miller, 2005; Waltz, Addis, Koerner, & Jacobson, 1993). We used the MITI coding instrument, primarily because we were familiar with it through participation in a research study. It is a useful tool for coding some important aspects of MI and can serve as a useful mechanism for providing specific feedback to learners. We also used it because the SAMHSA grant required information on how we would ensure fidelity to the selected evidence-based intervention. As with many standardized instruments, users need training in coding for reliability purposes. I initially attended a 1-day coding training in 2009 and then our agency provided a 2-day training in coding and coaching in 2012.
Coaching and feedback typically occurred over the telephone. Learners were encouraged to listen to their session prior to participating in this feedback session and were sometimes asked to type a segment in transcript format for more in-depth analysis. Each coaching session begins by identifying what the learner is doing well and the improvements that have been made since the last session. The monthly 1½ hr booster sessions were another opportunity where we used a blended learning approach to improve practitioner competency. These staff booster sessions included the following activities: (a) participating in real role plays; (b) reading transcripts out loud and taking on the role of the expert practitioner; (c) studying transcripts to get “inside the mind of” an expert MI practitioner; (d) reading and studying sections of text as a small group; and (e) listening to their own recordings and focusing on a particular aspect/component of MI.
In order to do this effectively, staff are expected to be able to be knowledgeable and skillful in certain aspects of a wide range of theories and models, including Carl Rogers’ Person-Centered Approach and Client Centered Therapy, Deci and Ryan’s Self-Determination Theory, Albert Bandura’s Self-Efficacy Theory, Daryl Bem’s Self-Perception Theory, Janis and Mann’s Decisional Balance, Cognitive-Behavioral Therapy, and Relapse Prevention.
Despite these efforts, we found it difficult to motivate some of our practitioners to participate in the training and coaching process. Interestingly, I have recently read that I am not alone in this challenge. For example, Martino (2010, p. 32) writes, “Despite these promising findings [of aforementioned research on training], it can be difficult to engage counselors in intensive supervision. Many trainers have noted that counselors are reluctant to provide their supervisors with recordings of their client sessions or to participate in session reviews even when they are offered at no cost.” In addition, at the International Conference on Motivational Interviewing Plenary Presentation in 2012, a description of training efforts provided by Dr. Theresa Moyers summed up our experience at SEARCH quite well, “It is much easier to get people who are using cocaine to give you data at follow-up than to get work samples from therapists.” Even after staff are made aware of the voluminous research demonstrating the efficacy of MI, they still drag their feet and do not turn in their fidelity recordings. We believed one way to get around this hesitancy would be to address the expectations up front during the interviewing and hiring process and to only hire staff who were open to participating in fidelity and outcome evaluation. Everyone we have interviewed, without exception, expressed excitement—proclaiming that they want to learn MI. They are very excited until they find out you really, really mean it. We have found that the primary barriers with practitioners include (a) being passive in their continuing education—not reading or seeking out materials that will help them do their job better; (b) actively avoiding recording their sessions and providing the excuse that none of their clients are willing to have their session recorded; and (c) although less frequent now, not buying into the client-centered approach. For example, one counselor believed that not confronting a client on his or her substance use was condoning bad behavior.
There were other barriers beyond practitioner participation in fidelity efforts that should be noted. Our agency has decided it is too costly to pay for coaching for all 40 case managers. Ideally, coaching would begin immediately after initial training and be ongoing until competence was reached and then decrease to once a month for another period of time to account for drift. As some staff become competent, they could take on a coaching or training role in the organization to help with sustainability. Over the past 4 years, with limited coaching, six people have become “in-house” coaches, three have left the agency, and three remain. There are also issues with line staff turnover—training them and then having them leave and having to start the process over again.
Evaluating Performance
In addition to measuring fidelity, we also used several measures to assess client outcomes including the Medical Outcomes Study 36-item short form health survey for improved health status, decreased recidivism, and housing retention. In addition, we also strive to improve our interactions with clients and assess this with client satisfaction surveys over time. Of course, there are challenges in gathering these data, especially with a population that often has long histories of substance abuse or dependence, serious mental illness, chronic physical health conditions, traumatic brain injury, as well as resignation to their situation. However, we continue to do our best to gather the data we can and to improve our access to such data.
Other Lessons Learned
While we have focused mainly on the training challenges related to fidelity, there are multiple organizational factors that have influenced and promoted staff learning and performance that require mention, which are are listed subsequently.
Program managers can help or hinder staff’s effort to learn
As managers develop skill, they begin to take ownership of their staff learning the models and subsequently use the language, have a deeper understanding of how people change, and understand the strategic use of MI. This knowledge gets modeled and trickles down to line staff, integrates into staff assessment forms, and provides the structure for clinical staffing conversations.
Lack of consistent funding for coaching significantly hinders learning
Most staff in our agency have received at least some minimal coaching. But with most, it is stop and go as we scramble to cobble together grant funding that will allow for training as an allowable cost. As such, it would be useful for more funding sources to allow for training as an allowable cost so that this infrastructure can be built and maintained.
Learning an empirically supported treatment takes time
Factors that help or hinder counselors’ efforts to implement have been largely related to the length of time it takes to develop competence in MI. For skilled clinicians who are very motivated, it can take 6 months to develop competence. But for staff who have not been trained in counseling skills and are not particularly motivated, it can take several years, if it is ever accomplished. As such, some staff begin to feel discouraged and resign when they do not seem to be making real improvement after several months of coaching. Also, some staff even after developing skills, it is easy for them to drift back into old patterns, so fidelity even needs to be monitored—even if less frequently—among staff who have demonstrated competency.
Justifying the connection between fidelity and outcomes
Also, because of varying levels of knowledge and skill across staff within each program, it can be difficult to assess the degree to which program outcomes become the result of a particular empirically supported intervention. We have not yet fully assessed the cost–benefit of providing MI with fidelity, although we realize that its implementation was essential to receiving SAMHSA grant funding.
Strong administrative support in the agency is critical
SEARCH has provided dedicated administrative support for the use of the TTM and MI over the years. We really stood behind our decision to use these models and this was reflected in the changing of our organizational values to include phrases that describe us as a “learning organization” and “client centered.” Also, our performance reviews have clear expectations as to what types of behaviors are consistent with these values, and these values are focused on quarterly at all staff meetings. We have also had strong buy-in from our CEO and Human Resources Director, both of whom participated in the initial 2-day training, read journal articles, and have adopted the language of the TTM and MI.
Have an agency champion
Initially, as an administrator, I was waiting for someone to come along and take over what was initially started with regard to the TTM and MI at SEARCH. After a while, I realized this was not going to happen. I had to be that person. This led to me put on my clinical hat and learn MI, becoming proficient both in practice and in training.
Our commitment to these approaches has improved the agency’s reputation in the community; we are regarded by our peer organizations as being very professional and competent. This leads to us being asked to join the table of many community efforts that improve the lives of our clients and allow us to apply for future funding. It also allows us to attract high-quality staff—some of whom would not initially necessarily consider working in a homeless service agency but want to learn MI and learn it well. I believe that our initial work with Carlo DiClemente and Mary Velasquez’ research teams led to some important changes within our agency, including the decision to adopt the TTM and MI, that have been sustained through future funding. I feel fortunate to have continued these collaborations with Dr. Velasquez and her research team, and I look forward to future agency–university collaborations that are mutually beneficial.
Footnotes
Authors’ Note
This article was previously presented at the conference on Bridging the Research and Practice gap: A Symposium on Critical Considerations, Successes and Emerging Ideas, sponsored by the University of Houston Graduate College of Social Work, Houston, TX, April 5 and 6, 2013. This article was invited and accepted by the Guest Editor of this special issue, Danielle E. Parrish, PhD, and further reviewed by the Editor, Bruce Thyer, PhD Further correspondence should be directed to Cathy Crouch at
Acknowledgments
We would like to thank Dr. Mary Velasquez, Centennial Professor in Leadership for Community, Professional and Corporate Excellence; Director, Health Behavioral Research and Training Institute at the University of Texas at Austin School of Social Work, for reviewing and suggesting useful edits of earlier drafts of this manuscript. We would like to thank Dr. Mary Velasquez for her feedback on earlier drafts.
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors received no financial support for the research, authorship, and/or publication of this article.