Performance-Based Practice: Clinical Dashboards for Addiction Treatment Retention

Abstract

Purpose:

Substance use disorders (SUDs) are on the rise, particularly concerning opioids, and existing services are still not adequately meeting treatment needs. Most treatment dropouts occur within the first few weeks of treatment, indicating health improvement is likely not occurring. In this study, a clinical dashboard tool has shown promising utility with increasing retention in SUD treatment.

Method:

This pilot study was designed to measure the effects of implementing a performance-based practice clinical dashboard tool and having a process for monitoring patients’ recovery in a Midwestern SUD treatment center.

Results:

The results demonstrated the efficacy of a clinical dashboard tool that complements therapists’ monitoring and measuring of their patients’ performance.

Discussion:

Results support a building knowledge base about the impact of real-time, evidence-based tools on the retention and support of clients seeking SUD treatment.

Keywords

clinical dashboards client retention substance use disorders

Addiction in the United States

An opioid overdose crisis exists in the United States, and the illness of opioid use disorder has become an epidemic in the United States with more than 200,000 cases reported annually. About 3.8 million people aged 12 and above reported having misused a prescription pain medication in 2015. Of these, within 1 year, 2.0 million people acquired a disorder of opioid use or dependence (in accordance with disorders listed in the Diagnostic and Statistical Manual of Mental Disorders, fourth edition). In 2015, over 5 million Americans (1.9% of persons in the United States aged 12 or above) were estimated to have used heroin at some point in their lives (National Academies of Science, 2019). Consequently, heroin-related overdose death is also on the rise with nearly 13,000 reported in 2015, and more than 72,000 Americans reportedly died from a drug overdose in 2017 (Scholl et al., 2018).

Substance use disorder (SUD)—broadly applied to the use of opioid, alcohol, tobacco, cannabis, hallucinogens, inhalants, sedatives, and stimulants—is a major threat to the nation. Nearly one third of all American adults have SUD at some point in their lives. Tens of thousands die because of SUD each year, and the associated health care costs in the United States soars to over US$200 billion annually (Bouchery et al., 2011; Davenport et al., 2019). Currently, a great need exists for SUD treatment in the United States (Johnson et al., 2020). Unfortunately, over 70% of people who seek professional help do not receive an adequate dose of treatment, and they drop out of services within a few weeks after admission (Patterson Silver Wolf et al., 2019).

Retention in SUD Treatment

To meet the challenges of SUD, which is a chronic illness, treatment should be based on the best, most up-to-date science. Treating individuals who have SUDs requires a planned, measurable sequence of events that lead to the prevention or reduction of harmful use and the improvement of biopsychosocial functioning (Walker, 2009). A wide array of treatment designs and many treatment durations are used to accomplish this difficult but important task (Kleber et al., 2007).

Regardless of any treatment configuration, treatment retention and completion have been continually shown to significantly improve health and wellness outcomes (Dalsbø et al., 2010). Regrettably, patients who have a SUD drop out much sooner in the treatment process compared to general psychotherapy patients (Howard et al., 1986; Szafranski et al., 2019). With about 50% of dropouts occurring in the first 2 weeks of SUD treatment (Patterson Silver Wolf, 2008; Stark, 1992), and with only about 20% remaining until a successful discharge (Patterson Silver Wolf et al., 2019), patients are receiving insufficient exposure to treatment and are not recovering from their illness.

Our historical literature has established that patients leaving SUD treatment before the 3-month mark show no significant health improvements (Brorson et al., 2013; Eaton, 2004; Hawkins et al., 2008). Focusing treatment resources toward—and specifically targeting—patient retention has critical, wide-ranging ramifications. Without a planned, purposeful effort to address treatment retention rates, SUD treatment agencies will continue to fall short of appropriately treating this chronic illness, and individuals will continue to suffer from their untreated illness.

How Clinical Dashboards Can Improve Patient Treatment Retention

Most of the 1.6 million professional therapists in the United States who offer relief services in SUD treatment facilities rely on their judgment and intuition rather than basing their decisions on scientific data and substantive evidence (Patterson Silver Wolf et al., 2018; Ramsey et al., 2017). Clinical practices without any real-time patient performance outcome data is intuition-based treatment (Lambert, 2010). Further, clinical practitioners are not well equipped to readily access patient performance data. Prior work has demonstrated that formal instruments are too lengthy or take too much time to conduct (Duncan & Miller, 2000). Even when therapists do collect useful patient health data and enter the data into electronic health record (EHR) systems, these vital data largely remain inaccessible to clinical practitioners as there are little technology resources to help clinicians extract these data leaving them in EHR data coffins. Ultimately, therapists are not likely to use any instrument or data-tracking system that takes longer than 5 min to complete, score, and interpret (Lambert, 2010; S. D. Miller et al., 2006). To address these issues, technologies such as clinical dashboards can be implemented to retrieve and display key patient health indicators quickly, in a visual format that is easy to digest, potentially improving clinical evaluation and decision making.

Dashboards are tools that display graphs at a glance in a visual pleasing and usable format. Dashboards originated in the business sector to integrate and extract key performance indicators (Pauwels et al., 2009). Research has demonstrated significant improvements in both retention and client outcomes when clinicians receive formal, real-time, real-world feedback from their clients (S. D. Miller et al., 2006; Reese et al., 2009). Clients whose therapists had access to data on their treatment progression were less likely to deteriorate, more likely to stay longer in treatment, and were two times as likely to achieve clinically significant changes (Whipple et al., 2003). Technology such as dashboards can help provide this necessary support and training for clinicians (Quanbeck et al., 2014) thus improving client retention and completion outcomes.

In a recent review conducted on the impact of dashboard use, researchers found an association with improved care processes and outcomes for patients (Dowding et al., 2015). In hospital-based care, higher levels of information automation are associated with lower mortality rates, lower rates of complications, and lower overall costs (Amarasingham et al., 2009). For every 10-point increase in information automation, fatal hospitalizations decreased by 15% (US$56.3 billion), risk of complications fell by 16% (US$60.1 billion), and savings on hospital admissions grew by US$116.4 billion (Pauwels et al., 2009). While hospital-based care research demonstrates significant benefits of technology, research investigating addiction treatment retention and completion outcomes with clinical dashboards is still emerging (Patterson Silver Wolf, 2018; Voelker, 2019). This study seeks to address this gap in the literature.

Purpose

The evolution of clinical dashboards has produced positive clinical outcomes; however, our current knowledge remains incomplete because of their limited use outside of medical/hospital environments. Clinical dashboards can provide pertinent patient information in a timely manner and improve decision making and quality of care (Amarasingham et al., 2009). The key to having effective dashboards is to populate them with essential, real-time, real-world recovery data. These data can result in actionable interventions to address the problem of high treatment dropout rates.

In this pilot project, we propose to explore the potential effects of implementing a clinical dashboard tool and the effect of implementing a process for monitoring patients’ recovery processes on successful SUD treatment completion outcomes. We propose to design and develop cutting-edge technologies that will gather real-world, in the moment patient information and that will organize and display on a clinical dashboard where therapists can see and make judicious and personalized treatment decisions. We hypothesize that the incorporation of this tool will enable a better understanding of and more effective models of caseload monitoring, thereby improving patient care, treatment retention, and successful completion.

Method

Business and Treatment Collaboration

The pilot study was led by Takoda (2018), a private technology business located in St. Louis, MO. Takoda’s mission is to create SUD technology developed from academic findings and knowledge gained from real-world SUD services. Its eight-member multidisciplinary team consists of academic researchers, behavioral health treatment professionals, software engineers, data scientists, and human-centered designers.

Members also have interdisciplinary expertise. A trained professional programmer has been a frontline SUD therapist before, and two of the team members have experience as patients themselves, receiving treatments in the past and sustaining long-term recovery from SUDs. Takoda has developed several technology tools for SUD treatment, including the stand-alone clinical dashboard tool tested for this project.

Pilot Testing Site

A large, multistate addiction program served as the site of the project, with this particular study taking place in St. Louis, MO. This specific location serves over 1,200 patients annually in its outpatient program. Frontline therapists are responsible for the treatment plans and overall care of patients and maintain a caseload of approximately 75 patients. Along with standard individual and group therapy, the treatment facility also provides ancillary services including peer and community support specialists, employment supports, and medications that assist treatment.

Therapist Training and Onboarding

All therapists working at one SUD outpatient treatment location were introduced and trained to use the clinical dashboard tool during an hour-long training session. During the first 30 min, Takoda staff led therapists through a workshop to qualitatively discuss how they currently monitored patient recovery progress. As therapist caseloads are large, the team asked therapists how they monitored the basic treatment performance of their patients. Therapists were asked simple but vital questions pertaining to caseload profile and treatment performance. The questions are as follows: What is the percentage of successful treatment completion for your caseload and/or this clinic? Who does better (i.e., completes with staff approval) on your caseload by age, race, and gender? What percentage of patients on your caseload stopped attending after 5 days? and What percentage of patients on your current caseload are making quantifiable progress toward meeting their treatment goals?

Therapists gave a variety of answers to questions, with no therapist able to show any real-time data on the performance of their current or closed patient caseloads. The purpose of the exercise was to expose the lack of any data guiding clinical services and reveal that therapist use their own intuitions in measuring patient’s treatment progression through outpatient treatment.

During the second 30 min, Takoda staff demonstrated how to enter patients into the tool, log patient sessions, edit patient details, and analyze the performance displays. Takoda emphasized that this software was under development and that therapists should readily submit feedback and report issues they counter. The feedback would be utilized to shape the growth of the technology.

Regular Dine and Dashboard Meetings

Over the next 3 months, Takoda staff held hour-long, bimonthly meetings on-site at the treatment facility with all therapists. At the beginning of every meeting, to inform product development and technology support, therapists were asked to fill out an evaluation of system usability, willingness to use, and perceived level of usefulness. The results were used by the lead designer to measure receptiveness to new features or modifications introduced to the system since the previous meeting.

Additionally, to show therapists how well their overall staff adopted new technologies, Takoda staff prepared usage graphs to display statistics. The staff members were anonymized with numerical IDs so as not to cause negative associations with therapists who were not readily adopting the platform. The purpose of sharing these data was to demonstrate the therapists’ overall willingness to engage with the dashboard and to encourage usage by giving therapists a sense of progression toward a goal of entering the majority of their patient caseload and patient sessions into the platform.

Finally, Takoda staff conducted unique usability workshop activities to determine what new ideas therapists had for the platform and what pain points they faced in their regular workflows that the dashboard could help alleviate. Because of Takoda’s limited capacity to develop many of the more transformative ideas, they were archived for later discussion.

Key Indicators Displayed on Clinical Dashboard

The clinical dashboard is based on mixed-methods data collected from frontline therapists, leaders, and patients. The development team consists of the Takoda team and an academic researcher. Every therapist was given a dashboard that displayed their own caseload. The main goal of the tool was to replace therapist’s intuition with real, quantifiable data. The dashboard has four screens that simultaneously display key indicators of patient data. The first screen shows active caseload patient demographics. Because therapists are unable to see, at a glance, basic patient data, this first screen allowed therapists to monitor their caseload by race, age, gender, and other treatment performance outcomes such as number of days in treatment, last sessions attended, and other graphs to use when measuring treatment plan performance.

The second screen displays metrics from a set of questions asked in regular one-on-one counseling sessions. The questions are tied to Dimensions That Support a Life in Recovery by Substance Abuse and Mental Health Services (Strashny, 2013). They are related to the following: health, overcoming or managing one’s disease(s) or symptoms; home, a stable and safe place to live; purpose, meaningful daily activities; and community, relationships, and social networks that provide support, friendship, love, and hope (SAMHSA, 2012). The third screen shows treatment plan information on medication, short- and long-term goals, interventions used, specific services administered, and other supporting services engaged.

The clinical dashboard was designed for therapists to use it to track verbal and behavioral signs that demonstrate patients’ progress, or lack thereof, within the dimensions that support a life in recovery. The graphical representation of the patients’ progress will also be used in direct discussions with patients to mutually decide which dimensions need more effort.

Primary Outcome Measure

We chose to measure successful discharge percentage because it is a reliable quantitative measure and is decided by clinical staff rather than by individual therapists. Successful discharge is defined as “discharged with staff approval,” which means the patient, clinical team, and supervisor concurred that the patient had made significant progress toward recovery and met treatment plan goals. Although the Takoda technology system tracked discharge reasons, this project worked with the clinic’s administration to check Takoda’s data with their EHR, ensuring reliable data for analysis. The clinic’s administration provided deidentified, aggregated data for a 12-month period describing the clinic’s discharge outcomes. Regarding human subject protection, this study met the criteria for exemption as it was not research with human subjects, and all data were deidentified and aggregated to include only the number of successful and unsuccessful discharges each month.

Hypothesis

Hypothesis 1: After implementation of the dashboard, percentages of successful discharges will increase.

Results

The project officially trained and onboarded therapists using the dashboard tool in March 2019. During the project period, the clinic discharged a total of 1,337 patients. In Table 1, the breakdown of monthly discharge outcomes is presented. Discharge data were collected from the EHR system beginning in September 1, 2018, until August 9, 2019. In the first 6 months before implementing the new tool and practice protocol (deemed the control period), the percentage of discharge with staff approval hovered around 11%. We hypothesized that after the implementation of the dashboard, percentages of successful discharges will increase. During the 6 months of dashboard tool usage and new monitoring practices, the successful discharge rate doubled to more than 27% (Figure 1). Based on this increase in successful discharge percentages, we conclude that our hypothesis was supported.

Table 1.
Discharge Overview by Month.

Discharge Month Successful Discharge Count All Other Discharge Grand Total Discharge Successful Discharge Rate (%)

September 2018 6 57 63 9.5

October 2018 12 110 122 9.8

November 2018 11 71 82 13.4

December 2018 14 171 185 7.6

January 2019 19 158 177 10.7

February 2019 18 105 123 14.6

March 2019 15 116 131 11.5

April 2019 24 78 102 23.5

May 2019 24 94 118 20.3

June 2019 19 88 107 17.8

July 2019 26 70 96 27.1

August 2019 7 24 31 22.6

Grand total 195 1,142 1,337 14.6

Note. Successful discharge noted as “patient completed treatment.” Time period is September 1, 2018, through August 9, 2019.

Figure 1.
Discharge with staff approval, percentage over time.

Discussion and Applications to Practice

The goal of this pilot study was to gauge potential impacts of a clinical dashboard tool on SUD outpatient treatment completion rates. The authors hypothesized that therapists would be able to better monitor their patient caseloads, develop improved patient care, and increase successful treatment completion. After the dashboard tool was implemented and regular meetings were held with therapists to discuss clinical protocols to best leverage the new tool, the results demonstrated that the number of patients successfully completing this SUD treatment program steadily increased. During the 6 months before implementing the new tool, the treatment completion success rate averaged 11%. During the 6 months of implementation, the success rate doubled to more than 27%. Future research should apply inferential statistics to evaluate a scaled-up version of this work with an increased sample size and account for spurious effects in order to isolate indicators of dashboard effectiveness.

This pilot study has several limitations. First, the research design had no random sampling. Disadvantages consist of selection bias, the inability to minimize other variables, and low confidence of the intervention’s direct impact on outcomes. Maturation bias may explain some of our results as participants may have learned throughout the multiple dine and dashboard meetings. Similarly, our study is subject to testing bias given the pretest assessment of the dashboards. The positive outcomes from this pilot study cannot be explained beyond the sample of patients admitted to this treatment location. Further, we did not measure whether the technology tool alone or the additional regular dine and dashboard meetings impacted outcomes.

Given the previously mentioned limitations, this simple pilot study does provide some opportunities for further SUD treatment research and practice endeavors. There is an ongoing concern regarding the gap between research and practice (Kohn et al., 2000; Pincus et al., 2007). The primary issue is that when interventions tested outside of the real world of community-based treatment services—in places where caseloads are very small and interventions require multiple weeks to effectively administer (Bradizza et al., 2018; Stasiewicz et al., 2013)—are finally attempted in the real-world treatment system, their required protocols and outcomes fall victim to limited resource capacity (Patterson Silver Wolf, 2015). The National Institute of Drug Abuse (2012) has questioned the worthiness of continuing to develop and test interventions in the face of these treatments not being adopted in real-world practice systems.

This study has exposed the problems associated with having a large patient caseload (around 75 for this study) and the challenges of closely monitoring a population that has an illness that is both chronic and complex and that may have multiple other conditions. With so many evolving and dynamic variables related to SUD treatment and recovery, frontline therapists are relying mostly on their intuitions rather than real-time patient performance data (Patterson Silver Wolf et al., 2019).

During this study’s first dine and dashboard meeting, we asked frontline therapists what their clinic’s average completion success rate was. Their responses hovered around 30%–40%. When further probed how they determined that rate, it was mostly grounded in anecdotal guesses and the myths of the thirds (W. R. Miller et al., 2001). In Miller’s large 2001 national alcoholism study in 2001, the authors reported that about one third of individuals treated for addiction remained clean and sober for the rest of their lives. Another third who received treatment still used drugs or alcohol but not at as high a risk level as they did before treatment. The final third never recovered. While that 20-year-old study focused on sustained recovery and not successful treatment completion, this myth of the thirds lives on inside of today’s treatment systems.

In reality, frontline therapists are unlikely to know their treatment program’s successful completion rates or their own performance outcomes. In this study, therapists reacted with a mixture of disbelief and disappointment when learning their clinic’s actual success rate averaged 11% during the 6 months before implementing the new tool, with some months dipping into single digits.

Future research should pursue a randomized control trial where conditions can be more reliably controlled. Because it can be assumed that clinical therapists working on the frontlines of SUD treatment are motivated by the desire to improve the lives of their patients, it is important to control for these behavior change effects after learning of their patient’s poor treatment completion success rates. Also, reasons for SUD therapists not using new research-supported treatments should be measured in forthcoming investigations (Patterson Silver Wolf & Dulmus, 2012 , 2018; Patterson Silver Wolf et al., 2013 , 2018).

As with trying to implement any new intervention or technology, some therapists will be early adopters, while others will lag behind. In addition, there will be a cohort of therapists who might never be moved to adopt a new intervention (Patterson Silver Wolf et al., 2018). Further, any future research measuring the effects of an intervention on treatment retention and completion will also have to control for patient–therapist matching. In a recent study, it was revealed that certain gender and racial matching led to large differences in treatment completion rates (Patterson Silver Wolf et al., 2019).

With the disruptions of outpatient treatment caused by COVID-19, where services have been transitioned to cyberspace without much planning or forethought, the future of SUD treatment will continue to move toward telehealth. Technology tools that were supposed to assist the clinical process, much like EHR systems, have proven less than beneficial for informing and guiding treatments (Kruse et al., 2016). Not only does the clinical dashboard tool that was investigated in this study monitor patient treatment and recovery performance, but it also measures and monitors the therapist’s performance in relation to success with specific patient profiles (Patterson Silver Wolf, 2018).

With SUD treatment completion being associated with improved health outcomes, it is important to focus on barriers and pathways to patients remaining in treatment until successful completion. This pilot study has demonstrated the initial efficacy of a clinical dashboard tool that assists therapists’ efforts to monitor and measure their patients’ performance. Throughout our outpatient treatment infrastructure where services are guided mostly by therapist’s intuitions, technology tools that can easily replace anecdotal validation with real-time data are crucial. Just like psychotherapy interventions have been empirically investigated on various outcomes, similar investigations are needed for emerging technologies that might impact patient outcomes.

Discharge Month	Successful Discharge Count	All Other Discharge	Grand Total Discharge	Successful Discharge Rate (%)
September 2018	6	57	63	9.5
October 2018	12	110	122	9.8
November 2018	11	71	82	13.4
December 2018	14	171	185	7.6
January 2019	19	158	177	10.7
February 2019	18	105	123	14.6
March 2019	15	116	131	11.5
April 2019	24	78	102	23.5
May 2019	24	94	118	20.3
June 2019	19	88	107	17.8
July 2019	26	70	96	27.1
August 2019	7	24	31	22.6
Grand total	195	1,142	1,337	14.6

Footnotes

Declaration of Conflicting Interests

The authors declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: The first author, Patterson Silver Wolf, is a cofounder, chief clinical officer, and board member and holds shares in Takoda, Inc.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Sara Beeler-Stinn is a NIDA predoctoral fellow T32DA015035 (mPI/Director R. Cunningham-Williams and K.K. Bucholz).

ORCID iD

David A. Patterson Silver Wolf

Sara Beeler-Stinn

References

Amarasingham

Plantinga

Diener-West

Gaskin

D. J.

Powe

N. R.

(2009). Clinical information technologies and inpatient outcomes: A multiple hospital study. Archives of Internal Medicine, 169(2), 108–114. https://doi.org/10.1001/archinternmed.2008.520

Bouchery

E. E.

Harwood

H. J.

Sacks

J. J.

Simon

C. J.

Brewer

R. D.

(2011). Economic costs of excessive alcohol consumption in the U.S., 2006. American Journal of Preventive Medicine, 41(5), 516–524. https://doi.org/10.1016/j.amepre.2011.06.045

Bradizza

C. M.

Brown

W. C.

Ruszczyk

M. U.

Dermen

K. H.

Lucke

J. F.

Stasiewicz

P. R.

(2018). Difficulties in emotion regulation in treatment-seeking alcoholics with and without co-occurring mood and anxiety disorders. Addictive Behaviors, 80, 6–13. https://doi.org/10.1016/j.addbeh.2017.12.033

Brorson

H. H.

Arnevik

E. A.

Rand-Hendriksen

Duckert

(2013). Drop-out from addiction treatment: A systematic review of risk factors. Clinical Psychology Review, 33(8), 1010–1024.

Dalsbø

T. K.

Hammerstrøm

K. T.

Vist

G. E.

Gjermo

Smedslund

Steiro

Høie

(2010). Psychosocial interventions for retention in drug abuse treatment. Cochrane Database of Systematic Reviews. https://doi.org/10.1002/14651858.CD008220

Davenport

Weaver

Caverly

(2019). Economic impact of non-medical opioid use in the United States: Annual estimates and projections for 2015 through 2019. Society of Actuaries. https://www.soa.org/globalassets/assets/files/resources/research-report/2019/econ-impact-non-medical-opioid-use.Pdf

Dowding

Randell

Gardner

Fitzpatrick

Dykes

Favela

Hamer

Whitewood-Moores

Hardiker

Borycki

Currie

(2015). Dashboards for improving patient care: Review of the literature. International Journal of Medical Informatics, 84(2), 87–100. https://doi.org/10.1016/j.ijmedinf.2014.10.001

Duncan

B. L.

Miller

S. D.

(2000). The heroic client: Principles of client-directed, outcome-informed therapy. Jossey-Bass.

Eaton

(2004). Numbers starting treatment for drug misuse increase by 20% over two years. British Medical Journal, 392(7474), 1066.

10.

Hawkins

E. J.

Baer

J. S.

Kivlahan

D. R.

(2008). Concurrent monitoring of psychological distress and satisfaction measures as predictors of addiction treatment retention. Journal of Substance Abuse Treatment, 35, 207–216.

11.

Howard

K. I.

Kopta

M. S.

Krause

M. S.

Orlinsky

D. E.

(1986). The dose–effect relationship in psychotherapy. American Psychologist, 41, 159–164.

12.

Johnson

Rigg

K. K.

Hopkins Eyles

(2020). Receiving addiction treatment in the US: Do patient demographics, drug of choice, or substance use disorder severity matter? International Journal of Drug Policy, 75, 102583. https://doi.org/10.1016/j.drugpo.2019.10.009

13.

Kleber

H. D.

Weiss

R. D.

Anton

R. F.

George

T. P.

Greenfield

S. F.

Kosten

T. R.

O’Brien

C. P.

Rounsaville

B. J.

Strain

E. C.

Ziedonis

D. M.

Hennessy

Connery

H. S.

McIntyre

J. S.

Charles

S. C.

Anzia

D. J.

Cook

I. A.

Finnerty

M. T.

Johnson

B. R.

Nininger

J. E.

,…Steering Committee on Practice Guidelines. (2007). Treatment of patients with substance use disorders, American psychiatric association. The American Journal of Psychiatry, 164(4 Suppl.), 5–123. PMID: 17569411.

14.

Kohn

L. T.

Corrigan

J. M.

Donaldson

M. S.

(Eds.). (2000). To err is human: Building a safer health system. The National Academies Press. https://doi.org/10.17226/9728

15.

Kruse

C. S.

Kristof

Jones

Mitchell

Martinez

(2016). Barriers to electronic health record adoption: A systematic literature review. Journal of Medical Systems, 40(12), 1–7. https://doi.org/10.1007/s10916-016-0628-9

16.

Lambert

M. J.

(2010). Yes, it is time for clinicians to routinely monitor treatment outcome. In Duncan

B. L.

Miller

S. D.

Wampold

B. E.

Hubble

M. A.

(Eds.), The heart and soul of change: Delivering what works in therapy (2nd ed., pp. 239–266). American Psychological Association.

17.

Miller

S. D.

Duncan

B. L.

Brown

Sorrell

Chalk

M. B.

(2006). Using formal client feedback to improve retention and outcome: Making ongoing, real-time assessment feasible. Journal of Brief Therapy, 5(1), 5–22.

18.

Miller

W. R.

Walters

S. T.

Bennett

M. E.

(2001). How effective is alcoholism treatment in the United States? Journal of Studies on Alcohol, 62(2), 211–220. https://doi.org/10.15288/jsa.2001.62.211

19.

National Academies of Science. (2019). Medications for opioid use disorder save lives. Medications for opioid use disorder save lives. https://doi.org/10.17226/25310

20.

National Institute of Drug Abuse. (2012, March). Adoption of NIDA’s evidence-based treatments in real world settings: A national advisory council on drug abuse workgroup report (Vol. 3201). https://www.apa.org/about/gr/science/spin/2012/09/evidence-based-treatments.pdf

21.

Patterson Silver Wolf

D. A.

(2008). Motivational interviewing: Does it increase retention in outpatient treatment? Substance Abuse, 29(1), 17–23.

22.

Patterson Silver Wolf

D. A.

(2015). Factors influencing the implementation of a brief alcohol screening and educational intervention in social settings not specializing in addiction services. Social Work in Health Care, 54(4), 345–364. https://doi.org/10.1080/00981389.2015.1005270

23.

Patterson Silver Wolf

D. A.

(2018). The new social work. Journal of Evidence-Informed Social Work, 15(6), 695–706. https://doi.org/10.1080/23761407.2018.1521321

24.

Patterson Silver Wolf

D. A.

Dulmus

C. N.

(2012). Organizational barriers to adopting an alcohol screening and brief intervention in community-based mental health organizations. Best Practices in Mental Health: An International Journal, 8(1), 16–28. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3953346/

25.

Patterson Silver Wolf

D. A.

Dulmus

C. N.

Maguin

Linn

B. K.

Hales

T. W.

(2019). Therapist–patient demographic profile matching: A movement toward performance-based practice. Research on Social Work Practice, 29(6), 677–683. https://doi.org/10.1177/1049731518783582

26.

Patterson Silver Wolf

D. A.

Maguin

Dulmus

C. N.

Nisbet

B. C.

(2013). Individual worker level attitudes toward empirically supported treatments. Research on Social Work Practice, 23(1), 95–99.

27.

Patterson Silver Wolf

D. A.

van den Berk-Clark

Williams

S. L.

Dulmus

C. N.

(2018). Are therapists likely to use a new empirically supported treatment if required? Journal of Social Work, 18(6), 666–678. https://doi.org/10.1177/1468017317743138

28.

Pauwels

Ambler

Clark

B. H.

LaPointe

Reibstein

Skiera

Wierenga

Wiesel

(2009). Dashboards as a service: Why, what, how, and what research is needed? Journal of Service Research, 12(2), 175–189. https://doi.org/10.1177/1094670509344213

29.

Pincus

H. A.

Page

A. E. K.

Druss

Appelbaum

P. S.

Gottlieb

England

M. J.

(2007). Can psychiatry cross the quality chasm? Improving the quality of health care for mental and substance use conditions. American Journal of Psychiatry, 164(5), 712–719. https://doi.org/10.1176/ajp.2007.164.5.712

30.

Quanbeck

A. R.

Gustafson

D. H.

Marsch

L. A.

McTavish

Brown

R. T.

Mares

M. L.

Johnson

Glass

J. E.

Atwood

A. K.

McDowell

(2014). Integrating addiction treatment into primary care using mobile health technology: Protocol for an implementation research study. Implementation Science, 9(1), 1–11.

31.

Ramsey

A. T.

Baumann

Patterson Silver Wolf

D. A.

Yan

Cooper

Proctor

(2017). The need for data-informed clinical supervision in substance use disorder treatment. Journal of Addictive Diseases, 36(2), 117–126. https://doi.org/10.1080/10550887.2017.1291051

32.

Reese

R. J.

Norsworthy

L. A.

Rowlands

S. R.

(2009). Does a continuous feedback system improve psychotherapy outcome? Psychotherapy: Theory, Research, Practice, Training, 46(4), 418.

33.

Scholl

Seth

Kariisa

Wilson

Baldwin

(2018). Drug and opioid-involved overdose deaths—United States, 2013–2017. Morbidity and Mortality Weekly Report, 67(5152). https://doi.org/10.15585/mmwr.mm6751521e1

34.

Stark

M. J.

(1992). Dropping out of substance abuse treatment: A clinically oriented review. Clinical Psychology Review, 12, 93–116.

35.

Stasiewicz

P. R.

Bradizza

C. M.

Schlauch

R. C.

Coffey

S. F.

Gulliver

S. B.

Gudleski

G. D.

Bole

C. W.

(2013). Affect regulation training (ART) for alcohol use disorders: Development of a novel intervention for negative affect drinkers. Journal of Substance Abuse Treatment, 45(5), 433–443. https://doi.org/10.1016/j.jsat.2013.05.012

36.

Strashny

(2013). Recovery services provided by substance abuse treatment facilities in the United States. In The CBHSQ Report (pp. 1–9). Substance Abuse and Mental Health Services Administration. http://www.ncbi.nlm.nih.gov/pubmed/27656736

37.

Substance Abuse and Mental Health Services Administration. (2012). SAMHSA’s working definition of recovery: 10 guiding principles of recovery. https://store.samhsa.gov/system/files/pep12-recdef.pdf

38.

Szafranski

D. D.

Gros

D. F.

Acierno

Brady

K. T.

Killeen

T. K.

Back

S. E.

(2019). Heterogeneity of treatment dropout: PTSD, depression, and alcohol use disorder reductions in PTSD and AUD/SUD treatment noncompleters. Clinical Psychology & Psychotherapy, 26(2), 218–226.

39.

Takoda. (2018). Takoda’s treatment platform. https://www.takoda.io/

40.

Voelker

(2019). App aids treatment retention for opioid use disorder. JAMA, 321(5), 444. https://doi.org/10.1001/jama.2018.21932

41.

Walker

(2009). Retention in treatment-indicator or illusion: An essay. Substance Use & Misuse, 44(1), 18–27.

42.

Whipple

J. L.

Lambert

M. J.

Vermeersch

D. A.

Smart

D. W.

Nielsen

S. L.

Hawkins

E. J.

(2003). Improving the effects of psychotherapy: The use of early identification of treatment and problem-solving strategies in routine practice. Journal of Counseling Psychology, 50, 59–68.