Using Telepsychiatry to Enrich Existing Integrated Primary Care

Abstract

Background:

Integrated care is characterized by evolving heterogeneity in models. Using telepsychiatry to enhance these models can increase access, quality, and efficiencies in care.

Introduction:

The purpose of this report is to describe the process and outcomes of adapting telepsychiatry into an existing integrated care service.

Materials and Methods:

Telepsychiatry was implemented into an existing integrated care model in a high-volume, urban, primary care clinic in Colorado serving patients with complex physical and behavioral needs. Consultative, direct care, educational/training encounters, provider-to-provider communication, process changes, and patient-level descriptive measures were tracked as part of ongoing quality improvement.

Results:

Telepsychiatry was adapted into the existing behavioral health services using an iterative team meeting process within a stepped care model. Over 35% of the requests for psychiatry services were medication related–and medication changes (type/dose) were the most frequent referral outcome of psychiatric consultation. Forty percent of patients in the service had multiple behavioral health diagnoses, in addition to physical health diagnoses.

Discussion:

Telehealth will become an increasingly necessary component in building hybrid/blended integrated care teams. Examples of flexible model implementation will support clinics in tailoring effective applications for their unique patient panels.

Conclusions:

An adapted integrated care model leveraging telepsychiatry is successfully serving the complex deep end of a primary care patient population in Colorado. Lessons learned in implementing this model include the importance of team attitudes.

Introduction

Integrated Care

There is increasing recognition that integrated care, the systematic coordination of general and behavioral healthcare, is the most effective approach to achieving successful health outcomes for patients with multiple healthcare needs.^1

–4 The integrated approach proffers that comorbid conditions (i.e., diabetes, depression) are best treated with whole-person approaches. These address common and interacting etiologies and recognize that many patients are most likely to seek behavioral care in a primary care setting because of persistent access challenges and stigma.^5,6 As current healthcare transformation efforts strive to improve patient outcomes, patient experiences, and cost efficacy in the United States,⁷ integration is expanding and is characterized by evolving heterogeneity in models.^5,8 There are currently a few predominant models of integrated care implementation into primary care settings. These models differ primarily, in the roles, configuration, and interface of the behavioral health team members within larger care teams.

Models of Implementation

Collaborative care models, such as Improving Mood–Providing Access to Collaborative Treatment (IMPACT) and Depression Improvement across Minnesota Offering a New Direction (DIAMOND) are included in the Substance Abuse and Mental Health Services Administration (SAMHSA)'s list of evidence-based practices. These models were developed to improve depression care, and have demonstrated efficacy with mental health conditions most frequently seen in primary care settings. The models leverage patient registries and care management to track, engage, and follow up with populations of patients. Psychiatric consultation provides case-based consultation to primary care physicians (PCPs) improving the ability of primary care providers to manage patients' behavioral health symptoms and prescriptions in the primary care setting.¹ A Millbank metareview found high-quality evidence from over 90 published studies citing improved mood disorder and quality-of-life symptoms related to collaborative care models.⁸

In addition, modified applications have demonstrated efficacy with other specific diagnoses in specialized and primary care settings. For example, studies of patients suffering from post-traumatic stress disorder (PTSD) following physical trauma have shown greater PTSD symptom improvements following a collaborative care treatment plan than treatment as usual at a Level I acute care inpatient setting.⁹ Similarly, Fortney et al. have shown that a collaborative care model implemented at Veterans Health Administration (VHA) community-based outpatient clinics (CBOCs) for Veterans with PTSD was more clinically effective¹⁰ and cost efficient¹¹ than usual care. Randomized controlled trials have shown that the Life Goals Collaborative Care model has been associated with reduced physical and behavioral symptoms and improved quality-of-life indicators for patients of community mental health centers, VHA outpatient, and primary care clinics. This collaborative care application used group sessions led by nurses/social workers/health workers and telephonic care management follow-up for patients with comorbid bipolar disorder and chronic disease, such as diabetes, hypertension, obesity, chronic obstructive pulmonary disease (COPD), etc. as well as other comorbid behavioral health disorders such as schizophrenia.¹²

The Cherokee Health Systems (CHS) model–another SAMHSA evidence-based model–was designed as a comprehensive service delivery system to promote mind and body wellness. The CHS philosophy is to meet all patient needs within the primary care setting. Serving underinsured and underserved Tennessee residents, their unified model serves patients with a wide variety of multiple, chronic, and complex diagnoses and social challenges. The CHS model uses team-based care with comanagement of behavioral health symptoms and medication across psychology, psychiatry, and primary care with community health workers supporting patient access to social services.¹³

Telemental Health

All models can leverage technology, including telemental health in the form of videoconferencing to enhance access to care and program efficiencies. A more recent body of literature finds that in some instances, the outcomes of virtual applications of these integrated models surpass in-person care models.^14
–16 Telemental health encompass a continuum of intensity, and published descriptions of application range from virtual physician education¹⁷ to provider consultation,¹⁸ patient self-management,¹⁹ and even virtually located primary care team psychiatrists.^16,20 For example, a study of depression care at a federally qualified health center found comparatively better symptom improvement for virtually delivered, over practice-based collaborative care. It is hypothesized that these virtual models not only are more cost effective for smaller practices, but also have higher fidelity to the collaborative care model than practice-based implementations.¹⁶

With the ability to support diverse patient panels, clinical models, and practice profiles, telemental health is being used to flexibly adapt to meet an increasing variety of clinical situations.¹⁴ The Telemedicine Outreach for PTSD (TOP) program specifically relies on telemental health technology to treat PTSD for veterans accessing care at VHA CBOCs. PTSD is elevated in veteran populations and care quality is of particular concern for rural veterans who constitute over 30% of afflicted veterans.¹¹ The TOP intervention leverages PTSD care teams comprised of five types of providers; onsite PCPs, and offsite nurse care managers and clinical pharmacists (available telephonically), and psychologists and psychiatrists (providing care to patients and consultation with PCPs through live, interactive televideo). This intervention was associated with significantly greater improvements in PTSD and depression symptom severity (with small—medium effect sizes) and was cost effective for those with comorbid depression, anxiety, and panic disorder; as compared with treatment as usual.¹¹

Integrated Care Using Telemental Health in Colorado

The purpose of this report is to add to the existing literature on effective telemental health models and describe how these can be effectively implemented, with details from a modified application of a virtual integrated care model. In Colorado, there are several programs already providing coordinated to fully integrated primary care. Virtual care platforms can easily enhance and improve efficiencies, but implementation must adapt to existing integrated workflow processes to be of optimal value to the practice. In describing CHS success, evaluators highlight four breakthrough innovations: telehealth, developing behavioral health clinicians (BHCs) trained to support psychiatric consultation in primary care, electronic health record (EHR), and physical workspace that supports integration.¹³ While all four apply to the Colorado model, the first two, telehealth and developing BHCs trained to support psychiatric consultation in primary care, are particularly relevant to successful implementation (Table 1).

Table 1.

Integrated Care Models: Comparison of Key Model Components

	PATIENT REGISTRIES	POPULATION HEALTH APPROACH	SHARED EHR	PCP AND PSYCHIATRIST CO-MANAGE PATIENTS	CRITICAL ROLE OF CARE MANAGEMENT	WORKFORCE MULTIPLICATION VIA BH EDUCATION	REFERRAL TO SPECIALTY MH FOR RARE/COMPLEX BH	PSYCHOLOGY ON CARE TEAM	TELEPSYCHIATRY
Collaborative care; primary care (IMPACT; DIAMOND)	✓				✓	✓	✓		✓
Collaborative care; specialty locations (TOP, LGCC)	✓		✓		✓	✓		✓	✓
Cherokee health systems		✓	✓	✓		✓		✓	✓
Colorado model			✓	✓		✓		✓	✓

DIAMOND, depression improvement across Minnesota offering a new direction; EHR, electronic health record; IMPACT, improving mood–providing access to collaborative treatment; LGCC, Life Goals Collaborative Care; PCP, primary care physician; TOP, telemedicine outreach for post-traumatic stress disorder.

In this report, we describe the addition of telepsychiatry to an existing integrated primary care model—as adapted to meet the needs of a high-volume urban primary care clinic. This clinic was able to meet the behavioral health needs of the majority of their patients through embedded psychologists, but was seeking to add critical psychiatric support to help manage the subset of their patient panel with particularly complex physical and behavioral needs. The adapted care model centered on the embedded psychology team and leveraged the virtual psychiatrist to support the team in management of the “deep end” of their patient panel through ongoing care collaboration. The adaptation of telemental health into an existing integrated care model has not been previously described in the literature. Describing effective and flexible applications of telemental health and increasing the awareness of diverse models of integration will support integrated care teams in enhancing integrated care models by adding telehealth.

Materials and Methods

Site Description

In May 2015, the regional Medicaid health plan provider/telehealth subsidiary entity and a University primary care site embarked upon a virtual integrated care initiative to bring psychiatry services for adults into an integrated, primary care setting. The practice was already using an integrated care model with team-based care that leveraged secure electronic staff messages, consultation between BHCs and PCPs, and weekly team meetings. The practice is home to 40 Primary Care/Family Practitioners, 3 BHCs (one full-time psychologist, a psychology intern, and a postdoctoral fellow,) and 18 residents. They see ∼13,000 patients per year; 10% youth, and 90% adult, who account for 29,000 visits annually. Approximately 60% have private insurance, 20% Medicare, and 20% Medicaid benefits. The majority of visits are from white patients, with Hispanic patients accounting for the highest minority proportion (14%) of visits annually.

To enhance care integration and the ability of the providers to manage both behavioral health symptoms and psychotropic medication, video conferencing was used to bring virtual psychiatry to the center. One psychiatrist and one nurse practitioner from the University provided 1 day/week each to the clinic. These psychiatric consultants supported the existing clinic team by providing a combination of direct patient care and provider consultation. Before implementation, the psychiatric providers dedicated significant time to meeting with the primary care team to understand unique clinic needs. This step intentionally helped the new team develop positive relationships and process conducive to tailoring psychiatric services to meet clinic needs through ongoing, iterative program evolution. During initial implementation, the psychiatric consultants spent substantial time on-site, nurturing strong working relationships with clinic staff and designing an effective clinic workflow for the new service and technology. Psychiatrists gradually decreased time on-site until all psychiatric services were virtual. Test sessions using video conferencing began on April 2015; ongoing virtual and in-person service began on June 2015 (Table 2).

Table 2.

Implementation Timeline

ON-SITE PRACTICE TELEPSYCHIATRY SESSIONS	ONGOING VIRTUAL AND IN-PERSON TELEPSYCHIATRY	INTERMITTENT IN-PERSON AND VIRTUAL TELEPSYCHIATRY, TECHNOLOGY CHALLENGES, TWO PLATFORM CHANGES.	DEDICATED BANDWIDTH SOLUTION AND FULLY FUNCTIONING, VIRTUAL TELEPSYCHIATRY MODEL
April 2015	June 2015	July 2015 through December 2016	January 2017

Evaluation

Consultation, direct care, and education/training encounters were tracked as part of ongoing quality improvement (QI). For this article we reviewed with the University of Colorado IRB that this evaluation was indeed QI and did not constitute research. Monthly virtual team meetings provided a critical forum for program evolution with feedback about program successes and challenges, strengths, and weaknesses leading to continual iterative program changes. Meetings included the psychiatric providers, the clinic manager and administrative staff, the clinic's BHC evaluation staff, and a telemental health expert/consultant. Process-focused measures and qualitative feedback were summarized to document the evolution of the implementation model. Qualitative feedback included semistructured interviews with psychiatric providers, monthly team meeting summaries and correlating topics, themes, challenges, and resolutions.

Communication during the team meetings influenced the evolution of the evaluation design. While original QI was focused on descriptive analysis of encounters, it became clear that documented direct care and provider-to-provider encounters described only a portion of effective collaborative care implementation. Team members consistently described the importance of ongoing and informal team communication. To better understand “behind the scenes” communication, a content analysis of a convenience sample of the team's EHR system-based electronic messages was completed. E-messaging is frequently used for provider-to-provider consultation, providing a record of communication. Fifty-three unique message strings were reviewed. Provider satisfaction was also tracked.

Results

Over 1 year, the psychiatric consultants provided an average of 26.5 documented services per month (318 total) with an average session length of 38 min. The largest proportion of services 137, 48%, were curbside consultations between a PCP and psychiatric provider. Seventy-eight encounters were direct service—a third of all psychiatric consultant services, with an average of six direct care services per month. Sixty (21%) of sessions included the psychologist.

Approximately 70% of psychiatric consultant time was spent on documented direct care, curbside, and co-consult sessions. This includes some e-messaging through the clinic's EHR, but does not include administrative consultation-related activities, such as chart reviews, notes, and ongoing daily provider-to-provider messages. Over 35% of the requests for psychiatry services were medication related, and medication changes (type/dose) were the most frequent referral outcome. The largest number of encounters was related to mood disorders and 40% of patients had multiple behavioral health diagnoses in addition to physical health diagnoses. Nine patients had psychotic symptoms, and sleep, chronic pain, and PTSD were noted as frequent and complicating factors. A subset of patients had three or more comorbid health diagnoses. Examples of these medically and behaviorally complex diagnosis constellations are included in Table 3 below.

Table 3.

Examples of Comorbid Diagnoses Lists

Patient 1	Severe depression, renal failure, hypertension, CHF, morbid obesity, insulin-dependent diabetes, sleep apnea
Patient 2	Bipolar, hypothyroidism, migraines, severe premenstrual syndrome, including suicidality
Patient 3	Bipolar, diabetes, history of substance use disorder, chronic back pain, cardiomyopathy, pulmonary hypertension, and suicide ideation
Patient 4	Joint degeneration muscular atrophy, social anxiety, factitious disorder, chronic pain, phobias
Patient 5	Perinatal depression, grief, anxiety, insomnia, psychosocial stress
Patient 6	Depression, anxiety, PTSD, end-stage renal disease, on dialysis, refusal to continue behavioral care with community mental health center

PTSD, post-traumatic stress disorder.

Fifty-three electronic message chains were analyzed for content, participants, and patient complexity. The analysis found strong indices of patient complexity, with the largest number of messages addressing patients with three or more diagnoses. Seventeen unique primary care providers reached out for support with complex patients, with 38% of messages involving more than two participants–most typically a psychiatric provider, a PCP, and a psychologist. A patient access representative was involved in 25% of messages. Medication and scheduling-related messages were most frequent (42% and 40% of messages); and 40% (n = 21) messages addressed multiple content categories.

Experience of Implementation

“Technology implementation, and change management take a tremendous amount of time, care, and attention specifically to personal relationships.”

— Integrated Care Psychiatrist.

Workflow

In beginning months of implementation, psychiatric consultations were in-person. The team found that up-front, in-person time expedited positive relationships between on-site and virtual team members, and helped them answer ongoing workflow questions (Table 4) and develop functional workflow.

Table 4.

Initial Implementation: Key Considerations

What level of communication between the PCPs and psychiatric consultants is appropriate? How much is too much? Or, not enough?

Should the psychiatric consultants be appraised about each behavioral health patient on an ongoing basis? Or just as needed?

How should the psychiatric consultants respond to PCPs when psychiatric consultants are not quite comfortable answering co-consult questions in the moment? (i.e., psychiatrist does not feel like she has enough information to answer an email or quick phone question.)

When should a primary care provider refer a patient to the behavioral health therapists before consulting with psychiatry?

What kind of template can effectively collect the minimum necessary case/patient information needed for the psychiatrists to support e-messaging consultation–without being too burdensome to PCPs or psychiatrists?

The team, with technical/university-based telehealth and integrated care support, spent time normalizing the fluid nature of implementation. The team members came to realize that the psychiatric providers could not be expected to have immediate answers to all questions—in many cases, psychiatric consultants need to engage in discussion with onsite team members, or to see patients directly. Through ongoing team meetings, roles and expectations for the psychiatric providers changed and evolved. Table 4 describes some of the most critical implementation questions that clinical team members worked to address over initial implementation. The majority of these questions addressed how to efficiently communicate between providers. Instead of having conclusive resolution, many of these questions were addressed iteratively, with informal and ongoing education provided by the psychiatrists. As the PCPs and residents became more familiar with the clinical style of each psychiatrist, they began tailoring e-messaging and case notes to support efficient case conceptualization. The on-site team over time, became more aware of the specific type of information psychiatric consultants need to respond to a PCP's “quick question.” This led to a truly stepped mode of care. Psychiatric services flexibly met the needs of each case and each primary care provider, as services ranged from brief, generic, diagnostic, or medication-related questions, to ongoing collaboration for diagnostic clarification, and sequential treatment strategies, including not only medications but also various forms of psychotherapy and social services. Virtual service also expanded to include staff e-messaging, on-demand provider-to-provider consults, co-consults with multiple team members, and direct care visits with multiple team members and patients, and integrating the psychiatrists into larger multidisciplinary care team meetings for the most complex patients. The team members' positive attitudes, flexibility, and adaptability consistently emerged as critical factors identified in resolving ongoing questions and challenges.

Experience with virtual technology

The early months of implementation were marked by technology challenges related to bandwidth limitations, discrepancy between the design and actual use of the virtual platform, and the clinic's need for workflow efficiency. Initial challenges included scheduling and user-access permission problems with the telehealth platform. The team also initiated a “Lucy Time” (from the Peanuts Comic); unscheduled psychiatrist availability time, during which on-site providers (PCPs, residents, and the psychologist) would stop in virtually with the telepsychiatrist to ask quick questions about, and follow-up on ongoing cases. The team members found that access to this ongoing, quick support was a useful additive service to help efficiently manage cases, and strengthen provider rapport across physical and behavioral health domains.

Before this pilot implementation, the two psychiatric providers had not participated in virtual care and neither felt particularly comfortable with the idea of telehealth. By 3 months in, both felt comfortable using the virtual platform and would recommend telehealth to other providers. Qualitative interviews revealed that the high level of project enthusiasm from all providers strongly contributed to perceptions of success. Technology issues aside, the providers found the platform easy to use and in general, liked the virtual access to psychiatry. Providers did not feel the technology impeded their ability to communicate and collaborate with either patients or other providers, and psychiatric providers were able to assess patient symptoms and treatment needs as well as virtually in-person.

Discussion

To our knowledge, this is the first article that describes the implementation process for enhancing and existing in-person integrated care service with telepsychiatry. As integrated care becomes more widely disseminated, telehealth will become a critical component in building hybrid/blended integrated care teams with virtual and in-clinic teams. Examining pragmatic and flexible models and processing for developing these services will be important for the continued success of integrated services.

The typical profiles of patients served by the telepsychiatry component of the integrated care model confirm complexity in a subset of primary care patients. The Colorado model was developed to address whole-person care within these subpopulations. While this evaluation is limited by lack of clinical outcome data, process measures (i.e., indications of diagnosis and medication changes and physician report) and qualitative results indicate efficacy and provide support for adapting and expanding established integrated care models. Such models, originally designed to treat less complex profiles of mood disorders in the primary care setting, can also serve the complex deep end of the primary care patient population. Additional research on expanded applications to include patient-level, clinical outcome data is warranted.

Like the original collaborative care and unified care models described above, the Colorado model is an efficient use of professional capacity and leverages existing workflow/integration expertise and a consultative model for workforce optimization. The part-time availability of psychiatric expertise successfully supported a large team of PCPs in enhanced behavioral health knowledge and care across a wide and complex array of diagnostic patient profiles. The iterative workflow change process illustrates how critical the continued flexibility of the model and the care team were to developing a model both effective and practical for the individual practice needs. The questions cited in Table 4 above likely represent questions that all clinics should be prepared to face, and resolve iteratively in ways that work for their unique clinical makeup. The team members in Colorado experienced new technology, workflow, and levels of integration as part of this pilot. One of pilot's psychiatric providers keenly translated the ongoing implementation challenges into three key “Lessons Learned.”

Have Patience: Organizational and change management takes more time than you might expect. It takes time and ongoing education.

Become Part of the Team: While in many instances, providers must operate independently, successful integration requires what she called “A Beginner's Mind.” This is a humility-based mindset that requires an ongoing learning orientation as providers on each side become exposed to an entirely different medical culture and figure out how to best be useful to each other. This mindset may require a shift from a hierarchical orientation to a team-based practice.

Relationships Matter: Relationships between providers WILL IMPACT patient care. This lesson involves being aware of different personalities and priorities, and building trust while navigating a new technological implementation

These lessons learned reflect published literature. Care integration is a known, particularly challenging application of healthcare innovation expansion.^8,21,22 Integrating care involves providers from disparate clinical models collaboratively changing care delivery. As described by Kirchner et al., this type of change necessitates ongoing QI efforts to encourage and steer adoption of new practices. These evaluators recommend QI efforts that attend to four key properties of healthcare change: (1) Leadership support at the top and midlevel of the organization and within the local community, (2) A culture that actively encourages ongoing learning and care improvement, (3) Focus on developing and strengthening effective teams, and (4) Using information technology for QI and external accountability.

Implementation of the Colorado model reinforces these recommendations, where top-level university and clinic level support were critical to overcoming ongoing technology and other care integration challenges, local support from Medicaid payor partners helped finance the pilot project, and clinical program champions kept the program momentum going. Implementing the program into a clinic with a large residency program took advantage of an existing culture that embraced ongoing learning and change, and external evaluation with regular reporting helped with accountability. Monthly team check-ins helped to support evolving team development—although the positive attitude and effort of the clinical team members was the real critical success factor in the recipe of ever-improving team efficacy.

Footnotes

Disclosure Statement

Drs. Shore and Thomas work for AccessCare, a provider of telehealth clinical services.

References

Substance Abuse and Mental Health Services Administration (SAMHSA), Health Resources and Services Administration (HRSA) Center for Integrated Care Health Solutions. Available at www.integration.samhsa.gov/integrated-care-models/behavioral-health-in-primary-care (last accessed December 27, 2018 ).

Berge

, Trump

, Trudeau

, Urzan

, Mandrich

, Slattengreen

, Nissley

, Miller

, Baird

, Colemena

, Wootten

. Integrated care clinic: Creating enhanced clinical pathways for integrated behavioral health care in a family medicine residency clinic serving a low-income, minority population. Fam Syst Health, 2017; 35:283–294.

Reiss-Brennan

, Brunisholz

, Dredge

, Briot

, Grazier

, Wilcox

, James

. Association of integrated team-based care with health care quality, utilization, and cost. JAMA, 2016; 316:826–834.

Reiss-Brennan

Mental health integration; normalizing team care. J Prim Care Community Health, 2014; 5:55–60.

Joseph

, Kester

, O'Brien

, Huang

. The evolving practice of psychiatry in the era of integrated care. Psychosomatics, 2017; 58:466–473.

Shim

, Rust

. Primary care, behavioral health, and public health: Partners in reducing mental health stigma. Am J Public Health, 2013; 103:774–776.

Berwick

, Nolan

, Whittington

. The triple aim: Care, health, and cost. Health Affair, 2008; 27:759–769.

Gerrity

. Evolving models of behavioral health integration: Evidence Update 2010–2015. 2016. Millbank Memorial Fund New York, NY. ISBN 978-1-887748-81-0. Available at www.milbank.org/publications/evolving-models-of-behavioral-health-integration-evidence-update-2010-2015 (last accessed February 20, 2018 ).

Zatzick

, Roy-Byrne

, Russo

, Rivara

, Droesch

, Wagner

, Dunn

, Jurkovich

, Uehara

, Katon

. A randomized effectiveness trial of stepped collaborative care for acutely injured trauma survivors. Arch Gen Psychiatry, 2004; 61:498–506.

10.

Fortney

, Pyne

, Kimbrell

, Hudson

, Robinson

, Schneider

, Moore

, Custer

, Grubbs

, Schnurr

. Telemedicine-based collaborative care for Posttraumatic Stress Disorder: A randomized clinical trial. JAMA Psychiatry, 2015; 72:58–67.

11.

Painter

, Fortney

, Auste

, Pyne

. Cost-effectiveness of telemedicine-based collaborative care for posttraumatic stress disorder. Psychiatr Serv, 2017; 68:1157–1163.

12.

Whiteman

, Naslund

, Di Napoli

, Bruce

, Bartels

. Systematic review of general medical and psychiatric self-management interventions for adults with serious mental illness. Psychiat Serv, 2016; 67:1213–1225.

13.

Khatri

, Perry

, deGruy

. Integrated health care at Cherokee Health Systems. In: Feinstein

, Connelly

, Feinstein

, eds. Integrating behavioral health into primary care. Oxford England: University Press, 2017:17–30.

14.

Hilty

, Ferrer

, Parish

, Johnston

, Callahan

, Yellowlees

. The effectiveness of telemental health: A 2013 review. Telemed J E Health, 2013; 19:444–454.

15.

Hubley

, Schneck

, Thomas

, Shore

. Review of key telepsychiatry outcomes. World J Psychiatry, 2016; 6:269–282.

16.

Fortney

, Pyne

, Mouden

, Mittal

, Hudson

, Schroeder

, Williams

, Bynum

, Mattox

, Rost

. Practice-based versus telemedicine based collaborative care for depression in rural federally qualified health centers: A pragmatic randomized comparative effectiveness trial. AM J Psych, 2013; 170:414–425.

17.

Arora

, Kalishman

, Thornton

, Komaromy

, Katzman

, Struminger

, Rayburn

. Project ECHO: A telementoring network model for continuing professional development. J Contin Educ Health Prof, 2017; 37:239–244.

18.

Hilty

, Yellowlees

, Tarui

, Viramontes

, Kerrigan

, Sprenger

, Shore

Mental health services for California Native Americans—Usual service options and a description of telepsychiatric consultation to select sites. In: Ramesh

, Shahram

eds. Telemedicine, 2013: InTech: ISBN 978-953-51-1211-2, DOI: 10.5772/56569. Available at www.intechopen.com/books/telemedicine/mental-health-services-for-california-native-americans-usual-service-options-and-a-description-of-te (last accessed February 22, 2018 ).

19.

Pratt

, Bartels

, Naslund

, Wolfe

, Pixley

, Mueser

, Josephson

. Feasibility and effectiveness of an automated telehealth intervention to improve illness self-management in people with serious psychiatric and medical disorders. Psychiatr Rehabil J, 2013; 36:297–305.

20.

Fortney

, Pyne

, Turner

, Farris

, Normoyle

, Avery

, Hilty

, Unṻtzer

. Telepsychiatry integration of mental health services into rural primary care settings. Int Rev Psychiatr, 2015; 27:525–539.

21.

Kirchner

, Edlund

, Henderson

, Daily

, Parker

, Fortney

. Using a multi-level approach to implement a primary care mental health (PCMH) program. Fam Syst Health, 2010; 28:161–174.

22.

Carter

, Chalouhi

, McKenna

, Richardson

. What it takes to make integrated care work. Health International 2011. McKinsey & Company. Available at www.mckinsey.com/∼/media/mckinsey/dotcom/client_service/healthcare%20systems%20and%20services/health%20international/issue%2011%20new%20pdfs/hi11_48%20integratedcare_noprint.ashx (last accessed February 19, 2018 ).