Keeping care connected: e-Consultation program improves access to nephrology care

Abstract

Introduction

Health systems are seeking innovative solutions to improve specialty care access. Electronic consultations (eConsults) allow specialists to provide formal clinical recommendations to primary care providers (PCPs) based on patient chart review, without a face-to-face visit.

Methods

We implemented a nephrology eConsult pilot program within a large, academic primary care practice to facilitate timely communication between nephrologists and PCPs. We used primary care referral data to compare wait times and completion rates between traditional referrals and eConsults. We surveyed PCPs to assess satisfaction with the program.

Results

For traditional nephrology referrals placed during the study period (July 2016–March 2017), there was a 51-day median appointment wait time and a 40.9% referral completion rate. For eConsults, there was a median nephrologist response time of one day and a 100% completion rate; 67.5% of eConsults did not require a subsequent face-to-face specialty appointment. For eConsults that were converted to an in-person visit, the median wait time and completion rate were 40 days and 73.1%, respectively. Compared to traditional referrals placed during the study period, eConsults converted to in-person visits were more likely to be completed (p = 0.001). Survey responses revealed that PCPs were highly satisfied with the program and consider the quick turnaround time as the greatest benefit.

Discussion

Our eConsult pilot program reduced nephrology wait times and significantly increased referral completion rates. In large integrated health systems, eConsults have considerable potential to improve access to specialty care, reduce unnecessary appointments, and optimize the patient population being seen by specialists.

Keywords

eHealth remote consultation telecare teleconsulting telehealth

Introduction

As health systems grow larger and demand for specialty care increases, it is becoming increasingly important to reduce care fragmentation and enhance the primary care to specialty care interface.¹ In the US, the movement away from fee-for-service payment and towards value-based reimbursement and accountable care organizations creates financial incentives to improve patient care coordination and to keep specialty referrals within health systems.² Methods to ensure that referrals are tracked and completed – a concept known as closing the referral loop – support the Quadruple Aim of Healthcare Reform by improving provider-to-provider communication, patient experience, population health, and cost reduction.^3,4 In order to realize these aims, strategies must be adopted that support treatment within the primary care setting and optimize coordination between primary care and specialty services.^5,6

Referrals serve as the gateway to specialty care. With referral volume doubling within the past decade, access to specialists has become increasingly problematic for both patients and primary care providers (PCPs).⁷ Patients face multiple obstacles to specialty care – long wait times, cost and travel burdens to see specialists – each of which contribute to poor patient satisfaction and outcomes. These barriers disproportionately affect patients with chronic diseases who suffer from increased rates of morbidity and mortality due to delayed referrals.⁸ Even within integrated health systems, to maintain timeliness, PCPs may choose to refer patients to external specialists with shorter wait times, at the cost of care fragmentation.⁹ Communication breakdowns between providers frequently occur, which can result in duplicate diagnostic testing, polypharmacy, and confusion about conflicting care plans.^1,9 For specialists, concerns also exist about the appropriateness of referrals and lack of pre-requisite workup.¹⁰ Many referrals are requested for straightforward problems that might not have required a specialist visit, had other lines of communication between providers existed.

Various telehealth interventions have been proposed to improve communication, reduce care fragmentation, and shorten wait times.^11,12 One such intervention is the electronic consultation (eConsult), which is a method for one provider (typically a PCP), to request clinical management advice from another (typically a specialist) based on patient chart review, without a face-to-face visit.¹³ Specialists may provide management advice or recommend an in-person visit for more complex patients, which allows for more efficient use of specialty resources.^14–18 Early adopters of eConsults have demonstrated reduced waiting times for specialty appointments and fewer unnecessary face-to-face specialty visits.^11,19–22 Interviews and surveys reveal high provider satisfaction, educational benefits of eConsults for PCPs, and improved co-management of complicated patients.^23–27 Cost savings for health systems and patients have also been examined.^28–30

Despite the potential benefits, the adoption of eConsults remains limited within fee-for-service health systems in the US due to lack of reimbursement.³¹ Many studies have been conducted in capitated, safety-net systems and Veterans Affairs (VA) medical centers.³² Further exploration of eConsult programs and more objective measures of their impact in different settings for various clinical conditions can help to encourage the adoption of payment models to cover this service.^2,32,33 Nephrology is ideally suited for an eConsult model since timely nephrology referral is crucial for patients with high-risk chronic kidney disease (CKD), and yet many new patients referred to nephrology have low-risk CKD and do not require in-person specialist visits.^14,15

Within a large, integrated, fee-for-service health system, we developed a nephrology eConsult pilot program to address challenges to specialty care access. Our quality improvement study sought to characterize the demand for nephrology care and identify the areas where eConsults could most improve the referral process. We aimed to identify which clinical conditions in nephrology were most amenable to eConsults. We hypothesized that eConsults would improve closure of the specialty referral loop, reduce nephrology care wait times, and increase referral completion rates for patients within our primary care network.

Methods

Setting

Duke Primary Care (DPC) is a large, academically affiliated primary care network that contains 34 clinic sites across seven counties, with over 230 providers and approximately 700,000 patient encounters annually. DPC is part of the Duke University Health System, a large academic health system with three hospitals and over 2000 affiliated specialty physicians. A single electronic health record (EHR), EPIC Systems Corporation (Verona, WI), is used among all PCPs, specialists, and hospitals. Our study took place between July 2016 and March 2017.

Intervention

Stakeholders from health system leadership, nephrology, primary care, and health informatics, convened to develop a nephrology eConsult program. The program contributors designed an eConsult workflow embedded within EPIC that would use existing functionality for ordering, documentation, and messaging. Using a “nephrology e-communication” order, PCPs were given the option to request diagnostic and treatment recommendations from a Duke nephrology provider based on chart review, without a face-to-face visit. PCPs were encouraged to use this optional eConsult pathway for clinical management advice; however, it was not mandatory, and PCPs could still submit a traditional ambulatory referral to nephrology without restriction. In the eConsult order, PCPs specified their clinical concerns and whether they preferred the nephrology care coordinator to contact the patient to facilitate any additional recommended workup and/or referrals. The order routed the request to the nephrology eConsult queue as an “in-basket” message. An eConsult note template was created for ease of documentation of specialist recommendations for diagnostic testing, treatment, or referral for in-person evaluation. The written note was then returned to the PCP as an “in-basket” message. PCPs retained primary responsibility for patient care and had the final authority to accept or reject eConsult recommendations.

The service was deployed to four DPC clinics during the study period. The four pilot clinic sites were chosen based on geography (with priority given to sites distant from Duke nephrology clinics) and average-to-high nephrology referral rates (i.e. greater than 50 referrals per year) to provide sufficient volume for eConsults. A series of face-to-face stakeholder visits were conducted at the selected primary care clinic sites to introduce the program to PCPs, to build engagement, and to obtain input into the design and workflow. One nephrologist (C Blake Cameron) completed eConsults during the study period.

Analyses

We conducted a retrospective analysis of traditional nephrology referral leakage rates (i.e. the percentage of referrals sent to external, non-Duke providers), completion rates, and wait times. DPC’s referral data served as our data source for referrals placed to nephrology within the study period. The data set included the referring provider; clinic; referral type (i.e. internal vs. external); referral order date; and for internal referrals, specialty appointment date and specialty appointment status (i.e. completed, scheduled, cancelled, no-show, or no status). We defined appointment wait time as the number of days elapsed between the referral order and the specialty appointment.

A retrospective cohort study design was used to evaluate the eConsult program. Data recorded via the medical record included the date the eConsult order was made; the date the specialist completed the eConsult note; clinical diagnosis; reason(s) for the consult request; and specialist recommendations for diagnostic testing and treatment, including referrals. The eConsult nephrologist measured the length of time for eConsult completion, inclusive of chart review and documentation. For eConsults that were converted to in-person visits, the date of the appointment and the appointment status were tracked. The proportion of eConsults converted to in-person visit was stratified by clinical diagnosis.

This quality improvement study protocol was deemed exempt by the Duke Institutional Review Board.

Questionnaire

To measure physician satisfaction with eConsults, an online survey was sent to PCPs at the end of the pilot study in March (Qualtrics, Provo, UT) (Appendix 1).

Analyses

Differences in referral completion rates were assessed using chi-squared testing and median wait times were assessed using the Mann–Whitney test (α < 0.05). JMP Pro 12 (SAS Institute Inc., Cary, NC) was used to run statistical analyses.

Results

Impact on referral wait times and completion

During the study period (July 2016–March 2017), 1239 referrals were made to nephrology by 173 DPC providers at 30 clinic sites; 68.8% of referrals were sent within the Duke network, while 31.2% were sent to external nephrologists. External referrals did not have documented appointment dates or completion statuses due to lack of EHR integration with those providers. Therefore, our analysis focused on internal referrals (n = 853). Of the internal referrals, 40.9% were marked as complete, 12.8% were cancelled, 13.4% were scheduled after the study period, and 6.3% were no-shows. The remaining 26.6% of internal referrals did not have a documented status or appointment date. The median wait time for the 626 completed, cancelled, scheduled, and no-show internal appointments was 51 days (0–371 days).

From July 2016 to March 2017, 80 nephrology eConsults were submitted for 78 patients (Figure 1). These eConsults were placed by 21 DPC providers at four pilot clinic sites and by one Duke endocrinologist, who was involved in the program launch. All of the eConsults received were completed during the study period. The time for eConsult note completion ranged from 10 to 18 minutes (mean = 15 minutes), as measured by the nephrologist. The median response time for eConsults was one day; 45% were completed on the same day and 80% were completed within two days. The nephrologist did not recommend a face-to-face visit for 67.5% of patients. For the 32.5% of patients for whom a face-to-face nephrology visit was recommended, wait times and visit outcomes were measured. The median wait time was 40 days (1–102 days) (Figure 2). This wait time was shorter than the wait times for traditional referrals (p = 0.15). Approximately three quarters (73.1%) of the visits were completed, 15.4% were cancelled, 7.7% were scheduled, and 3.8% were no-shows (Figure 3). The 26 in-person visits were scheduled among eight different Duke nephrology providers. An appointment status and wait time were available for all eConsults that were converted to in-person visits.

Figure 1.

Flow diagram of nephrology sent during the study period (FY2017).

Figure 2.

Median wait times for nephrology care (eConsults, eConsults converted to in-person visit, traditional referrals).

Figure 3.

Specialty visit outcomes following traditional referrals versus eConsults.

Pre-visit triage and diagnostic testing

eConsults were requested for a variety of clinical concerns: CKD (65%), resistant hypertension (15%), electrolyte disturbance (11%), proteinuria (7.5%), and other conditions such as elevated creatinine, acute kidney injury, and renal cyst (7.5%). Of the patients with CKD, 78.8% were diagnosed with CKD stage 3 and 19.2% with CKD stage 4. The specialist reviewer recommended that 63.4% of patients with CKD stage 3 be managed in primary care with specialist guidance, without the need for in-person evaluation. Among those with CKD stage 4, 60% were recommended to be seen in-person by a nephrologist. For the 26 patients who were scheduled for an in-person visit, recommended pre-visit diagnostic testing included completion of labs (65%) and imaging studies (23%). An additional 35% had recommendations on medication changes prior to the in-person visit (Table 1).

Table 1.

eConsult requests and recommendations.

	Total eConsults	eConsults converted to in-person visit
Sample size (N)	80	32.5% (26)
Clinical concern
Chronic kidney disease (CKD)	52	36.5% (19)
CKD stage 3	41	36.6% (15)
CKD stage 4	10	60% (6)
Resistant HTN	12	33.3% (4)
Electrolyte disturbance	9	11.1% (1)
Proteinuria	6	0% (0)
Other	6	0% (0)
Requested action
Clinical advice management	78	30.8% (24)
Re-establish nephrology care after loss to follow-up	6	66.7% (4)
In-person referral triage	7	42.9% (3)
Patient has seen a nephrologist in the past two years
No	76	32.9% (25)
Yes	4	25% (1)
Prefer the nephrology care coordinator to contact the patient
No	22	22.7% (5)
Yes	58	36.2% (21)
Recommendations
Labs: basic (chemistry panel, CBC, urine protein/albumin)	29	34.5% (10)
Labs: advanced (anything else)	20	35% (7)
Medication recommendation	36	25% (9)
Imaging	16	37.5% (6)
Referral	24	87.5% (21)

HTN: hypertension; CBC: complete blood count.

PCP satisfaction

The online survey to measure PCP satisfaction had a 71.4% response rate. A majority (86.6%) of the PCPs rated their overall satisfaction with the program as a 9 or 10 out of 10 (“highly satisfied”) and 100% were “extremely likely” to continue using eConsults. Similarly, a majority (80%) of PCPs indicated that, in the absence of the eConsult pilot program, they would have sent a traditional nephology referral for most patients for whom they submitted eConsults. The remaining 13.3% of PCPs indicated that they would have simply managed the patient in primary care, and 6.7% would have contacted a specific nephrologist via phone, email, or staff message. PCPs were asked to provide open-ended, qualitative responses about the benefits and drawbacks of the eConsult program. A majority (78.6%) said that the quick turnaround time by the nephrologist was the biggest benefit, while the remaining 21.4% described the efficiency for providers and patients as the greatest benefit. Recommendations for improvement included: 1) using eConsults for follow-up specialist visits; 2) making eConsults available to all DPC practices; 3) expanding eConsults to other specialties. Specialties specifically requested included rheumatology, neurology, cardiology, gastroenterology, endocrinology, and dermatology (Table 2).

Table 2.

PCP satisfaction survey results.

	Percentage of PCPs
Response rate	71.4% (15/21)
1. On a 10-point scale (10 = highly satisfied, 1 = highly dissatisfied), how satisfied are you overall with the eConsult program?
10	73.3% (11/15)
9	13.3% (2/15)
7	6.7% (1/15)
2. For the majority of patients for whom you have submitted eConsults, what would you have done if eConsults were not available?
Ambulatory referral to nephrology with/without additional diagnostics prior to referral	80% (12/15)
Manage in primary care	13.3% (2/15)
Contact nephrologist via phone/email/staff message	6.7% (1/15)
3. What are the biggest benefits of using eConsults? [Free response]	93.3% (14/15)
Timeliness: “Timeliness of consults and ability to handle ‘simple’ issues without a formal consult” “Speed and ease of consult, quick response”	78.6% (11/14)
Efficiency: “Efficiency for provider and patient; triage of referrals – PCPs not always sure when to refer to nephrology” “Patients do not have to travel to Duke for evaluation”	21.4% (3/14)
4. How likely are you to continue using eConsults?
Extremely likely	100% (15/15)
5. How would you improve the eConsult program? [Free response]	40% (6/15)
“Other specialties, available to all PC practices” “Try to have them used for follow up visits with the specialist” “Create order set for AKI/CKD”
6. If we were to expand eConsults to other specialties, what would you like to see included? [Free response]	86.7% (13/15)
Rheumatology, neurology, cardiology, gastroenterology, endocrine, dermatology, psychiatry, haematology, infectious disease, hepatology, pulmonology, surgical specialties, pharmacy

PCPs: primary care providers; PC: primary care; AKI: acute kidney injury; CKD: chronic kidney disease.

Discussion

Our quality improvement study piloted a nephrology eConsult program in four primary care clinics affiliated with a large, academic health system. Sites and patients participating in our intervention gained access to nephrology care within hours/days, rather than weeks/months. Among those eConsults that were converted to in-person visits, we noted shorter wait times and significantly increased referral completion rates. The findings of our pilot program demonstrate that eConsults can improve access to nephrology care and reduce demand for face-to-face visits. As health systems and payers become increasingly accountable to measures of network adequacy and access, eConsults may be employed as part of a strategy to ensure patients receive appropriate and timely care.^12,23

Patients benefited from timely specialist expertise without incurring specialist visit costs, extra traveling costs, or time off from work. With 67.5% of patients not requiring a subsequent face-to-face visit, most patient care needs were met through the specialist’s written recommendations. PCP survey responses reveal that 80% of participating providers would have referred most of those patients to nephrology if eConsults did not exist, further supporting the notion that specialist triage deflected unnecessary referrals. These results are consistent with the wide range of avoided in-person visits (25% to 76%) for different specialties, including nephrology, that has been demonstrated in previous studies.^21,23,31

For eConsults converted to a face-to-face visit, visit completion rates were significantly higher than traditional referrals sent during the study period (73.1% versus 40.9%). A possible explanation for these results is the increased level of involvement shared among the PCP, nephrologist, and patient in the eConsult process. PCPs informed their patients that they were seeking a second opinion from a kidney specialist and that the nephrologist would review their chart to determine appropriate next steps. When an in-person visit was recommended by the nephrologist, patients were made aware by their PCPs of the importance and necessity of being seen in-person. The nephrologist expedited scheduling for patients urgently requiring an in-person visit. These additional interactions may have provided patients with a better understanding of the rationale for the referral and the necessity for their specialist visit, which could be lacking in our traditional referral system. Taken together, these points illustrate the potential for eConsults to provide patient-centred care.

Our results also illustrate the clinical value of eConsults. For most patients (95%), eConsults were their first exposure to nephrology care in the previous two years. For patients with CKD, early nephrology involvement has been associated with slower disease progression.³⁴ eConsults could be utilized as part of a comprehensive strategy to engage populations with chronic illness early in the disease course. The benefits of specialist triage were also evident in our results; 63.4% of patients with CKD stage 3 were managed in primary care without the need for in-person evaluation, whereas 60% of patients with CKD stage 4 were recommended to be seen in-person. By allowing patients with less complexity to be managed in primary care with specialist guidance, eConsults may optimize the case mix of nephrologists’ patient panels and allow providers to practice more often at “the top of their license.” Furthermore, when a face-to-face nephrology visit was required, eConsults ensured appropriate pre-visit diagnostic testing. Over half (65%) of patients were recommended to complete labs and 23% underwent an imaging study prior to their visit. More than a third (35%) made a change to their medications before their visit. In this way, eConsults may shorten the interval between referral and receipt of therapy, and reduce the number of specialist follow-up visits.

Buy-in from PCPs was necessary for a successful pilot. Survey responses demonstrated high levels of PCP satisfaction, reducing concerns for PCP resistance to a change in their workflow, which has been cited in previous studies.^13,35 This resistance may have been reduced by the nephrologist visiting primary care clinics and seeking design input prior to program implementation. Moreover, the nephrologist placed the orders for additional labs, imaging, and medication changes, and the PCP was simply responsible for signing them, thereby decreasing the burden of additional work for PCPs. The survey results reflect anecdotal feedback received from both providers and patients, which were highly positive.

Limitations

Our pilot study had several limitations. First, it was conducted on a small sample size of primary care clinics and providers, which limits generalizability. All eConsults were completed by one specialist reviewer who is the senior author of this paper and was a stakeholder involved in the eConsult pilot development. Thus, the results, including turnaround time, may not be representative of eConsults programs operating at a larger scale. Our eConsult pathway was optional and resulted in participating sites utilizing the service for approximately one-third of nephrology referrals during the study period. The high PCP satisfaction we observed may be the result of selection bias. It is possible that PCPs favourable to eConsults were more likely to participate. If eConsults or specialist-led triage were made mandatory for initial referral, it is possible PCP satisfaction would decrease. In our program, PCPs were not compensated for the additional work required to complete eConsults, and the nephrology consultant received salary support from the health system. An eConsult program operating at scale – and particularly with mandatory participation – would likely require a compensation model that not only supports consultants, but that also offsets the incremental workload incurred by PCPs.

Additionally, this study analysed the demand for specialty care only from DPC. Specialty demand is also generated from non-Duke PCPs, other Duke specialists, and self-referred patients. These groups were not included in the program or in the analysis. Furthermore, our analysis of traditional referral characteristics was limited by the lack of data about wait times and completion rates for referrals sent to external specialists. It is possible that external specialists have shorter wait times. However, we believe the inability to “close the loop” with external providers is problematic, regardless of wait time.

Conclusion

Our study describes the implementation of an eConsult program in a large integrated, fee-for-service health system. We demonstrated that eConsults improved closure of the specialty referral loop and dramatically shortened wait times for nephrology care. This study supports the growing body of literature on eConsult programs and provides additional evidence that eConsults can reduce specialty referral demand and improve efficiency. While salaried physicians and capitated reimbursement provides a more favourable environment for eConsult implementation, compensation models for fee-for-service health systems need to be explored.

Footnotes

Acknowledgements

Additional contributors include Mark Sendak (Duke Institute for Health Innovation), Genie Komives (Department of Community and Family Medicine, Duke University School of Medicine; Duke Primary Care; Duke Connected Care), Dev Sangvai (Department of Community and Family Medicine, Duke University School of Medicine; Duke Primary Care; Duke Connected Care), Katherine Heller (Department of Statistical Science, Duke University School of Medicine), Joe Futoma (Department of Statistical Science, Duke University School of Medicine), Uptal Patel (Divisions of Nephrology, Department of Medicine, Duke University School of Medicine); Stephen Smith (Divisions of Nephrology, Department of Medicine, Duke University School of Medicine), L Ebony Boulware (General Internal Medicine, Department of Medicine, Duke University School of Medicine). This work was presented at the 12th Annual Duke Health Patient Safety and Quality Conference (23 March 2017) and the 2017 Annual Scientific Session of the North Carolina Chapter of the American College of Physicians (24 February 2017).

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The Duke Institute for Health Innovation (DIHI) funded this year-long pilot program.

Appendix 1

References

Bodenheimer

. Coordinating care – a perilous journey through the health care system. N Engl J Med 2008; 358: 1064–1071.

Barnett

Mehrotra

Frolkis

et al.

Implementation science workshop: implementation of an electronic referral system in a large academic medical center. J Gen Intern Med 2016; 31: 343–352.

eCQI Resource Center. Closing the referral loop: receipt of specialist report, https://ecqi.healthit.gov/ecqm/measures/cms050v6 (2017, accessed 9 August 2017).

Bodenheimer

Sinsky

. From triple to quadruple aim: care of the patient requires care of the provider. Ann Fam Med 2014; 12: 573–576.

Zuchowski

Rose

Hamilton

et al.

Challenges in referral communication between VHA primary care and specialty care. J Gen Intern Med 2015; 30: 305–311.

Kirschner

Barr

. Specialists/subspecialists and the patient-centered medical home. Chest 2010; 137: 200–204.

Barnett

Song

Landon

. Trends in physician referrals in the United States, 1999-2009. Arch Intern Med 2012; 172: 163–170.

Navaneethan

Aloudat

Singh

. A systematic review of patient and health system characteristics associated with late referral in chronic kidney disease. BMC Nephrol 2008; 9: 3–3.

Mehrotra

Forrest

Lin

. Dropping the baton: specialty referrals in the United States. Milbank Q 2011; 89: 39–68.

10.

Singh

Esquivel

Sittig

et al.

Follow-up actions on electronic referral communication in a multispecialty outpatient setting. J Gen Intern Med 2011; 26: 64–69.

11.

Chen

Murphy

Yee

Jr . eReferral – a new model for integrated care. N Engl J Med 2013; 368: 2450–2453.

12.

Caffery

Farjian

Smith

. Telehealth interventions for reducing waiting lists and waiting times for specialist outpatient services: a scoping review. J Telemed Telecare 2016; 22: 504–512.

13.

Tuot

Leeds

Murphy

et al.

Facilitators and barriers to implementing electronic referral and/or consultation systems: a qualitative study of 16 health organizations. BMC Health Serv Res 2015; 15: 568–568.

14.

Mendu

McMahon

Licurse

et al.

Electronic consultations in nephrology: pilot implementation and evaluation. Am J Kidney Dis 2016; 68: 821–823.

15.

Scherpbier-de Haan

van Gelder

Van Weel

et al.

Initial implementation of a web-based consultation process for patients with chronic kidney disease. Ann Fam Med 2013; 11: 151–156.

16.

Stoves

Connolly

Cheung

et al.

Electronic consultation as an alternative to hospital referral for patients with chronic kidney disease: a novel application for networked electronic health records to improve the accessibility and efficiency of healthcare. Qual Saf Health Care 2010; 19: e54–e54.

17.

Horner

Wagner

Tufano

. Electronic consultations between primary and specialty care clinicians: early insights. Issue Brief (Commonw Fund) 2011; 23: 1–14.

18.

Tuot

Murphy

McCulloch

et al.

Leveraging an electronic referral system to build a medical neighborhood. Healthc (Amst) 2015; 3: 202–208.

19.

Chen

Kushel

Grumbach

et al.

Practice profile. A safety-net system gains efficiencies through ‘eReferrals’ to specialists. Health Aff (Millwood) 2010; 29: 969–971.

20.

Liddy

Maranger

Afkham

et al.

Ten steps to establishing an e-consultation service to improve access to specialist care. Telemed J E Health 2013; 19: 982–990.

21.

Keely

Liddy

Afkham

. Utilization, benefits, and impact of an e-consultation service across diverse specialties and primary care providers. Telemed J E Health 2013; 19: 733–738.

22.

Olayiwola

Anderson

Jepeal

et al.

Electronic consultations to improve the primary care-specialty care interface for cardiology in the medically underserved: a cluster-randomized controlled trial. Ann Fam Med 2016; 14: 133–140.

23.

Barnett

Yee

Jr Mehrotra

et al.

Los Angeles safety-net program eConsult system was rapidly adopted and decreased wait times to see specialists. Health Aff (Millwood) 2017; 36: 492–499.

24.

Straus

Chen

Yee

Jr et al.

Implementation of an electronic referral system for outpatient specialty care. AMIA Annu Symp Proc 2011; 2011: 1337–1346.

25.

Kim

Chen

Keith

et al.

Not perfect, but better: primary care providers’ experiences with electronic referrals in a safety net health system. J Gen Intern Med 2009; 24: 614–619.

26.

Davis

Gilchrist

Grumbach

et al.

Advancing the primary/specialty care interface through econsults and enhanced referrals. Ann Fam Med 2015; 13: 387–388.

27.

Liddy

Afkham

Drosinis

et al.

Impact of and satisfaction with a new eConsult service: a mixed methods study of primary care providers. J Am Board Fam Med 2015; 28: 394–403.

28.

Liddy

Drosinis

Deri Armstrong

et al.

What are the cost savings associated with providing access to specialist care through the Champlain BASE eConsult service? A costing evaluation. BMJ Open 2016; 6: e010920–e010920.

29.

Kirsh

Carey

Aron

et al.

Impact of a national specialty e-consultation implementation project on access. Am J Manag Care 2015; 21: e648–e654.

30.

Palen

Price

Shetterly

et al.

Comparing virtual consults to traditional consults using an electronic health record: an observational case-control study. BMC Med Inform Decis Mak 2012; 12: 65–65.

31.

Gleason

Prasad

Ackerman

et al.

Adoption and impact of an eConsult system in a fee-for-service setting. Healthc (Amst) 2017; 5(1–2): 40–45.

32.

Vimalananda

Gupte

Seraj

et al.

Electronic consultations (e-consults) to improve access to specialty care: a systematic review and narrative synthesis. J Telemed Telecare 2015; 21: 323–330.

33.

Kim-Hwang

Chen

Bell

et al.

Evaluating electronic referrals for specialty care at a public hospital. J Gen Intern Med 2010; 25: 1123–1128.

34.

Smart

Dieberg

Ladhani

et al.

Early referral to specialist nephrology services for preventing the progression to end-stage kidney disease. Cochrane Database Syst Rev 2014; 6: CD007333–CD007333.

35.

Angstman

Adamson

Furst

et al.

Provider satisfaction with virtual specialist consultations in a family medicine department. Health Care Manag (Frederick) 2009; 28: 14–18.