Abstract
E-consultations are being offered within clinic walls as an option for specialist advice without a face-to-face consultation appointment. In a six month time frame, nearly 100% of primary care internists and family medicine providers in a multispecialty practice had used an e-consultation at least once. Specialists also used e-consultations for advice from other specialists. E-consultations were often questions about interpreting images or laboratory tests, or questions about management of chronic conditions such as osteoporosis, hypertension, or headaches. Although e-consultations were offered as an alternative to face-to-face specialty consultations, 1,111 of 5,334 e-consultations eventually did receive face-to-face appointments in the same specialty. Within 30 days of the e-consultation 11.5% had a specialty face-to-face visit and 17.7% had seen a specialist face-to-face within 90 days of the e-consultation. The conversions of e-consultations to face-to-face consultations depended on the specialty providing the e-consultation (fewer for gastroenterology and infectious disease), patient distance from the clinic (fewer for international patients and those living greater than 800 kilometers from the clinic), and experience of specialist responding to the e-consultation (lower conversions for specialists providing 15 or more e-consultations).
Introduction
Consultations in the past were usually limited to either a formal face-to-face consultation or an informal “curbside” consult. 1 The e-consultation is now another option available for a provider to access specialty expertise.
An e-consultation is an asynchronous communication between a requesting provider and another provider, usually a specialist. Typically the e-consultation begins as a clinical question from a requesting provider who then sends the clinical question along with appropriate clinical material to a selected specialty. What happens after the transfer of information differentiates the e-consultation from a traditional consult. With the e-consultation there is no face-to-face interaction with a patient, so it can be done asynchronously. Since the consultation is completed electronically without a face-to-face visit, patients do not need to schedule an appointment to see the specialist and the specialist can perform the e-consultation at any time convenient for his or her schedule. Because the patient is not seen by the specialist, the requesting provider has the responsibility to communicate the specialist’s recommendations to the patient.
Wootton et al. distinguished e-consultations from e-referrals and e-transfers by noting that an asynchronous referral request (an e-referral) could be either a request for a patient transfer (e-transfer) or a request for information with no intent to transfer care of the patient (e-consultation). 2 This is a useful distinction that we use in this paper.
One use of e-consultations has been to improve access to specialized care in countries where specialists are scarce or where major travel barriers exist. The Swinfen Trust and other charitable organizations are examples of this use of e-consultations.3–5 Another motivation for e-consultations has been to reduce appointment wait times for specialists. In Canada and Scotland, wait times for specialists can be a year or more and e-consultations have been used to improve access by successfully handling cases asynchronously.6,7
In many healthcare organizations the infrastructure is now in place for more widespread use of e-consultations. Wootton et al. described the features needed for successful e-consultations and these include a single point of contact for the requesting provider to order an e-consultation, allocation to the correct specialist, and adequate data transfer and security. 2 The integrated and shared electronic medical record (EMR), electronic ordering, and scheduling systems have built in features that fulfill the Wootton et al. criteria. In addition, Wootton et al. criteria include e-consultation progress tracking and transfer of the information back to the requesting provider, all of which is handled within the EMR.
E-consultations requested and completed by providers within an integrated, multispecialty practice are what we refer to as internal e-consultations. Mayo Clinic also provides e-consultation services for non-Mayo providers (“external” e-consultations) but this paper is exclusively about internal e-consultations.
Internal e-consultations were developed and evaluated by Mayo Clinic as a potentially efficient yet quality, alternative to face-to-face consultations. 8 Specialists spend less time completing e-consultations when compared to face-to-face consultations, and requesting providers are satisfied with the process.9,10 Internal e-consultations are also now being integrated into care processes at Mayo Clinic to fulfill specific practice needs. 11 In addition, internal e-consultations are helping to resolve some patient scheduling and logistic issues. Patients travel to Mayo Clinic for integrated and comprehensive care, with an expectation of care to be provided in a compressed time frame. To that end, patients often make airline and hotel arrangements that limit their stay to a few days. When a new abnormal finding is discovered during a patient visit, additional consultations may be necessary. Examples are asymptomatic intracranial aneurysms, abnormal blood counts, electrocardiogram abnormalities, and asymptomatic intra-abdominal masses found during the course of a medical evaluation. It is not always possible to schedule additional face-to-face appointments within the few days the patient had scheduled for their Mayo Clinic visit. The internal e-consultation allows patients to obtain specialist advice without prolonging their stay or requiring a return visit. The ability of patients to access their Mayo Clinic medical record from their mobile device or online also allows these patients to directly view the results of the e-consultation after they return home.
Compared to e-consultations coming from external sources (outside the Mayo integrated practice) the internal e-consultations are efficient because the shared EMR eliminates the requesting provider burden of data transfer to the specialist and the specialist has access to all the patient data in a familiar searchable format. In addition, requesting providers have immediate access to the e-consultation opinion via the EMR, and specialists do not need to generate separate consultation letters. Our study examines the use and content of these internal e-consultations, the order set that facilitates them, and the extent to which face-to-face consultations enter into the outcome.
Methods
Study setting and design
Mayo Clinic in Rochester, Minnesota is an integrated multispecialty practice with over 1.5 million outpatient visits per year and about 1,860 patient care consultants. The Mayo Clinic primary care practice had 139,000 paneled patients and 260,000 outpatient visits in 2012. In 2008, e-consultations were introduced as a care model to be used by primary care providers to request specialty consultation. By 2010, the value of the e-consultation process was demonstrated and diffused to the entire Mayo Clinic practice allowing specialists to obtain e-consultations from other specialists.
We examined appointment data following e-consultations from January 2012 through June 2013. Face-to-face appointments occurring after the e-consultation but within the same specialty as the e-consultation were defined as an e-consultation face-to-face conversion.
E-consultation process
Orderable e-consultations by specialty.
The e-consultation request is electronically transmitted to the specialty area where the e-consultation is scheduled on the specialist’s appointment calendar. The expectation is that scheduling of e-consultations is to be done within two to three business days. After review of the e-consultation question and medical record information, the specialist enters a formal consultation note in the integrated EMR. As of 2013, all Mayo Clinic specialties were able to receive and respond to e-consultations. At the time of our study, internal e-consultations were restricted to the outpatient practice.
Capture of face-to-face conversion of e-consultations
Mayo Clinic Patient Access Management Analytics queried face-to-face appointments scheduled after each e-consultation patient for 2012 and the first six months of 2013. The e-consultations that subsequently had an outcome of a face-to-face consultation were called face-to-face conversions and we calculated the time interval from e-consultation to face-to-face consultation. We examined 7 medical specialties and psychiatry which represented 8 of 9 highest volume e-consultations for 2013 (Figure 1). We excluded 146 e-consultations in nephrology because they were intraspecialty e-consultations within nephrology so a subsequent appointment with a nephrologist was not a face-to-face conversion.
Counts of e-consultations by responding specialty (interspecialty = e-consultations between different specialties, intraspecialty = e-consultations within the same specialty, generalist = e-consultation referrals from primary care and general internal medicine.
In addition to the dates of subsequent visits, the patient access database also contained information about age, sex, insurance category, requesting provider, and patient distance from clinic. Patient distance from the clinic was categorized as local (residence within 10 counties including and surrounding Rochester, Minnesota), regional (outside the “local” 10 counties but within 500 miles of Rochester), national (within the USA, but greater than 500 miles from Rochester) and international (not in the USA).
Statistical analysis
Face-to-face conversion of an e-consultation was used in a failure time analysis model and patients were censored at the end of the study period (June 30, 2013) if no subsequent face-to-face consultation occurred. We used Kaplan Meier product limit curves and the log rank test for analysis of the e-consultation to face-to-face consult conversion. A Cox proportional hazard model was used to determine explanatory factors for differences in face-to-face conversion. We used JMP 10.0 (SAS institute, Cary, NC) for modeling and statistical analysis.
This study was approved by the Mayo Clinic Institutional Review Board.
Results
There were 3,242 internal e-consultations for the first six months of 2013. We studied 3,008 of these after excluding 234 e-consultations concerning patients who did not have appropriate research consent. E-consultations concerned females 55% of the time and were about 2,885 unique patients of whom 2,769 (96%) had only one e-consultation. Patients age 18 and over accounted for 94% of e-consultations and 32% were for ages 65 and over.
Geographic representation of e-consultations was 47% local (1411), 35% regional (1062), 15% national (459), and 3% international (76). Generalist (primary care and general internal medicine) to specialist e-consultations accounted for 1,521 (50%) while there were 1,343 (45%) interspecialty e-consultations and 5% e-consultations within the same specialty (intraspecialty). Figure 1 shows how the referral types varied by specialty and proportions of referral source varied significantly (generalist referral, interspecialty, and intraspecialty referral, p < 0.0001 for proportions between specialties). There were 28 specialties that performed at least one e-consultation but 10 adult specialties (Figure 1) accounted for 78% of the total e-consultations.
E-consultations were assigned to 353 specialists, and 90 of those specialists were responsible for answering 10 or more e-consultations in the 6 months. Forty specialists handled 1,479 (49%) of the e-consultation volume. There were 850 distinct requesting providers with 480 (56%) ordering no more than 2 in the 6 months. The other 370 (44%) accounted for 85% of the e-consultations.
At least one e-consultation was ordered in the 6 month study period by 96% (49 of 51) of primary care internal medicine providers and 98% (47 of 48) of family medicine providers. Median e-consultation use per provider for the six months was 5 (Mean 6.0, SD 4.0) for internal medicine and 8 (Mean 8.8, SD 6.0) for family medicine. Use in community paediatrics was less with 65% (14 of 22 paediatricians) ordering at least one e-consultation in 6 months and median e-consultations ordered per provider in six months was 1.5 (mean 2.2 SD 2.7).
Top 4 indications for e-consultations by specialty (n = 3008).
There were some noticeable differences in the use of e-consultations by generalists versus specialists. Specialists were proportionately obtaining more e-consultations from hematologists and psychiatrists while primary care providers were more proportionately obtaining gastroenterology and cardiovascular e-consultations. The higher use of gastroenterology e-consultations by primary care providers is probably explained by the number one indication, polyp and cancer screening. At Mayo Clinic, primary care providers are responsible for ordering colon screening exams and follow ups. The higher requests by specialists for e-consultations from psychiatrists is explained by the requirement for bone marrow transplant patients to get e-consultations prior to transplant to assess their stability to go through demanding treatments and medication regimens. 11
Figure 2 shows the exponential growth of e-consultation use by primary care providers from 2009 through 2013 (R2 = 0.99 compared to fitted exponential growth curve).
e-consultations per 1000 panel members primary care with fitted (dashed) exponential growth curve, Q = calendar quarter.
Conversions of e-consultations to face-to-face consultations
We examined 5,334 internal e-consultations occurring from January 2012 through June 2013 for subsequent face-to-face visits in the same specialty. Of those there were eventually 1,111 face-to-face visits identified from the specialties of cardiology (725 e-consultations, 169 face-to-face conversions), endocrinology (765 e-consultations, 178 f2f conversions), gastroenterology (901 e-cons, 160 f2f), haematology (1020 e-cons, 227 f2f), infectious disease (391 e-cons, 29 f2f), nephrology (561 e-cons,107 f2f), psychiatry (543 e-cons, 142 f2f), pulmonary (428 e-cons, 99 f2f). Within 30 days of the e-consultation 11.5% had a specialty face-to-face visit and 17.7% had seen a specialist face-to-face within 90 days of the e-consultation.
Using a Cox proportional hazard model we found the patient’s travel distance from Mayo Clinic, specialty type of e-consultation, and specialist e-consultation experience were explanatory variables for differences in face-to-face conversions (p < 0.001 for each of these three variables). Patient age as a factor for differences in face-to-face conversions did not meet statistical significance (p = 0.19), nor did patient sex (p = 0.11), nor patient insurance category (p = 0.68), nor primary care as the referral source (p = 0.52).
Figures 3 and 4 show the Kaplan Meier product limit curves for face-to-face conversions of e-consultations. The face-to-face conversions stratified by patient distance were significantly different (Figure 3, p < 0.0001). When excluding the national and international patients, there was no significant difference between local and regional patients (p = 0.29). To isolate the effect of type of specialty consultation on face-to-face conversions, we examined only the local and regional patients. Figure 4 shows the Kaplan Meier product limit curves for face-to-face conversions in 8 different specialties. The specialties included the top seven internal e-consultations from the Department of Medicine. Figure 4 shows there were significant differences between specialties in face-to-face conversions even after excluding the national and international patients (p < 0.0001). Gastroenterology and infectious disease e-consultations were the influential specialties of the eight specialty groups responsible for creating the significant differences. There was a cluster of six specialties (cardiology, endocrinology, haematology, nephrology, psychiatry, pulmonary) that were statistically similar when analyzed separately as a group (p = 0.75).
Kaplan Meier curves for face-to-face conversion by patient distance from Mayo Clinic (n = 5334, local = black dashed; regional = gray dotted; national = black solid; international = gray solid). Kaplan Meier curves for face-to-face conversion by specialty responding to e-consultation, local and regional only (n = 3293, Cardiology = black dash dot; Endocrinology = gray dash dot dot; Gastroenterology = solid black; Haematology = gray dot dash; Infectious disease = solid gray; Nephrology = black dots; Psychiatry = gray dashed; Pulmonary = black dashed).
Figure 5 shows the Kaplan Meier product limit curves for face-to-face conversion by specialty experience with e-consultations. After excluding groups of national, international, gastroenterology and infectious disease consultations there were 2,441 e-consultations remaining. Of these there were 1,261 from 26 specialists who had responded to over 50 each, 803 from 50 specialists who had performed 15 to 49 e-consultations, 286 from 51 specialists who had performed between five to 14, and 91 from 55 specialists who had each performed less than five e-consultations. Figure 5 illustrates significant differences (p = 0.01) among specialists when stratified by experience in responding to e-consultations. After excluding specialists who answered fewer than 15 e-consultations, there was no difference in face-to-face conversion rates between specialists answering 15 to 49 compared to those answering 50 and more (p = 0.45). This suggests that experience in answering e-consultations is associated with lower face-to-face conversions but that the experience effect plateaus after responding to 15 or more.
Kaplan Meier curves for face-to-face conversion by experience of specialist responding for 6 specialties, excluding national and international patients, gastroenterology and infectious disease e-consultations (n = 2441 e-consultations, specialists who responded to over 50 e-consultations = black solid; specialists who responded to 15 to 49 e-consultations = black dashed; specialists who responded to 5 to 14 = gray solid; specialists who responded to 1 to 4 = gray dotted).
Discussion
Internal e-consultations were successfully used both by primary care providers and specialists in an integrated multispecialty practice. Almost all primary care, family medicine and internal medicine providers had requested at least one e-consultation in a 6 month period, and the primary care e-consultations per 1,000 panel members continued to grow over an 18 month time period (Figure 1).
The internal e-consultations we describe are substantially different in a number of ways from the e-consultations generally described in the literature. For e-consultations described in the literature, poor access to specialists was a motivating factor. For example, median appointment wait times of over nine weeks was the motivating factor for Ontario, Canada e-consultations. 6 Limited resources for specialty care was another factor motivating e-consultations in the Swinfen Trust and other humanitarian e-consultation networks.5,12 For the Mayo primary care practice, extremely long wait times for specialists and availability of specialty appointments were not major factors influencing internal e-consultations. Specialists were located on the same campus or within a few miles of a primary care clinic; wait times were usually not more than a few days. Another major difference between our internal e-consultations and those in the literature is the information management involved in e-consultations. E-consultations reported in the literature generally had major information transfer barriers. Requesting providers transferred patient records to the specialist by mail, fax, or scanning and uploading to a server. The specialist was dependent on the requesting provider to abstract and send the essential parts of the medical record. Internal e-consultations are not subject to this information barrier; a shared EMR provides requesting providers and specialists complete access to all tests and evaluations. Familiarity with the shared EMR also allows the specialist to save time by being able to efficiently search the patient record and independently review images.
Compared with e-consultations described in the literature, the ordering and responding processes are less burdensome for both the requesting provider and the specialist. Internal e-consultations are ordered electronically in the same manner as diagnostic testing. There is no referral letter, email, or need to access a separate secure server and the clinical question is embedded in the order or included in a clinical note. From the specialist, the only required action is a consultation note in the EMR documenting their response to the e-consultation question; no duplicate letters or correspondences are necessary. Part of the success and continued growth of internal e-consultations at Mayo Clinic is that they are integrated into established care flow processes.
Many of the advantages of internal e-consultations are dependent on a shared EMR and an electronic ordering system. Mayo Clinic currently has the infrastructure ingredients to make the e-consultation process relatively seamless and efficient. 11 In the future, fully interoperable EMRs will likely be widespread and make e-consultations an option for providers in other healthcare systems.
In the US, insurers have generally not paid for internal e-consultations. However, if internal e-consultations can be low-cost substitutes for expensive face-to-face consultations, insurers will want to support their use. Quantifying the face-to-face conversions of these e-consultations is a necessary step in determining the value to an insurer since a high face-to-face conversion rate would result in duplication of work and higher cost.
Questions about interpreting laboratory results and images comprise a large volume of the e-consultation indications. Management questions are more limited and occur in osteoporosis, hypertension, psychiatric disorders, and headache where physical exam findings are not frequently used for management decisions. Demand for dermatology e-consultations are noticeably absent from our internal e-consultation volumes even though they ranked as the number one e-consultation in the Ontario, Canada experience (20% of all e-consultations). Without the lengthy (up to 12 months) wait time for dermatology appointments as noted in Keely’s Canadian e-consultation study 6 and with a dermatologist nearby, internal dermatology e-consultations have not been a priority at Mayo Clinic.
While a large percent of the e-consultations did not have a follow-up face-to-face consultation even a year later, 18% did have a face-to-face specially visit within 90 days after the e-consultation. The e-consultations that convert to a face-to-face consultation do so in a time-dependent fashion (Figure 3 and 4). Considering differences among the specialties in e-consultation indications (Table 1), it was interesting that there were no statistical differences in the face-to-face conversion curves for six of the eight specialties. For example, haematology and infectious disease have no restrictions on e-consultation indications, while other specialties limited the indications for their e-consultations (Table 1). Despite allowing e-consultations of any indication, there were no greater face-to-face conversions for haematology than other specialties with restricted indications. Infectious disease e-consultations, which also had no restrictions, actually had fewer face-to-face conversions than the other specialties. Our findings suggest that restricting e-consultations to specific indications does not necessarily result in lower face-to-face conversions.
National and international patients had lower rates of face-to-face conversion than local and regional patients. Patients living at a distance would have to make additional travel arrangements, making a face-to-face visit less attractive to them compared to someone living closer. We did not have the capability to quantify how many patients obtained specialist face-to-face visits elsewhere, so we do not really know if the overall rate of face-to-face conversion of these patients was less than or greater than those closer to Mayo Clinic.
It was not immediately apparent why gastroenterology and infectious disease e-consultations were associated with lower rates of face-to-face conversions. We do know that gastroenterology and infectious disease specialties receive higher numbers of curbside consultations compared to other specialties. 13 Perhaps higher use of curbside consults has given providers more experience and insight on questions that gastroenterologists and infectious disease specialists want to reserve for face-to-face visits. Another possibility is that, especially in the case of gastroenterology e-consultations, some e-consultation topics may almost never need a face-to-face consultation. For example, during the time of the study, certain colonoscopy findings generated e-consultations. Providers had questions about follow-up for patients having multiple polyps or sessile serrated adenomas. Twenty one percent of the e-consultation questions concerned these questions about follow-up of polyps or colon cancer screening (Table 2), but these e-consultation questions could be answered by gastroenterologists just by looking at the colonoscopy report and associated pathology. These particular questions in gastroenterology appear to decrease the overall face-to-face conversion rate of gastroenterology e-consultations because they result in no face-to-face conversion. In fact, after the data was collected for this study, gastroenterologists changed their colonoscopy reports; they included several new paragraphs of instructions explaining tailored follow-up according to the number of polyps and pathology. Perhaps this change to colonoscopy reports will result in fewer straight forward gastroenterology e-consultations that lower the overall face-to-face conversion rate. Future research is needed to determine whether face-to-face conversion rates change with introduction of guidelines that answer high frequency e-consultation questions.
This study suggests that experience in responding to e-consultation requests lowers the face-to-face conversion rate. Figure 5 shows that specialists who had responded to 15 or more e-consultations had lower face-to-face conversion rates. However, those who had responded to more than 50 e-consultations did not have significantly lower e-consultation conversions than specialists who had authored between 15 and 49. Based on this finding, it appears to take no more than 15 e-consultations to maximize the value of e-consultation experience on face-to-face conversions. Provider education was focused on the requesting process and minimal education was provided to specialists on formulating a response. It is unknown whether additional training aimed toward specialists would decrease conversions.
Further study is necessary to understand why face-to-face visits are ordered after e-consultations. A preliminary examination of e-consultations that converted to a face-to-face consultation within one week suggests that this early conversion is often driven by the specialist receiving the e-consultation. In these cases the specialist often considered the question too complex to provide an answer without a face-to-face visit. We found several examples of a phone call from the specialist to the requesting provider explaining the complexity of the issue, and suggesting the provider order a face-to-face consult.
Within specific e-consultation indications there may be some variability in face-to-face conversions due to specialist, requesting provider, and patient factors. For example, we know that providers vary in the way they construct clinical question for consults. 14 It is unknown if the way the clinical question is presented influences the specialists’ recommendations for face-to-face visits. Requesting providers may also vary in how skillfully they communicate and implement e-consultation recommendations. These factors may also have a role in the face-to-face conversion rate.
Recently, Wootton and co-authors established a framework for evaluating the quality of e-consultations. 15 They developed a quality assessment questionnaire based on key quality attributes of the e-consultation process and value to the patient and requesting provider. Quality issues that they identified included adequacy of information from the requesting provider, timeliness of the specialty response, and helpfulness in management of the patient. In addition to developing specific quality measures for e-consultations, Wootton et al. have also been examining ways to collect and manage referring provider feedback from e-consultations. 16 This research on quality measures and user feedback is an important and critically needed step as the e-consultation process matures and disseminates. Wootton et al. found that 95% of referrers found an educational benefit to the e-consultation responses. This educational benefit should be taken into consideration when interpreting our results. E-consultations that convert to a face-to-face visit can still educate the referring provider and should not arbitrarily be viewed as a failure. Future changes to the e-consultation system will need to be guided by quality measures and user feedback.
A strength of this study is that the adoption, use, and face-to-face conversions of e-consultations were relatively free of confounding interests. All Mayo Clinic physicians are salaried and had no major direct or indirect incentives to initiate or perform internal e-consultations. There also were no major information sharing barriers. With internal e-consultations, specialists and requesting providers use a shared EMR so there are no medical record gaps that specialists have to fill in. There are required specific indication fields for some specialties (Table 2) as well as a required clinical question text field for every e-consultation. This means that specialists are rarely in the dark about the specific question asked by the requesting provider.
A limitation of our study is that our internal e-consultations used a shared EMR with complete interoperability. Practices with limited interoperability of medical records and orders will likely have different uptakes and use. Our results may also be dependent on the use of the specific orderable e-consultation list (Table 1). The orderable indications provided here might help others implement the internal e-consultation but organizations using less restricted internal e-consultations might have different outcomes.
Footnotes
Acknowledgements
We thank Elissa Nelson, Eleanor Wintheiser and Anthonia Igbinovia for their contributions to the work.