Paging the eCardiologist: insights into referral behaviour of primary care physicians from qualitative analysis of a cardiology eConsult service

Abstract

Objective

The purpose of this study is to investigate determinants of primary care physician cardiology referrals by performing qualitative analysis of questions asked by primary care physicians in cardiology electronic consultation services (eConsults).

Setting

A health region in eastern Ontario, Canada, where primary care providers have had access to an eConsult service since 2010.

Participants

We included all consecutive cardiology eConsults initiated by registered primary care provider users of our eConsult service and who initiated one or more eConsult between July 2014 and January 2015. We excluded eConsults in which the primary care provider attached a document without asking a question. A convenience sample of 100 consecutive eConsults initiated by 61 primary care providers was analysed after excluding 14 eConsults.

Primary and secondary outcome measures: Primary care provider eConsult questions are categorised into thematic categories based on the constant comparison method of qualitative analysis with external validation by content experts. Secondary outcomes include sample primary care provider eConsult questions to illustrate each theme and any emergent subthemes.

Results

Thematic saturation occurred after analysis of 30 eConsults. An additional 70 eConsults were coded with no new emergent themes. Themes include exceptions to clinical guidelines (n=13), non-cardiac treatment in a cardiac patient (n=13), specific investigation/management (n=18), interpretation of diagnostic testing (n=46), clinical concerns despite normal testing (n=4) and screening for positive family history (n=6). Subthemes include multiple comorbidities and mild abnormalities on cardiac tests.

Conclusions

We report categories of clinical questions that drive primary care provider cardiology eConsults. Multimorbidity leads to cardiology eConsults as primary care providers try to apply treatment guidelines in medically complex patients. Mild test abnormalities unrelated to clinical problems commonly lead to cardiology eConsult requests. Further research is needed to determine how guidelines can better account for multimorbidity, and how cardiologists can better communicate with primary care providers to put cardiac test results in clinical context.

Keywords

Referral and consultation cardiology primary healthcare

Background

The Champlain BASE (building access to specialist through eConsultation) electronic consultation service (eConsult) was launched in 2010 in order to improve access to specialist advice for primary care providers (PCPs). The eConsult service is an asynchronous, secure web-based communication platform where PCPs submit patient-specific clinical questions to specialists. Several groups have demonstrated that eConsults are safe, cost-effective and improve timeliness of access to specialist advice.^1–3 Overall, PCPs report high levels of satisfaction with eConsults.² Classification of PCP eConsult questions reveals that most eConsult questions relate to general management, appropriate diagnostic work-up and interpretation of abnormal test results.^1,4–6 While preliminary research suggests that gaps in PCP knowledge may lead to eConsult questions, additional factors may influence PCP decisions to initiate an eConsult.

As the gatekeepers of the healthcare system, much interest has focused on referral behaviours of PCPs. Variation in PCP referral behaviour has implications for downstream resource utilisation and healthcare expenditure.⁷ Analysis of diagnosis frequency and specialist referral rates in a large PCP clinical database reveals that PCP decisions for specialist consultations relate to patient, physician and healthcare system factors.⁸ Specialist referrals are more likely for patients with a disease that is not common in PCP practice, and for patients with comorbidities.⁹ Given the increasing prevalence of chronic disease in clinical practice, these findings suggest that PCPs may increasingly request specialist consultations, and this has implications for healthcare resource utilisation.¹⁰

Our eConsult system logs all PCP questions and all specialist responses, resulting in the accumulation of a large database of qualitative data. The purpose of this study is to use qualitative analysis of cardiology eConsult questions to investigate the determinants of cardiology eConsults by PCPs in our health region.

Methods

This is a retrospective, qualitative analysis of 114 cardiology eConsults completed between July 2014 and January 2015. This study has been approved by the Ottawa Hospital Research Ethics Board. The majority of cases are from the Champlain Local Health Integrated Network (LHIN) a large health region in eastern Ontario, Canada. Participants include PCPs who are registered users of the eConsult service and who initiated at least one electronic consultation between July 2014 and January 2015. The unit of analysis is the PCP’s reason for consultation. In 14 entries, the PCP did not ask a specific clinical question as a reason for consultation and only stated: ‘please see attached’ with attachments of abnormal investigations. Although it can be deduced that the PCP’s intention for the eConsult was to obtain a second opinion on the interpretation of these investigations, such eConsults without a specific clinical inquiry were excluded from our analysis.

PCP eConsult questions were analysed using inductive and deductive content analysis.¹¹ In order to ensure representativeness, we sampled our data by analysing consecutive cases starting from the most recent eConsult in January 2015 and by working backwards. The qualitative analysis process is illustrated in the flow diagram in Figure 1.

Figure 1.

Flow diagram illustrates the qualitative analysis process including inductive analysis of 30 eConsults and deductive analysis of an additional 70 eConsults.

Initial inductive analysis was performed by two investigators (EC, CBJ), who performed open coding of eConsults. Similar cases were grouped into emergent themes and a coding frame for categorisation was developed. Constant re-evaluation, testing and refinement occurred throughout data collection. When no new themes emerged, a final set of themes was defined. This thematic saturation occurred after analysis of 30 eConsults. External validation was performed by having content experts (NG, MT, AS) review the identified themes to confirm results of our inductive analysis. After confirmation of themes identified by inductive analysis, deductive analysis of an additional 70 eConsults was performed by a single investigator (EC) for a total sample of 100 eConsults. During deductive analysis of the data, no additional themes were identified.

Results

Between July 2014 and January 2015, there was a total of 114 cardiology eConsult entries that were initiated in the Champlain BASE electronic service. Fourteen eConsults were excluded due to missing information about the reason for consultation. There was a total of 100 eConsults initiated by 61 different PCPs, of whom 92% were family physicians and 75% were women (Table 1). Women made up 47% of the patient population and the average age of patients was 59.2 years (Table 2). The flow of cardiology e-consultations through qualitative analysis is depicted in Figure 2.

Table 1.

Characteristics of primary care providers who are registered to use the eConsult service and submitted at least one eConsult between 12 July 2014 and 28 January 2015.

		PCP title		PCP location		Average years
Characteristics	Distribution	MD	NP	Urban	Rural	of service
Men	25% (15)	93% (14)	7% (1)	93% (14)	7% (1)	15.3
Women	75% (46)	91% (42)	9% (4)	89% (41)	11% (5)	14.2
Total	100% (61)	92% (56)	8% (5)	90% (55)	10% (6)	14.5

PCP: primary care provider; NP: nurse practitioner.

Figure 2.

Flow diagram Illustrates six themes and two subthemes that emerge from inductive and deductive analysis of 100 eConsults.

Table 2.

Patient demographics: gender and age.

Characteristics	Distribution	Average age (years)
Men	53%	59.2
Women	47%	59.1
Total	100%	59.2

PCP: primary care provider; NP: nurse practitioner.

Our qualitative analysis identified six major themes: (a) exceptions to clinical guidelines (13 cases); (b) non-cardiac treatment in a cardiac patient (13 cases); (c) specific investigation/management questions (18 cases); (d) interpretation of diagnostic testing (46 cases); (e) clinical concerns despite normal testing (four cases); and (f) screening for positive family history (six cases). Results are presented in these six categories and are illustrated by examples from e-consultation entries.

Theme 1: Exceptions to clinical guidelines

There was a total of 13 cases of PCP eConsult questions that related to the applicability of clinical guidelines. PCPs often asked about the management of multiple comorbidities, especially when several different guidelines applied. A typical example of such an eConsult question is the concomitant management of atrial fibrillation and other conditions.

Reason for consultation: question regarding cotreatment of ASA [acetylsalicylic acid] and Pradaxa in treatment of atrial fibrillation with history of TIA [transient ischaemic attack]. My patient is a heavy drinker. My question is whether the patient should on both ASA and Pradaxa? Could this be more harmful for bleed given his drinking history?

PCPs demonstrated they are familiar with the treatment guidelines and are comfortable with initiating treatment for a single disease. However, when faced with a patient with multiple conditions that require concomitant treatment, PCPs asked how to apply treatment guidelines in clinical contexts when patients have multiple diagnoses.

Patient is on Rivaroxaban for atrial fibrillation and undergoing prostate cancer radiation treatment. ASA was discontinued due to risk of bleed. Concerned about increased angina. Is it recommended to restart ASA due to increasing episodes of exertional chest pain or does the risk of bleed continue to outweigh benefit.

Patient is 82 years old with type 2 diabetes, hypothyroidism, chronic kidney disease, cirrhosis, lymphoma in remission, presents with new onset atrial fibrillation with heart rate below 95 bpm. On Aggrenox since a stroke in 2001. My question is how best to manage the atrial fibrillation regarding anticoagulation.

In other instances, PCPs asked about the applicability of certain treatment guidelines when this is not clearly defined. They questioned the need for treatment when the condition seems to be triggered by a reversible cause, or when the condition seems transient. Such aspects are not defined in current guidelines.

His CHADS2 score is 3, CHADSVASc score is 5, and HASBLED score is 3. There is no prior history of atrial fibrillation and a logical etiology of the recent atrial fibrillation is pneumonia. The question is whether to keep this patient on Apixaban given the bleeding risk?

Longest run of atrial fibrillation on the loop monitor was 16 seconds. The CHADSVASc score is 2 = 2.2% stroke rate per year. My question for you is if you see atrial fibrillation in such brief runs as this, do you consider the annual stroke risk to be even lower? Would you prescribe ASA in this situation?

Certain PCPs asked how to interpret risk stratification scores appropriately, in the presence of additional comorbidities that are unaccounted for by the original risk assessment tool.

Recently found to have a ten year cardiovascular risk of 14% based on current lipid values. While I have started low-dose statin, the patient is wondering if this is required since the risk estimation model does not take into account the patient’s treatment with an anticoagulant for venous thromboembolism. I would appreciate your guidance.

Framingham risk estimate for this patient dictates no need for statin, but I am not sure regarding the elevated level of Lp(a) [lipoprotein a]. Does this finding mean we should double the risk as we do for family history of premature coronary disease. In the past, I was told to start statin if LDL [low-density lipoprotein] greater than 3.5 in this situation. Your recommendation appreciated.

Theme 2: Non-cardiac treatment in a cardiac patient

PCP eConsult questions related to the safety of starting a non-cardiovascular drug in the setting of known cardiac disease in 13 cases of the entries studied. PCPs tended to seek advice from a specialist when they want to initiate treatment for a non-cardiac condition, but were hesitant, for fear of exacerbating a concomitant cardiac condition, such as a heart block or a known arrhythmia. A common example of this type of eConsult question is the safety of starting or up-titrating psychiatric medications in a patient with known arrhythmias.

My question is, since the psychiatric symptoms are complex and somewhat difficult to control, how significant is this prolonged QT? The two medications most likely to be responsible for his prolonged QT are mirtazapine and clomipramine. I am hesitant to decrease these given recent psychiatric symptoms but will do so if you think this QT puts the patient at risk.

…We would like to start the patient on a selective serotonin reuptake inhibitor. Would appreciate your guidance as to whether or not there are any concerns in doing this in the context of heart block?

The patient is currently not treated for attention deficit hyperactivity disorder, however they are quite symptomatic and I feel that appropriate treatment would dramatically improve mood, anxiety, and function. There is a history of Wolff–Parkinson–White syndrome. I am just wondering if it is safe to use a stimulant in this situation or would it be contraindicated?

Theme 3: Specific investigation/management questions

In 18 cases, PCPs had specific investigation or management questions. They often sought specialist advice when encountering more challenging clinical cases in which the diagnosis is unclear despite extensive investigations.

My patient is 84 years old scheduled for hip surgery. There is a history of cough for 6 months and respirology investigations are normal. Recently started to complain of increasing dyspnea, with or without cough, on stair climbing and walking, that takes 3–4 minutes to resolve. No chest pain. No waking at night with dyspnea. I am wondering if it is an angina variant and am wondering the best test to see if there is coronary disease.

Patient with history of asymptomatic Wenkebach and Mobitz 2 with 2:1 block on Holter in the past has been having occasional episodes of chest pain. Pain lasts for 5–10 min, occurs at the end of the day, both on walking and at rest. They resolve spontaneously, occasionally by taking deep breaths. No orthopnea, no paroxysmal nocturnal dyspnea, no palpitations, no nausea, vomiting or sweating. Your advice regarding further management of this would be very much appreciated.

In other instances, PCPs asked about management of medication side effects and seemed to question the utility of such medications in the context of the patient’s age and other co-morbidities.

This 60 year-old has been taking low dose ACE [angiotensin-converting enzyme) inhibitor daily for cardioprotection since a prior coronary stent. I understand that this is normally a long term medication. My only concern is that the BP [blood pressure] is quite low, from 90/50 to 120/70. No complaints of orthostatic dizziness. Do you recommend continuing the ACE inhibitor?

This 64 year-old had a stent and was put on a statin, and recently complained of leg cramps and pain so we stopped the statin although CK [creatine kinase] was normal. Symptoms have improved and I did lipids off statin. The results are good with a cholesterol fasting 4.3 mmol/L, an HDL [high-density lipoprotein] of 2.05 mmol/L LDL cholesterol fasting 1.5 mmol/L non-HDL cholesterol fasting 2.2 mmol/L, total cholesterol:HDL fasting ratio 2.1, triglycerides fasting 1.57 mmol/L. The patient prefers not to restart a statin. However, given the history, even though the lipids are good I was unsure, given the anti-inflammatory properties, whether statin is needed or not.

Theme 4: Interpretation of diagnostic testing

PCP eConsult questions related to the interpretation of diagnostic testing and the significance of mildly abnormal findings in 46 cases. When faced with diagnostic tests with mildly abnormal results, PCPs often sought reassurance from cardiologists that such findings are not related to the patient’s physical complaints. Common examples of this type of eConsult question are complaints of palpitations and a finding of left atrial enlargement on echocardiogram, or complaints of lightheadedness with a finding of first degree atrioventricular (AV) block on Holter.

This patient is 64 years old, 6 weeks ago had 3 episodes of light headedness, lasting several hours at a time. No decreased level of consciousness, no headache, no visual changes, no chest pain, no dyspnea. An ECG was ordered and the patient was found to have bradycardia at 50 bpm and 1st degree AV block, but otherwise normal. A holter monitor was ordered which shows the bradycardia and first degree AV block. Previous echo is normal. Based on the holter monitor, does the patient need any further investigations?

Initially tests were ordered when patient was having episodes of palpitations and shortness of breath. These episodes have since resolved. The echocardiogram shows severely dilated left atrium. The last echo several years ago was normal. At this time I am uncertain what to do regarding the dilated atrium.

In other instances, PCPs encountered mildly abnormal test results in an asymptomatic patient and asked about appropriate work-up or follow-up. Common examples include left atrial enlargement, mitral valve prolapse or mild mitral regurgitation on echocardiogram, in the absence of symptoms.

This is a pleasant 70 year old with established coronary artery disease with a stent several years ago. On cardiac CT [computed tomography] angiogram, a minimally dilated ascending aorta measuring up to 3.8 cm in diameter was found. My question to you is what sort of follow-up is needed for enlarged ascending aortas, if any.

This patient is healthy and active with a sibling recently diagnosed with bicuspid aortic valve. An echo was done to rule out bicuspid valve, which the patient does not have, however the echo reports prolapse of the posterior mitral leaflet. Is any further follow up recommended given this finding?

This 71 year old patient with previous stent has a new right bundle branch block. Could you please advise what the next steps might be? The patient is asymptomatic.

Theme 5: Clinical concerns despite normal testing

There was a total of four eConsult questions which related to clinical concerns despite normal testing. PCPs often sought cardiology advice when faced with patients’ ongoing clinical symptoms despite normal tests. A typical example of this type of eConsult question is ongoing presyncope despite normal ECG and laboratory tests.

62 year-old patient walked a short distance after a bike ride and became dizzy. The patient managed to sit down but then had a witnessed syncopal episode lasting approximately 1 minute. An exercise stress test was done which was normal except for a transient decrease in BP in the recovery phase and an echo was recommended. Echo showed: normal ejection fraction, mild mitral valve prolapse and mild mitral regurgitation. Do you recommend further investigations?

51 year old patient is having presyncopal episodes. An ECG reveals isolated premature ventricular beats. TSH [thyroid-stimulating hormone], CBC [complete blood count], HbA1c [haemoglobin A1c] are normal. I'm not sure what is triggering these presyncopal events. I'm thinking that because of the warning symptoms, they are less likely to be neurologic. So are they simply vasovagal? What are your thoughts?

In other instances, the patient’s symptoms seemed to be related to a more benign, non-cardiac aetiology and PCPs sought confirmation from specialists of their own clinical judgement that further work-up was not necessary.

This 17 year old patient felt dizzy, looked pale, and then fainted. The dizziness occurs most often when not eating or drinking properly and if they get up suddenly. In ER [emergency room], the patient was given fluids and felt better. Normal ECG in the emergency department. My concern is that there is a pulse difference of 41 with postural vital signs today. BP lying 112/61 heart rate 67, BP standing after 2 minutes 110/70, heart rate 108. Asymptomatic during postural vital signs but mentioned slight dizziness after cardiac exam. Based on the wide pulse difference and history of recurrent vasovagal symptoms, would you recommend any further investigations?

Theme 6: Screening in positive family history

PCP eConsult questions related to screening due to family history in four cases. PCPs asked about screening recommendations and appropriate follow-up for their asymptomatic patients who have a strong positive family history for structural cardiac abnormalities or arrhythmias.

Father diagnosed with hypertrophic cardiomyopathy, no records available. Patient is asymptomatic. I have attached the ECG. Can I do further work-up with echo and if normal does the patient still need to see cardiology?

This is a 51 year old patient whose sibling died suddenly at 44 years of age from a dilated cardiomyopathy. No cardiac symptoms, and screening revealed a normal ECG and echocardiogram. Please advise as to whether any further cardiac screening now or in the future is needed.

Discussion

Our qualitative analysis of PCP cardiology eConsult questions has identified themes related to why PCPs seek clinical advice from a cardiologist. Multimorbidity and mild cardiac test abnormalities are important determinants of PCP cardiology eConsults.

Multimorbidity

Similar to previous research, clinical management questions are important themes in cardiology eConsults.^4–6 Many PCPs ask if treatment guidelines apply in clinical contexts where patients have other medical diagnoses, such as atrial fibrillation during pneumonia. Patients with multiple comorbidities often present a particular challenge to PCPs because several guidelines can outline contradictory recommendations, or concomitant treatment of two conditions can predispose the patient to adverse events or side effects. PCPs ask for guidance when applying treatment guidelines for more than one cardiac diagnosis, such as the appropriateness of combining anticoagulant and antiplatelet therapy in patients with atrial fibrillation and coronary disease. Finally, PCPs are concerned about how a cardiac diagnosis may influence the safety of non-cardiac treatment, such as stimulant drugs for attention deficit hyperactivity disorder in patients with cardiac disease. Multimorbidity is a common subtheme and is an important driver of PCP cardiology eConsults. These results reflect the real world nature of clinical problems faced by PCPs in our region. In Canada, seniors with three or more chronic conditions use 40% of healthcare resources, and 63% of seniors take five or more different types of medications.¹² Treatment guidelines are based on randomised trials in which multimorbidity patients are often underrepresented.^13–15 Investigators have called for guideline writers to consider how multimorbidity influences the ability of PCPs to implement treatment recommendations.^13,14 Our results highlight the importance of efforts to consolidate guidelines across multiple specialities into comprehensive treatment approaches, as has been undertaken by Canada’s C-Change initiative, as our results suggest that PCPs are struggling to implement guidelines in their multimorbidity patients.¹⁶

Mild cardiac test abnormality

As with previous research evaluating eConsults, we identified diagnostic test results as an important theme.^4,5 In particular, PCPs often ask for specialist advice when a patient’s clinical issue does not match the results of mildly abnormal cardiac investigations. Common examples include brief runs of atrial tachycardia on a Holter monitor in the absence of clinical symptoms; mild left atrial enlargement on echocardiography in a patient with syncope; and left anterior fascicular block in a patient with shortness of breath. PCPs seek reassurance that the test abnormality does not explain the clinical problem, and they ask for advice about what action should be taken in response to mildly abnormal cardiac test results. Previous research has shown that mildly abnormal tests are common and may not represent a pathological disease process.^17,18 This is important in older patients, in whiom age-related changes may result in mildly abnormal cardiac testing.19 Mild test abnormalities with no clinical or prognostic significance lead to PCP cardiology eConsults. Further research is needed to determine how cardiac test reporting influences PCP decision-making.

Strengths and limitations

Our results are in keeping with previous studies, which examined doctors’ reasons for referral and their perceptions of the factors that influence their clinical decisions. The work of Bowling et al. revealed that PCPs were more likely to refer patients to cardiologists due to ‘uncertainty of diagnosis and treatment’ as well as due to the difficult management of certain symptoms.²⁰ Although the study employed semistructured interview methods in order to explore physicians’ perceptions of factors that influence their clinical decisions, the authors did not further elucidate the factors that prompt the physicians’ reported reasons for referral in the first place. Indeed, the unique and complex nature of each clinical scenario encountered in a PCP’s office and the factors that cause a physician to be uncertain of a situation and decide to seek specialist advice has not previously been well studied. Furthermore, in the study of Bowling et al., data were collected from self-reporting through interviews with physicians. This introduces social desirability bias that is not encountered in our current study in which there was objective analysis of the reasons of referral from PCPs.

Previous works examining the eConsults database have found similar recurrent themes in PCP enquiries to specialists.^1,4,6 However, to our knowledge, this is the first qualitative analysis looking at eConsults data in order to define the motive behind the questions and to determine the factors that influence the PCP’s decision to seek cardiology advice. While we chose to analyse a convenience sample of only 100 eConsults, we chose the most recent 100 eConsults rather than the initial 100 e-consultations when the eConsult service was first established. In the initial months that this electronic service was first introduced, PCPs and specialists alike may not have been familiar or as well versed with the functions of the database, which may in turn have affected the thoroughness of the PCPs’ electronic consultations. By sampling the most recent 100 eConsults, we were able to ensure that the collection of data is most representative of contemporary PCP practice. Our use of qualitative data gathered by an eConsult platform to gain insight into referral behaviour of PCPs is a novel use of an emerging healthcare technology, and we believe there is the potential to use these data to answer other clinical questions related to PCP specialist referrals.

Due to the retrospective nature of this study, respondent validation was not applicable. However, in order to ensure external validity, our inductive analysis results were verified by a panel of content experts which included two cardiologists: NG and MT, as well as one internal medicine resident who has experience in eConsults research: AS.

The eConsults included in our analysis were limited to the Champlain LHIN, thus making the generalisability of our results difficult. We have, however, found that results from our analysis are in agreement with previous research studying eConsults in different health networks,⁵ as well as eConsults of different specialties.^1,6 Furthermore, the intention of this study was to shed light on clinical factors that may influence PCP’s decision to refer to cardiologists and to provide insight on how these can be addressed in future research. Some Canadian cardiology guidelines have been updated since 2014–2015 when PCPs generated the eConsults that we used as a convenience sample. However, revised guidelines do not address multimorbidity and as such we believe that our findings remain relevant.

Conclusion

We have identified themes that drive PCP cardiology eConsult questions. Multimorbidity leads to cardiology eConsults as PCPs try to apply treatment guidelines in medically complex patients. This finding should prompt guideline writers to consider how multimorbidity influences the ability to implement treatment guidelines. Mild test abnormalities unrelated to clinical problems commonly lead to cardiology eConsult requests. Further research is needed to determine how cardiologists can better communicate with PCPs to put cardiac test results into appropriate clinical context.

Footnotes

Acknowledgements

The authors would like to thank Amir Afkham for his expert technical assistance in supporting the Champlain BASE eConsult service and for his technical support in accessing data for our research, and Lesley Ananny for her assistance with database management.

Conflict of interest

None declared.

Contributorship

This manuscript has been read and approved by all authors. All persons listed as authors have contributed to preparing the manuscript, and no person or persons other than the authors listed have contributed significantly to its preparation. Details of contributorship for each author appear below:

Elizabeth Chan: In collaboration with Archibald, Johnson and Liddy helped design the qualitative methodology for the study, performed acquisition, analysis and interpretation of all study data; wrote the first draft of the paper and participated in all subsequent revisions, approves the final draft, and agrees to be accountable in all respects for the accuracy of our work.

Christopher Johnson: Initial concept of using eConsult questions to investigate referral behaviour of PCPs, in collaboration with Archibald, Chan and Liddy designed the qualitative methodology for the study, performed acquisition of study data for qualitative analysis of the first 30 cases with Chan, analysis and interpretation of qualitative data, wrote multiple revisions of the manuscript in collaboration with all authors, approves the final draft, and agrees to be accountable in all respects for the accuracy of our work.

Clare Liddy: Initial concept of using eConsult questions to investigate PCP referral behaviour, helped design the qualitative methodology for the study in collaboration with Archibald, Johnson and Chan, interpreted study findings, participated in drafting multiple revisions, approves the final manuscript, and agrees to be accountable in all respects for the accuracy of our work.

Nadine Gauthier: Acquisition of study data by analysing 30 eConsults to verify the validity of emergent themes, analysed results of qualitative analysis, participated in drafting multiple revisions, approves the final manuscript, and agrees to be accountable in all respects for the accuracy of our work.

Erin Keely: In collaboration with Johnson and Turek, initial concept of using eConsult questions to investigate PCP referral behaviour, interpreted findings from qualitative analysis, participated in drafting multiple revisions, approves the final manuscript, and agrees to be accountable in all respects for the accuracy of our work.

Michele Turek: In collaboration with Johnson and Keely, initial concept of using eConsult questions to investigate referral behaviour of PCPs, acquisition of study data by analysing 30 eConsults to verify validity of emergent themes, participated in drafting multiple revisions, approves the final manuscript, and agrees to be accountable in all respects for the accuracy of our work.

Alborz Shoki: Acquisition of study data by analysing 30 eConsults to verify the validity of emergent themes, participated in drafting multiple revisions, approves the final manuscript, and agrees to be accountable in all respects for the accuracy of our work.

Douglas Archibald: Designed the qualitative methodology for the study, interpreted findings of qualitative analysis, participated in drafting multiple revisions, approves the final manuscript, and agrees to be accountable in all respects for the accuracy of our work.

Ethical approval

The ethics committee of the Ottawa Hospital Research Institute approved this study.

Funding

The Champlain BASE electronic consultation service (eConsult) is supported by the Ontario Ministry of Health and Long-Term Care, the Ottawa Hospital Academic Medical Organization Innovation Fund, eHealth Ontario and the Champlain Local Health Integration Network. C.B. Johnson and E. Keely are supported by the University of Ottawa, Department of Medicine.

Guarantor

C.B. Johnson.

Peer Review

This manuscript was reviewed by Mariotti Guiliano, LHU APPS, Referrals Service, Trento, Italy, and John Banks, University of Durham, Durham, UK.

References

Fogel

Khamisa

Afkham

et al . Ask the eConsultant: improving access to haematology expertise using an asynchronous eConsult system. J Telemed Telecare 2016; 22: 1–7.

Liddy

Afkham

Drosinis

et al . Impact of and satisfaction with a new eConsult service: a mixed methods study of primary care providers. J Am Board Fam Med 2015; 28: 394–403.

Olayiwola

Anderson

Jepeal

et al . Electronic consultations to improve the primary care-specialty care interface for cardiology in the medically underserved: a cluster-randomized controlled trial. Ann Fam Med 2016; 14: 133–140.

Shoki

Johnson

Liddy

et al . Using eConsults to inform cardiology continuing medical education needs of primary care providers. Oral presentation, 2015 Canadian Cardiovascular Congress, Toronto, On, Canada.

Wrenn

Catschegn

Cruz

et al . Analysis of an electronic consultation program at an academic medical centre: primary care provider questions, specialist responses, and primary care provider actions. J Telemed Telecare 2016; 22: 1–8.

Tran

Liddy

Liu

et al . eConsults to endocrinologists improve access and change primary care provider behavior. Endocr Pract Off J Am Coll Endocrinol Am Assoc Clin Endocrinol 2016; 22: 1145–1150.

Fang

Liu

Rizzo

JA.

Has the use of physician gatekeepers declined among HMOs? Evidence from the United States.

Int J Health Care Finance Econ 2009; 9: 183–195.

Forrest

Nutting

von Schrader

et al . Primary care physician specialty referral decision making: patient, physician, and health care system determinants. Med Decis Mak Int J Soc Med Decis Mak 2006; 26: 76–85.

Forrest

Reid

RJ.

Prevalence of health problems and primary care physicians’ specialty referral decisions.

J Fam Pract 2001; 50: 427–432.

10.

Dall

Gallo

Chakrabarti

et al . An aging population and growing disease burden will require a large and specialized health care workforce by 2025. Health Aff Proj Hope 2013; 32: 2013–2020.

11.

Vaismoradi

Turunen

Bondas

Content analysis and thematic analysis: Implications for conducting a qualitative descriptive study.

Nursing Health Sciences, 2013; 15: 398–405.

12.

Primary Health Care Information program. Seniors and the Health Care System: What is the Impact of multiple Chronic Conditions? (cited 3 October 2016). https://secure.cihi.ca/free_products/air-chronic_disease_aib_en.pdf (accessed 16 August 2018).

13.

Boyd

Darer

Boult

et al . Clinical practice guidelines and quality of care for older patients with multiple comorbid diseases: implications for pay for performance. JAMA 2005; 294: 716–724.

14.

Uhlig

Leff

Kent

et al . A framework for crafting clinical practice guidelines that are relevant to the care and management of people with multimorbidity. J Gen Intern Med 2014; 29: 670–679.

15.

Shekelle

Woolf

Grimshaw

et al . Developing clinical practice guidelines: reviewing, reporting, and publishing guidelines; updating guidelines; and the emerging issues of enhancing guideline implementability and accounting for comorbid conditions in guideline development. Implement Sci IS 2012; 7: 62.

16.

Tobe

Stone

Brouwers

et al . Harmonization of guidelines for the prevention and treatment of cardiovascular disease: the c-change initiative. CMAJ 2011; 183: E1135–E1150.

17.

Brigden

Heathcote

JC.

Problems in interpreting laboratory tests. What do unexpected results mean?

Postgrad Med 2000; 107: 145–146, 151–152, 155–158 passim.

18.

Houben

PHH

Winkens

RAG

van der Weijden

et al . Reasons for ordering laboratory tests and relationship with frequency of abnormal results. Scand J Prim Health Care 2010; 28: 18–23.

19.

Leibowitz D, Stessman-Lande I, Jacobs J, et al. Cardiac Structure and Function in Persons 85 Years of Age. Am J Cardiol 2011; 108(3): 465–470.

20.

Bowling

Harries

Forrest

et al . Variations in cardiac interventions: doctors’ practices and views. Fam Pract 2006; 23: 427–436.