Who Uses eConsult? Investigating Physician Characteristics Associated with Usage (and Nonusage)

Abstract

Background:

The Champlain BASE™ eConsult Service was developed in a Local Health Integration Network (LHIN) in Ontario, Canada in 2010 to reduce wait times and improve access to specialist care. The service allows primary care providers to receive advice from specialists via a secure electronic platform without necessarily requiring a face-to-face consultation.

Introduction:

As of 2015, over half of the LHIN's family physicians were registered and trained to use the service. However, 24% of registrants never went on to submit a case. The purpose of this study is to examine the demographic characteristics associated with usage.

Materials and Methods:

Usage data for the pool of physicians registered between January 1, 2011 and September 30, 2015 were linked to physician characteristics retrieved from the College of Physicians and Surgeons of Ontario database. Probit regressions were estimated to determine characteristics associated with usage.

Results:

Neither sex, being an international medical school graduate–documented predictors of electronic medical records adoption-nor proximity to specialists were found to explain usage. Only length of time in practice was found to be predictive. Being out of medical school an additional 10 years was estimated to decrease the probability of ever using eConsult by five percentage points (p < 0.01).

Conclusion:

Lower use by veteran physicians may reflect their lower need for services like eConsult given their well-established specialist networks, or their greater confidence in practicing medicine. Future work should explore the reasons and barriers for not registering, or not using eConsult, with an aim toward increasing the appropriate use of this cost-effective and innovative service.

Introduction

Traditionally in Canada, when a patient's primary healthcare provider (PCP) deems it necessary for their patient to see a specialist, a formal face-to-face consultation between the patient and a specialist is arranged.¹ A lack of specialist availability or a large distance between a patient's residence and the specialist's office can prolong the time it takes for the consultation to occur.¹ The average wait time to see a specialist in Canada increased from 3.7 weeks in 1993 to 8.5 weeks in 2014; with an average additional 9.8 week wait afterward to receive treatment.² PCPs have developed informal methods for consulting with specialists about their cases before a meeting of the patient and specialist, including speaking with specialists in person, by telephone, or by e-mail.¹ However, privacy and security concerns, as well as concerns over the adequacy of information exchanged, have limited the expansion of such informal consultations.¹

To reduce the growing wait times for specialist advice and reduce unnecessary specialist referrals, the Champlain Building Access to Specialists through eConsultation (BASE™) eConsult Service was developed and deployed in the Champlain Local Health Integration Network (LHIN) in Ontario, Canada.³ This secure, Internet-based service allows registered PCPs to submit questions and electronic files (such as lab results and images) to specialists and receive advice on next steps. If the specialist requires more information before they can provide an opinion, they can either request more information through the service, or recommend a face-to-face consultation.³ The pilot phase of the project ran between April 1, 2010 and June 30, 2012,⁴ and now the service is open to all PCPs in the Champlain LHIN on a voluntary basis. As of May 31, 2015, over half of the PCPs in the Champlain LHIN were registered for the service, including 595 family physicians and 119 nurse practitioners.³

Given the demonstrated high value eConsult has been shown to provide for users and patients,⁵ as well as the cost savings associated with the service,⁶ the provincial funder and provider of healthcare (the Ministry of Health and Long-Term Care) is committed to ensure that this service is used to its fullest potential. In this article we take an important first step in examining the characteristics of the users of this service. We are particularly interested in the ∼24% of family physicians who registered to use the service between 2011 and 2015, but never completed a single eConsult. This study centers around two research questions. What physician demographic characteristics predict going on to use the service after registration? What physician demographic characteristics predict becoming a frequent user?

Materials and Methods

Setting

The eConsult service was established in the Champlain LHIN, a health region in eastern Ontario with a population of 1.2 million.⁷ Approximately half of the region's population lives in its major metropolitan area (Ottawa), with the other half living in suburban or rural communities up to 2 h away by car. Demographics in the region are comparable to Ontario and the rest of Canada. Medical services in Canada are publically funded and administrated at the provincial level, with some exceptions (aboriginal groups, the military, and residents of the territories) under federal administration. In the case of the Champlain LHIN, residents are covered by the Ontario Health Insurance Plan.

All PCPs in the Champlain LHIN have access to the eConsult service for free. PCPs need to self-register and setup a user account. After the PCPs credentials have been verified by either the College of Physician and Surgeons of Ontario (CPSO) or the Nurse Practitioners Association of Ontario, a member of the eConsult team contacts the PCP to arrange a brief training webinar (average length of 30 min). Once this training has occurred, the PCP is able to submit a case. This process can be completed within 1 day, depending on the PCPs availability for the training call.

Dataset Creation

The main source of data for this study was the database of eConsult registrants (“registrants” hereafter). Registrants from January 1, 2011 to September 30, 2015, along with their completed cases up to December 31, 2015 were examined. Registrants from the final quarter of 2015 are excluded so that all registrants can be observed for at least one full quarter after registration. The following sample restrictions were carried out. First, only family physicians were selected. While nurse practitioners and administrative staff can also register for eConsult, demographic characteristics for these registrants are not readily available. More importantly, family physicians are the main users of eConsult. Over the study period, 88% of all eConsults were submitted by family physicians. Second, only practices in the Champlain LHIN were examined. The rurality and distance measures used in this study are only meaningful in the context of this LHIN. Further, the majority of specialists available through eConsult are physically located in this LHIN.

The CPSO database was used to match registrants with their demographic characteristics, using their names as listed in the eConsult registrant database. The CPSO contains information for every physician licensed to practice in Ontario. Physician sex, year of graduation, medical school, languages spoken, and specialities were retrieved for each registrant.

Data from the 2014 Postal Code Conversion File⁸ was used to calculate the distance between the postal codes of registrants' practices and the center of Ottawa, where the majority of specialists associated with eConsult Service are located. This distance served as a proxy for the distance a patient would have to travel to see the specialist.

Outcome Measures

Upon registration, registrants receive a unique identifier that can be used to track their usage of the eConsult service. Based on these identifiers, we calculated each registrant's average number of cases completed per quarter and then constructed the following two outcomes measures.

The first was a binary measure capturing whether the family physician completed at least one eConsult since registration—we called such physicians “users.” The second measure was also binary, classifying registrants as either “infrequent users” if their average number of cases completed per quarter is less that the median in their registration year, and “frequent users” if the average number of cases per quarter is greater than the median. The median as opposed to the mean was used as the cutoff given the positively skewed distribution of cases completed per quarter (Fig. 1).

Fig. 1.

Distribution of cases completed per quarter for users.

Data Analysis

The number of eConsult registrants and descriptive statistics summarizing both physician characteristics and usage patterns were reported by study year. In addition to the two usage measures that form the main focus of the article (“users” and “frequent users”), we also reported the average total number of eConsults, average eConsults per quarter, and the proportion of registrants who only completed eConsults during their quarter of registration, to provide a more comprehensive picture of physician usage. Next, descriptive statistics were reported to compare the sample of registrants, nonusers, and frequent users pooled across all registration years.

Probit regressions were used to estimate the effect of demographic characteristics on the probability of being (1) a user versus a nonuser (using the sample of all registrants), and (2) a frequent versus an infrequent user (using the sample of users only). We controlled for the length of time physicians have been practicing using the difference between a physician's year of graduation from medical school and 2013, the year of graduation of the most recent graduates in the sample. Other demographic characteristics included in the regressions are sex, an indicator for having graduated from a medical school outside of Canada or the United States, indicators for whether the physician speaks French or another language other than English, and the distance from the physician's practice to the center of Ottawa. Distances were calculated using the practices' latitude and longitude associated with their postal codes, and the great-circle distance formula from igeocode.com.⁹ Finally, dummy variables for registration year were included to control for possible differences in usage by registration year including (1) differences associated with earlier versus later registrants, (2) the fact that more recent registrants have had less opportunity to use the service and (3) the fact that the number specialty groups available through eConsult increased significantly over time (there were initially 5 specialty services offered in 2010, 17 in 2011, and 60 in 2015), increasing the usefulness of the service to a broader set of patients. To make the coefficient estimates from the probit models interpretable, estimated marginal effects, which is the estimated partial effects of each explanatory variable on the probability that the dependent variable takes on the value 1 (user or frequent user) associated with an infinitesimal change in a continuous variable or a discrete change in a dummy variable holding all over variables constant, were reported in the tables along with the estimates' robust standard errors correcting for possible heteroscedasticity.

Six specifications were reported for each of the two outcome measures. The first three (full sample, female and male subsamples) are the baseline specifications that include all variables as described above. To account for possible nonlinearities in the effect of years since graduation and distance to Ottawa, we included a second set of specifications with quadratic terms for these variables.

Results

Table 1 summarizes the demographic characteristics and usage patterns by registration year. Few differences were found across registration cohorts. The average distance to Ottawa was significantly lower (p < 0.05) for the 2014 cohort as compared to the 2011 cohort. The average number of years since graduation is 8.1 years higher for the 2011, as compared to the 2015, registration cohort (p < 0.05). This is partially by construction, as years since graduation was constructed relative to a static baseline (graduating in 2013). Graduates from 2013 appear in our sample in later years only. With respect to usage, the number of cases completed per user per quarter peaked in 2013, with the two previous years having significantly lower cases (p < 0.05). The percentage of registrants who only used eConsult during the registration quarter is very low for all registration years except 2015, at 2% overall. The percentage of registrants from 2015 who became eConsult users was significantly lower (p < 0.05) than several previous years.

Table 1.

Characteristics of Registrants by Registration Year

	2011	2012	2013	2014	2015 (Q1–Q3)	POOLED SAMPLE (2011–2015Q3)
Physician characteristics
Female (%)	62.3 (49.8–74.8)	82.9 (73.8–91.9)	66.7 (57.6–75.7)	76.9 (69.2–84.7)	71.3 (62–80.6)	72.2 (68.1–76.4)
International medical graduate (%)	6.6 (0.2–12.9)	8.6 (1.8–15.3)	8.3 (3.0–13.6)	14.5 (8.0–21.0)	11.7 (5.1–18.3)	10.4 (7.6–13.3)
Languages spoken (%)
French	34.4 (22.2–46.7)	41.4 (29.6–53.3)	42.6 (33.1–52.1)	42.7 (33.6–51.8)	43.6 (33.4–53.8)	41.6 (37–46.1)
Other	16.4 (6.8–26)	14.3 (5.9–22.7)	19.4 (11.9–27.0)	19.7 (12.3–27.0)	17.0 (9.3–24.8)	17.8 (14.2–21.3)
Years since graduation^*	20.3 (17.9–22.7)	16.2 (14.0–18.4)	15.7 (13.7–17.6)	14.2 (12.2–16.1)	12.2 (10–14.4)	15.3 (14.3–16.2)
Distance to Ottawa (mean in km)	32.5 (22.4–42.6)	23.3 (17.0–29.6)	24.1 (15.8–32.5)	14.2 (10.3–18.0)	22.6 (17.2–27.9)	22.2 (19.2–25.2)
Usage
eConsults completed
Total	1449	1628	3062	2064	435	8638
Per user, per quarter (median)	0.9 (0.6–1.2)	1.2 (0.7–2.0)	2.5 (2.0–3.2)	2.1 (1.3–3.0)	1.7 (1.0–2.4)	1.7 (1.5–2.0)
Only during reg. quarter (%)	0.0 (0.0–0.0)	1.4 (0.0–4.3)	0.0 (0.0–0.0)	2.6 (0.0–5.5)	6.4 (1.3–11.4)	2.2 (0.9–3.6)
Outcome variables
User (at least one eConsult) (%)	68.9 (56.9–80.8)	80.0 (70.4–89.6)	86.1 (79.5–92.7)	82.1 (75.0–89.1)	62.8 (52.8–72.7)	76.9 (73–80.8)
Frequent user (%)	34.4 (22.2–46.7)	38.6 (26.9–50.3)	48.1 (38.6–57.7)	41.9 (32.8–51)	37.2 (27.3–47.2)	40.9 (36.3–45.4)
N	61	70	108	117	94	450

95% confidence interval shown in parentheses. ^*Years since graduation is calculated relative to 2013.

Table 2 compares demographic characteristics across registrants, nonusers, and frequent users. While there are no statistically meaningful differences between registrants and nonusers, years since graduation is lower (p < 0.05) for frequent users (12.8) compared to registrants (15.3) and nonusers (17.7).

Table 2.

Characteristics of Registrants, Nonusers, and Frequent users

	REGISTRANTS (n = 450)	NON-USERS (n = 104)	FREQUENT USERS (n = 184)
Female	72.2 (68.1–76.4)	69.2 (60.2–78.3)	74.5 (68.1–80.8)
International medical graduate	10.4 (7.6–13.3)	16.3 (9.1–23.6)	6.5 (2.9–10.1)
Languages spoken
French	41.6 (37–46.1)	49 (39.3–58.8)	38 (31–45.1)
Other	17.8 (14.2–21.3)	24 (15.7–32.4)	15.2 (10–20.5)
Years since graduation^* (mean)	15.3 (14.3–16.2)	17.7 (15.7–19.6)	12.8 (11.3–14.3)
Distance to Ottawa (in km)	22.2 (19.2–25.2)	28.0 (20.7–35.3)	21.9 (17.2–26.5)

95% confidence interval shown in parentheses. ^*Years since graduation is calculated relative to 2013.

Table 3 presents the estimated marginal effects from probit regressions modeling the effect of physician characteristics associated with the probability of ever using eConsult. Three key findings emerged. First, greater years since graduation decreased the probability of ever using eConsult, conditional on having registered. Each additional year was estimated to decrease the probability of being a user by 0.5 percentage points in the baseline model and 0.7 percentage points (estimated at the mean years since graduation) in the robustness specification for the combined sample of men and women. Considering male and female doctors separately leads to similar estimates: all else equal, a male/female registrant graduating 10 years later is estimated to be 6.3/7.1 percentage points less likely to ever use eConsult. Second, the estimated effect of distance to Ottawa is very sensitive to how distance is modeled in the regressions. Statistically significant but very small effects are found in the baseline model: all else equal a physician practicing 22.2 km outside of Ottawa (the sample average) is estimated to be 3.1 percentage points less likely to be a user as compared to a physician whose practice is located in the center of the city. In our robustness specifications the estimated distance effect was not statistically significant. Lastly, the other characteristics do not appear to be important predictors of registrants becoming users.

Table 3.

Estimated Marginal Effects from Probit Regression: Probability (User = 1)

	FULL SAMPLE	FEMALES	MALES	FULL SAMPLE	FEMALES	MALES
Years since graduation	−0.00504^*** (0.00195)	−0.00499^** (0.00235)	−0.00451 (0.00367)	−0.0202^*** (0.00704)	−0.0175^** (0.00846)	−0.0268^** (0.0134)
(Years since graduation)²				0.000425^** (0.000188)	0.000361 (0.000236)	0.000586^* (0.000328)
Distance to Ottawa	−0.00138^** (0.000602)	−0.00135^* (0.000690)	−0.00183 (0.00126)	−0.000530 (0.00176)	−0.000715 (0.00204)	−0.000177 (0.00357)
(Distance to Ottawa)²				−7.02e-06 (1.33e-05)	−4.83e-06 (1.54e-05)	−1.46e-05 (2.77e-05)
Inter'l medical graduate	−0.0955 (0.0862)	−0.166 (0.108)	0.0672 (0.128)	−0.0928 (0.0872)	−0.164 (0.109)	0.0612 (0.131)
Female	0.00654 (0.0443)			0.0123 (0.0447)
French	−0.0938^** (0.0431)	−0.0721 (0.0501)	−0.156^* (0.0858)	−0.0949^** (0.0432)	−0.0767 (0.0502)	−0.139 (0.0853)
Other language	−0.0560 (0.0635)	−0.0310 (0.0744)	−0.0956 (0.122)	−0.0489 (0.0623)	−0.0298 (0.0738)	−0.0698 (0.114)
2012	0.0763 (0.0590)	0.0205 (0.0788)	0.140 (0.102)	0.0757 (0.0589)	0.0252 (0.0778)	0.113 (0.112)
2013	0.135^*** (0.0516)	0.0823 (0.0703)	0.200^** (0.0896)	0.136^*** (0.0521)	0.0841 (0.0705)	0.212^** (0.0948)
2014	0.0811 (0.0583)	0.00748 (0.0793)	0.203^** (0.0861)	0.0718 (0.0593)	0.00171 (0.0801)	0.192^** (0.0880)
2015	−0.0957 (0.0737)	−0.179^* (0.0974)	0.0202 (0.119)	−0.118 (0.0761)	−0.199^** (0.0998)	−0.00422 (0.124)
N	450	325	125	450	325	125

***

, ^**, ^*Indicates statistical significance at the 1%, 5% and 10% levels respectively.

Notes: Robust standard errors in parentheses.

Table 4 presents marginal effects for the probit regressions examining the effect of physician characteristics associated with the probability of being a frequent user, conditional upon being a user. None of the characteristics considered consistently predicts whether a physician will become a frequent user. While in the baseline models, years since graduation is found to decrease the probability of being a frequent user, the estimates are not statistically significant in the robustness specifications. The distance measure is found to be only marginally statistically significant (p < 0.10) in the female and male baseline specifications.

Table 4.

Estimated Marginal Effects from Probit Regression: Probability (Frequent User = 1)

	FULL SAMPLE	FEMALES	MALES	FULL SAMPLE	FEMALES	MALES
Years since graduation	−0.00772^*** (0.00287)	−0.00604^* (0.00341)	−0.0112^** (0.00551)	−0.0163 (0.00996)	−0.0152 (0.0125)	−0.0279 (0.0187)
(Years since graduation)²				0.000247 (0.000271)	0.000272 (0.000354)	0.000439 (0.000464)
Distance to Ottawa	0.000432 (0.000926)	0.00185^* (0.00112)	−0.00308^* (0.00177)	0.00254 (0.00263)	0.00233 (0.00310)	0.00518 (0.00605)
(Distance to Ottawa)²				−1.76e-05 (2.04e-05)	−4.16e-06 (2.41e-05)	−7.72e-05 (5.22e-05)
Int'l medical graduate	−0.133 (0.112)	−0.273^** (0.132)	0.173 (0.180)	−0.133 (0.112)	−0.266^** (0.134)	0.146 (0.194)
Female	0.0192 (0.0622)			0.0276 (0.0623)
French	−0.0257 (0.0577)	0.00983 (0.0675)	−0.0300 (0.119)	−0.0202 (0.0579)	0.0147 (0.0677)	−0.00353 (0.119)
Other language	0.0431 (0.0841)	0.0984 (0.101)	−0.0560 (0.160)	0.0526 (0.0841)	0.101 (0.101)	−0.0116 (0.168)
2012	−0.0490 (0.105)	−0.00444 (0.122)	−0.331^* (0.184)	−0.0467 (0.106)	−0.00423 (0.122)	−0.346^** (0.170)
2013	0.0442 (0.0947)	0.0787 (0.114)	−0.142 (0.177)	0.0594 (0.0964)	0.0749 (0.114)	−0.0629 (0.196)
2014	−0.0145 (0.0981)	−0.00941 (0.116)	−0.0562 (0.195)	−0.0182 (0.0993)	−0.0204 (0.117)	−0.0452 (0.197)
2015	0.0277 (0.106)	−0.0623 (0.127)	0.115 (0.210)	0.0125 (0.110)	−0.0870 (0.133)	0.101 (0.212)
N	346	253	93	346	253	93

***

, ^**, ^*Indicates statistical significance at the 1%, 5% and 10% levels respectively.

Notes: Robust standard errors in parentheses.

Discussion

Our study links usage data from the pool of eConsult registrants with family physician characteristics retrieved from the College of Physicians and Surgeons of Ontario database to better understand physician characteristics associated with (1) ever using the eConsult Service, and (2) the likelihood of being an above the median, or frequent user.

This work relates to studies that examine the physician and practice characteristics associated with the adoption of various health information technologies (HITs), in most cases electronic medical records (EMR). Higher rates of adoption are documented for physicians who are younger,^10
–12 male,^12,13 from larger practices,^10

–15 and graduates from United States, as opposed to international medical schools.^12,16 Being part of a practice that shares resources with other practices, that is part of an integrated delivery system or that is eligible for financial incentives for caring for patients with special needs are also associated with increased rates of HIT adoption.¹⁵ Practice rurality yields inconsistent effects on adoption. Some studies find that urban practices are more likely to adopt,¹⁰ others find that rurality has no effect.^12,17

While these studies help inform our choice of demographic characteristics to examine, there are important contextual differences that bear highlighting. First, the effect of location on eConsult usage is not expected to be the same as for HIT adoption since eConsult, unlike say EMR, was created in part to reduce unnecessary travel by patients to specialists. All else equal, it might be expected that practices located further from specialists, that is, more rural, would be more, rather than less, likely to use eConsult. Second, other factors such as physician language may be important given that the Champlain LHIN has many French physicians treating French-speaking patients and eConsult has only recently added access to certain specialties in French (DocToc, 2016).¹⁸

We have four key findings. First, comparing earlier and later registrants reveals surprisingly little in the way of differences. Earlier registrants have on average been practicing medicine longer. This finding is surprising given that younger physicians have been shown to more readily use new technologies.^10

–13 However, as explained by Greenhalgh et al. and Gagnon et al., social networks and opinion leaders can have a strong impact on the adoption of new technologies. The trend we document is consistent with more senior physicians being the more socially connected opinion leaders.^19,20

Second, 24% of registrants never go on to use the eConsult service. Neither sex, being an international medical school graduate—characteristics shown to predict adoption of EMRs were found to predict usage.^12,16 Even though the a priori benefits of eConsults should be greater for physicians and patients located further away from specialists, our measure of distance to specialists was not found to predict usage. In the few cases where the estimates of distance were statistically significant, they suggested that distance from Ottawa decreased the likelihood of becoming a user.

Third, only the length of time in practice was found to predict usage: all else equal, being out of medical school an additional 10 years was found to decrease the probability of a registrant ever using eConsult by five percentage points (p < 0.01). This could be due to the fact that more established physicians have less need for such a service either because they have developed their own well-established network of specialists, or because they have greater confidence practicing medicine, or (possibly) they are less interested in technology.

Finally, an important finding in this article is that once physicians use the service, they are likely to keep using the service. Only 2% of physicians use the service only in their quarter of registration. Thus, the fact that 24% of physicians who register but do not go on to complete an eConsult, should not be viewed as a rejection of the service (in the spirit of Bhattacherjee et al.²¹) but rather that these physicians never saw the potential value of the service. As a result of this finding, when physicians are now being training on eConsult, they are asked to bring a real case to submit right away.

This study has some limitations. First, practice size, a factor found to be important to the adoption of HIT, was not available.^10

–15 To the extent that practice size and other physician characteristics may be correlated, the interpretations of our estimated coefficients may be biased. For example, if more recent graduates are more likely to work in group practices, we might be falsely attributing the increased probability of usage with years since graduation when in fact the result we estimate is in fact practice size. Second, our measure of distance, based on the postal codes of physician practices is an approximation. Postal codes can cover large areas, especially in rural areas. Further, not all specialists are located right in the center of the city. Measurement error is well known to be associated with attenuation bias, driving our estimates closer to zero. Third, the findings of this study cannot be used to predict eConsult usage by all family physicians in the Champlain LHIN (or elsewhere)—as the study sample is not representative of the population of all Ontario family physicians. For example, according to the 2014 National Physician Survey, 46% of family physicians in Ontario were female in 2014, compared to 77% of our sample of 2014 eConsult registrants.⁹

While there is considerable interest in eConsult already, there remains a significant portion of the population of potential users that is either unaware of the service, or do not perceive there to be enough value to incorporate it into their practice. There are also informed and trained registrants who have chosen not to use eConsult. Future work needs to delve into the reasons and barriers associated with these choices to increase the appropriate use of this cost effective and innovative service.

Footnotes

Acknowledgments

We would like to acknowledge our four founding partners (Champlain LHIN, Winchester District Memorial Hospital, Bruyère Research Institute and the Ottawa Hospital) as well as our primary care and specialist physicians who are participating in the service. Funding for the eConsult service is provided by the Ontario Ministry of Health and Long-Term Care and the Champlain Local Health Integration Network. Research funding is provided by the Canadian Institute of Health Research. The funders were not involved in the study design, data collection, data analysis, or article preparation, or in the decision to publish the results. The views expressed do not necessarily reflect those of the Province of Ontario.

Disclosure Statement

No competing financial interests exist. The authors have no conflicts of interest to declare, real or perceived.

References

Liddy

, Rowan

, Afkham

, Maranger

, Keely

. Building access to specialist care through e-consultation. Open Med, 2013; 7:e1–e8.

Barua

, Fathers

. Waiting your turn: wait times for health care in Canada, 2014 report. Vancouver: Fraser Institute, 2014.

Liddy

, Keely

. Critical requirements and considerations for establishing and participating in an eConsultation service: Lessons learned from the Champlain BASE team. Electron Healthc Law Rev, 2015; 5:5–6.

Keely

, Liddy

, Afkham

. Utilization, benefits, and impact of an e-consultation service across diverse specialties and primary care providers. Telemed J E Health, 2013; 19:733–738.

Liddy

, Afkham

, Drosinis

, Joschko

, Keely

. Impact and satisfaction with a new eConsult service: A mixed methods study of primary care providers. J Am Board Fam Med, 2015; 28:394–403.

Liddy

, Drosinis

, Deri Armstrong

, McKellips

, Afkham

, Keely

. What are the cost savings associated with providing access to specialist care through the Champlain BASE eConsult Service? A Costing Evaluation. BMJ Open, 2016; 6:e010920.

Champlain LHIN. Population Characteristics for Champlain Health Link Areas. Available at www.champlainlhin.on.ca/∼/media/sites/champlain/Accountability/Integration/HL%20Docs/201410ChHLAsPopCharEN.pdf?la=en (last accessed August 8, 2017 ).

Statistics Canada. Postal Code Conversion File (PCCF), Reference Guide (92-154-G). Available at www.statcan.gc.ca/pub/92-154-g/92-154-g2013001-eng.pdf (last accessed August 8, 2017 ).

National Physician Survey. 2014 National Physician Survey (NPS): Ontario Demographics. Available at http://nationalphysiciansurvey.ca/wp-content/uploads/2014/10/NPS-2014-Demographics-ON-EN.pdf (last accessed August 8, 2017 ).

10.

Menachemi

, Perkins

, VanDurme

, Brooks

. Examining the adoption of electronic health records and personal digital assistants by family physicians in Florida. J Innov Health Inform, 2006; 14:1–9.

11.

Menachemi

, Powers

, Brooks

. Physician and practice characteristics associated with longitudinal increases in electronic health records adoption. J Healthc Manag, 2011; 56:183–198.

12.

Xierali

, Phillips

, Green

, Bazemore

, Puffer

. Factors influencing family physician adoption of electronic health records (EHRs). J Am Board Fam Med, 2013; 26:388–393.

13.

Decker

, Jamoom

, Sisk

. Physicians in nonprimary care and small practices and those age 55 and older lag in adopting electronic health record systems. Health Affairs, 2012; 31:1108–1114.

14.

Stream

. Trends in adoption of electronic health records by family physicians in Washington State. J Innov Health Inform, 2009; 17:145–152.

15.

Audet

A-M

, Squires

, Doty

. Where are we on the diffusion curve? Trends and drivers of primary care physicians' use of health information technology. Health Serv Res, 2014; 49(1 pt 2):347–360.

16.

Mazurenko

, Gupte

, Yeager

. International medical graduates and health information technology use in the United States. Health Information Technology in the International Context. Bingley, UK: Emerald Group Publishing Limited, 2012:121–140.

17.

Whitacre

. Rural EMR adoption rates overtake those in urban areas. J Am Med Inform Assoc, 2015; 22:399–408.

18.

DocToc. Champlain BASE eConsult Service Update. Available at http://amodocs.ca/sites/default/files/public_files/DocToc/2016Winter/2016winter.html#update (last accessed August 8, 2017 ).

19.

Greenhalgh

, Robert

, MacFarlane

, Bate

, Kyriakidou

. Diffusion of innovations in service organizations: Systematic review and recommendations. Milbank Q, 2004; 82:581–629.

20.

Gagnon

, Desmartis

, Labrecque

, Car

, Pagliari

, Pluye

, et al. Systematic review of factors influencing the adoption of information and communication technologies by healthcare professionals. J Med Syst, 2012; 36:241–277.

21.

Bhattacherjee

, Davis

, Hikmet

. Physician reactions to healthcare IT: An activity-theoretic analysis. IEEE, 2013:2545–2554.