Abstract
Introduction
: Interprofessional electronic consultations (eConsults) can reduce health care utilization and improve access to specialty care. However, health care utilization and access impacts of eConsults for headache disorders remain incompletely characterized.
Methods
: We conducted a retrospective, 1:3-matched cohort study comparing patients referred for in-person headache evaluations to patients who had a headache-related eConsult. The cohorts were propensity score-matched by age, sex, race, preferred language, provider specialty, insurance status, and medical comorbidities. Our primary outcome was the presence of one or more headache-related ambulatory encounters in the 12 months following the index referral date. We used univariable and conditional logistic regression models to ascertain the associations between referral type and outcome.
Results
: We identified 74 and 222 patients with eConsult and in-person referrals, respectively. Over the follow-up period, the proportion of patients with the primary outcome was significantly greater in the eConsult cohort than the in-person cohort (46.0% vs. 43.2%, p < 0.0001). A greater proportion of the in-person cohort had one or more ambulatory headache encounters in the 12 months preceding their referral than the eConsult cohort (10.8% vs. 5.4%, p < 0.0001). In the adjusted analysis, eConsult usage was not associated with significantly increased odds of the primary outcome (adjusted odds ratio [aOR] 1.1, 95% confidence interval [CI] 0.6–2.0, p = 0.71), although patients with one or more ambulatory neurology encounters in the preceding 12 months had significantly increased odds of the primary outcome (aOR 3.1, 95% CI 1.2–7.9, p = 0.015).
Conclusion
: Compared to in-person referrals, eConsult use for headache was not associated with significantly increased odds of having subsequent ambulatory headache-related encounters.
Introduction
Electronic consultations (eConsults) are asynchronous telemedicine services that enable health care professionals to seek and obtain advice on clinical questions from specialists through electronic communication without the necessity for direct patient evaluation. eConsults are associated with decreasing the need for in-person consultations, 1 reducing health care expenditures, 2 –4 and improving clinical outcomes. 5 These benefits of eConsults have been similarly reported in the field of neurology, particularly in neuro-ophthalmology, 6,7 pediatric neurology, 8 and headache medicine, 9 as well as in providing care in rural health care settings. 10
Although headache disorders are one of the most burdensome and disabling medical conditions affecting the United States (U.S.) population, 11 there is a significant gap in clinical knowledge about headache disorders in the medical community. Only over a quarter of neurology residency programs in the U.S. have a mandatory headache clinical rotation, highlighting the inadequate level of exposure to headache in neurology education. 12 This shortcoming likely leads to underdiagnosis and inadequate management of headache disorders among the neurology workforce. This is further exacerbated by the national and global shortage of neurologists, particularly headache specialists. 12 –14 eConsults offer a promising solution for health care providers and patients in providing robust and practical recommendations for managing headache disorders.
While a promising avenue for offering neurological services, the impact of eConsults on health care utilization for headache disorders remains understudied. Prior work has emphasized the utility and effectiveness of eConsults for headache disorders but concluded that significant work is warranted to optimize efficient collaboration between primary care providers and headache specialists, as well as timely management of headache patients early in their disease course. 9 Downes et al. previously sought to evaluate health utilization patterns of patients who had an eConsult for headache at a large academic center over a 6-month follow-up period, finding that patients evaluated with an eConsult were significantly less likely to incur an ambulatory headache-related or general neurology encounter compared with patients who had an in-person referral. 15 However, the follow-up period in this study was relatively short compared with other studies of eConsults, and both comparison groups were not matched on key sociodemographic variables, such as insurance category, thereby raising questions regarding the validity of the study’s findings. We therefore aimed to conduct a similar investigation with a more balanced representative comparison group and evaluate outcomes over a longer follow-up period than our previous analysis.
Methods
THE MOUNT SINAI NEUROLOGY ECONSULT PROGRAM
In October 2019, the Department of Neurology at the Icahn School of Medicine at Mount Sinai, a multicampus academic health system in New York City, launched an ambulatory eConsult service based on a specialized, secure workflow within the organization’s Epic electronic health record system (EHR). Through this process, a requesting provider can submit an eConsult request for three common neurological symptoms (headache, lower back/neck pain, and numbness) by inputting relevant clinical information about the patient’s condition. In addition, there is an option for a general neurology eConsult, allowing the provider to detail their consultation query in a free-text entry. Once the request is completed and submitted, it is directed to a communication pool managed by five neurologists who respond to eConsults on designated days of the week. The responding neurologist then assesses the eConsult, creates a clinical note with advice, and determines if an in-person neurological assessment is necessary. This note is then sent back to the original requesting provider for review (Fig. 1).

Clinical workflow diagram of Neurology eConsult process at Mount Sinai.
STUDY DESIGN AND PATIENT POPULATION
We conducted a retrospective cohort study of our institutional eConsult program, including all adult patients aged 18 years and older who had a headache-related eConsult request or an in-person referral for headache to any ambulatory practice across our entire health care system between January 1, 2020, and June 15, 2021. Based on factors identified in a prior analysis, we used greedy-nearest neighbor, 1:3 case:control, propensity-score matching without replacement to match both cohorts based on demographic and clinical characteristics 15,16 (age, sex, race, preferred language, provider specialty, insurance status, and Charlson-Deyo Comorbidity Index [CCI]). The number of matching variables was chosen to decrease bias and the mean square error. 16 We determined the form of the matching variables to optimize the number of good matches.
PATIENT CHARACTERISTICS
We extracted age at referral (i.e., “index”) date, sex, race, preferred language (dichotomized to English and non-English), and insurance category as well as the specialty from which the referral or eConsult request originated. CCI was calculated using patients’ medical history at the index date as recorded in the EHR by the International Classification of Diseases, 10th Edition Clinical Modification (ICD-10-CM) codes. 17 We additionally collected baseline health care utilization, including ambulatory and emergency department (ED) encounters during the 12 months preceding the index date. We defined “ambulatory encounters,” as any encounters that were completed using in-person office, video, or telephone modalities.
OUTCOME MEASUREMENTS
All outcomes were binary and determined via manual review of patient charts in the EHR. Given that, our goal was to determine the degree of health care utilization related to headache disorders; our primary outcome was the presence of one or more ambulatory headache-related encounters in the 12-month period following the index date. Our secondary outcomes included the presence of one or more of the following encounters: ED encounters (either headache-related or related to any cause), ambulatory encounters with a neurologist for any neurological complaint (i.e., “all-cause” neurology encounters), encounters for botulinum injections, and ambulatory encounters in clinics staffed by resident and fellow trainees.
STATISTICAL ANALYSIS
We compared demographic, clinical, and baseline health care utilization characteristics between cohorts using Wilcoxon rank sum tests for continuous variables and McNemar’s tests, Cochran-Mantel-Haenszel statistics, or Fisher’s exact tests for categorical variables. We conducted univariable logistic regression to model the relationship between patient-level characteristics and the primary outcome, as well as three of our secondary outcomes (all-cause ambulatory neurology encounters, headache-related ED encounters, and all-cause ED encounters). To consider the matched pairs that were created during propensity score matching, we then used conditional logistic regression to examine the relationship between a chosen set of prognostic factors and our chosen outcomes. We accounted for matched data by including patient identifiers as strata in our model statement. We selected covariates for inclusion in the multivariable regression models based on prior associations established in our previously published article. 15
In addition, we conducted a Kaplan–Meier survival analysis to analyze the differences in event-free survival of incurring the primary outcome within the 12-month follow-up period. Time to event was defined as the time from the index date to the first ambulatory headache-related encounter. Patients were censored from the analysis if they had no encounters during the follow-up period. p-Values smaller than 0.05 were considered significant. Statistical analyses were performed using SAS software version 9.4 (SAS Institute Inc., Cary, NC).
Results
Over the study period, we identified 74 patients who had an eConsult for headache, who we matched in a 1:3 ratio to 222 patients who underwent an in-person evaluation for headache. Compared with the eConsult cohort, patients in the in-person cohort were more likely to be male (87.8% vs. 82.4%, p = 0.0003) and English-speaking (96.4% vs. 93.2%, p < 0.0001), although there were no significant differences in age, race, insurance category, or CCI between cohorts. The eConsult cohort had more referrals from obstetrics/gynecology specialists than the in-person cohort (17.6% vs. 12.6%, p = 0.0059). In the 12 months preceding the index date, a significantly larger proportion of patients in the eConsult cohort had one or more all-cause neurology (48.7% vs. 41.0%, p < 0.0001) and headache-related (13.5% vs. 9.9%, p < 0.0001) ED encounters than patients in the in-person cohort. By contrast, a significantly larger proportion of patients in the in-person cohort had one or more all-cause ambulatory neurology (13.1% vs. 12.2%, p = 0.0002) and headache-related encounters (10.8% vs. 5.4%, p < 0.0001) in the preceding 12 months than patients in the eConsult cohort. (Table 1).
Baseline Characteristics Stratified by Study Group
Ambulatory encounters include telephone, video, and office visit encounters. “Baseline” refers to 12-month period preceding the index referral date.
OBGYN, obstetrics and gynecology; ENT, ear, nose and throat; HA, headache; ED, emergency department; SD, standard deviation.
Over the 12-month follow-up period, patients in the eConsult cohort were significantly more likely to have one or more headache-related (46.0% vs. 43.2%, p < 0.0001) and all-cause ambulatory neurology (43.2% vs. 42.3%, p < 0.0001) encounters than patients in the in-person referral cohort. Compared to patients in the in-person referral cohort, patients in the eConsult cohort were significantly less likely to have one or more headache-related (10.8% vs. 11.7%, p < 0.0001) and botulinum toxin injection encounters (2.7% vs. 4.9%, p < 0.0001) but significantly more likely to have one or more all-cause ED encounters (40.5% vs. 38.3%, p = 0.0003) or encounters in clinics staffed by trainees (31.1% vs. 17.5%, p = 0.002) (Table 2).
Health Care Utilization over the Follow-up Period, Stratified by Study Group
HA, headache; ED, emergency department.
In the conditional logistic regression analysis, the use of eConsult for headache was not significantly associated with an overall increased odds of the primary outcome over the study period (adjusted odds ratio [aOR] 1.2, 95% confidence interval [CI] 0.7–2.2, p = 0.46), or having one or more all-cause ambulatory neurology (aOR 1.0, 95% CI 0.6–1.9, p = 0.88), all-cause ED (aOR 1.0, 95%CI 0.6–1.9, p = 0.91), or headache-related ED encounters (aOR 0.7, 95%CI 0.3–1.8, p = 0.46). The presence of one or more headache-related ambulatory (aOR 3.7, 95%CI 1.3–10.8, p = 0.017) or all-cause ambulatory neurology encounters (aOR 4.8, 95% CI 1.8–12.5, p = 0.0016) in the preceding 12 months was significantly associated with increased odds of the primary outcome. In parallel, having one or more all-cause ED encounters in the preceding 12 months was significantly associated with an increased odds of having one or more all-cause ED encounters in the follow-up period (aOR 2.9, 95% CI 1.6–5.4, p = 0.0008) (Table 3). In the Kaplan–Meier analysis, there was no statistically significant difference between groups in attaining the primary outcome (log rank p = 0.81) (Fig. 2).

Kaplan–Meier survival estimate of having at least one headache-related encounter over follow-up period.
Multivariable Conditional Logistic Regression Modeling the Association between Covariates and Selected Outcomes
Reference, female sex.
Reference, English language.
“Baseline” refers to preceding 12-month period prior to index date.
aOR, adjusted odd ratio; CI, confidence interval; HA, headache; ED, emergency department; 12M, 12 months.
Discussion
In this matched, retrospective analysis of health utilization among patients who had both eConsults and in-person referrals for headache at an academic medical center, we found higher rates of the primary outcome and all-cause neurology encounters in the 12 months following the index date among patients who had an eConsult, although eConsult usage was not significantly associated with increased odds of the primary outcome nor having one or more all-cause neurology ambulatory or ED encounters. We found, however, that baseline health care utilization was significantly different between the two cohorts and, importantly, that these variables were significantly associated with subsequent health utilization in the follow-up period, suggesting that underlying baseline utilization was a significant driving factor for our results.
Our findings contrast with a previous analysis in which patients with an eConsult order had significantly lower odds of all-cause ambulatory neurology or headache-related encounters over the 6 months following the initial referral than patients referred for in-person headache evaluations. 15 The discrepancy in findings may be related to administrative differences that are specific to trainee-staffed clinics at our institution, which were a significantly more common site for follow-up ambulatory encounters for patients in the eConsult cohort than in the in-person cohort. More specifically, these clinics are managed by the hospital system rather than the medical school faculty group practice and operate on different electronic patient scheduling mechanisms and pathways, which may have in turn resulted in longer wait times for ambulatory encounter scheduling in hospital clinics than for faculty group practice. As such, while our prior analysis suggested lower frequency of ambulatory headache encounters in the eConsult cohort, inherent delays in scheduling may have resulted in significant differences over the prior study’s 6-month follow-up period that were eliminated at longer (i.e., 12-month) follow-up intervals.
More importantly, the conditional logistic regression analysis did not demonstrate that eConsult utilization was significantly associated with the primary outcome despite the clear differences in underlying outcome rates between both cohorts. This suggests that the presence of neurology encounters in the preceding 12 months, which tended to be significantly higher in the in-person referral cohort than the eConsult cohort, may have been a principal driving factor underlying the differences in intercohort outcomes. One plausible explanation for our findings is also that greater rates of baseline neurology and headache ambulatory encounters in the in-person referral cohort may have resulted in better symptomatic control of underlying headache conditions, resulting in a lower utilization rate of subsequent headache-related and neurology encounters. Another explanation for the difference in frequency of baseline neurology encounters may also be that patients referred to eConsults were more likely to be de-novo consultations, whereas patients who had already seen a neurologist in the prior year were more likely to request in-person neurology referrals from providers.
Another possible explanation for our results could be that referring providers were more likely to require additional neurological follow-up for their patients after receiving initial eConsult recommendations. Although primary care providers have generally expressed a high degree of satisfaction with eConsults across multiple specialties, 18,19 referring providers could have perceived headache eConsult recommendations as potentially lacking in completeness since eConsults are typically structured around a focused consult question and represent a brief, monophasic, asynchronous encounter between patient and neurologist. By contrast, in “traditional” in-person consultations, a face-to-face relationship is typically established between patient and consultant, and a more comprehensive evaluation is performed than in an eConsult. This greater comprehensiveness could have decreased the need for subsequent outpatient encounters with a neurologist. Importantly, we also found that there was an approximately 1% higher rate of botulinum toxin injection encounters in the in-person cohort, which likely decreased the need for follow-up appointments due to improved headache control.
Interestingly, our analysis showed a significantly lower proportion of eConsult-receiving patients that incurred at least one headache-related ED encounter than in the in-person cohort, though the eConsult cohort did have a significantly higher proportion of all-cause ED encounters. This suggests that while not associated with a reduction in the likelihood of ambulatory neurology services, eConsults may have reduced the need for patients to seek care for headache-related complaints in an ED setting. Given that emergency encounters are typically felt to represent a more costly form of utilization than ambulatory encounters, these findings may carry significant economic impact. 20 Furthermore, we saw a significantly increased odds of having at least one headache-related ED encounter with an established baseline utilization of headache-related ED encounters.
Despite these mixed results, the potential of eConsults to efficiently tailor patient care in the face of specialist shortages is particularly compelling. The U.S. faces a neurologist shortage in virtually every state as the burden of neurological conditions substantially increases, especially with an aging population suffering from chronic conditions. 21 In the U.S., the density of headache specialists is only 1.2 per 100,000 people with migraine which is strikingly inadequate given the projected need for at least 3,700 headache specialists to meet the current national demand for headache medicine services. 14 While there is a dire need for more headache specialists, eConsults may be utilized in the clinical workflow to alleviate the case burden on currently practicing neurologists and headache specialists by empowering PCPs to address their patients’ medical needs with evidence-based recommendations and diverting away from unnecessary in-person referrals. Indeed, prior work from multiple North American academic health systems has suggested that a substantial proportion of neurologically related eConsults can be resolved without an in-person patient encounter. 1,18,22
Furthermore, eConsults have been shown to help educate referring providers on the clinical management of a broad spectrum of clinical scenarios. 19 It is likely that eConsults provide similar benefits for neurological conditions, which in turn can reduce the need for neurologist intervention and ultimately allay neurologist shortages. However, questions regarding appropriateness, acceptance of clinical eConsult workflows, and clinical quality remain in neurological eConsults, particularly for headache disorders. In a prior study of headache eConsults at a large academic medical center, PCP teams minimally utilized eConsults for headache-related complaints, despite adequate provision of resources and education on the use of eConsults. 9 Fortunately, recent studies suggest that eConsults for neurological conditions are associated with a very low risk of adverse clinical events. 23
Our study has several notable limitations. First, significant differences in baseline utilization of neurology visits between the two cohorts were discovered after the case-control matching had occurred. We did not match cohorts using baseline rates of health care utilization based on our prior analysis, which failed to demonstrate any significant differences in baseline health care utilization rates between groups. Although we adjusted for baseline health care utilization in a multivariable regression analysis, the intergroup imbalance in this variable may also suggest the presence of other unmeasured confounders that could have differentially impacted outcome rates. Second, our study was a retrospective cohort study, which may implicate selection bias and reduce the generalizability of our findings in the context of eConsults’ utility for patients with neurological diagnoses. Third, because we matched our two cohorts by greedy-nearest neighbor propensity score matching, complete matching could not be fully achieved, and the two cohorts remained significantly different with respect to two variables (i.e., sex and language). Finally, our study was limited to the setting of one academic center, which reduces the study’s generalizability to the larger population of patients with headache disorders.
Conclusions
This retrospective cohort study highlights notable trends in the health care utilization of patients with an eConsult order in the realm of neurology, specifically in the context of headache disorders. We observed a small but statistically significantly greater frequency of one or more neurology and headache-related encounters among our eConsult cohort compared with the in-person referral cohort during a 12-month follow-up period, which occurred primarily through telemedicine modalities such as telephone and video visits. These findings implicate the utility of eConsults for headache disorders given the prevalent shortage of neurologists and headache specialists nationwide. However, future studies should further describe the impact of eConsult utilization on the clinical management of headache disorders and associated patient outcomes.
Footnotes
Acknowledgments
The authors thank Rachelle Morgenstern for her contributions to the data extraction process integral to this study.
Authors’ Contributions
S.H.: Investigation, writing—original draft, writing—review and editing, and visualization. P.A.: Conceptualization, methodology, statistical analysis, data curation, data interpretation, and writing—review and editing. M.D.: Investigation. M.H.D.: Investigation. B.K.: Conceptualization, methodology, resources, writing—review and editing, supervision, data interpretation, and project administration.
Data Availability Statement
The data that support the findings of this study are not publicly available due to the presence of personal identifiers but are available from the corresponding author upon reasonable request and subject to IRB approval.
Ethics Approval Statement
Use of patient data for this study was approved by the Institutional Review Board of the Icahn School of Medicine at Mount Sinai.
Disclosure Statement
B.K. has served as consultant for NeuraHealth and Syapse, holds equity ownership for serving on the advisory board of Syntrillo, and has held speaking engagements with the American Medical Association and the American Academy of Neurology. The remaining authors have no relevant conflicts of interest to disclose.
Funding Information
The authors have no funding sources to declare.
