Abstract
Background:
An electronic consultation (e-consult) platform was implemented to support pediatric primary care providers (PCPs) in providing gender-affirming care to transgender and nonbinary (TNB) adolescents. Following implementation, a study was conducted to (1) explore how access to this e-consult platform impacts PCP confidence and referral patterns, (2) describe the content of questions, and (3) evaluate PCP's perspectives regarding platform usability.
Methods:
Following each submission, providers completed a 17-item survey. A total of 20 providers submitted 38 e-consults and 26 follow-up surveys between October 2021 and December 2022.
Results:
All PCPs reported a high overall value and increased confidence caring for TNB adolescents. Nearly one in five (19%) felt it allowed them to avoid submitting a specialty referral. Mean System Usability Scale score was 78.2 indicating good usability.
Conclusion:
This e-consult platform shows great promise in increasing PCP confidence providing gender-affirming care adolescents. More widespread utilization could help improve access to care and decrease specialty care referrals.
Introduction
Gender-affirming medical care, including puberty-blocking medications and gender-affirming hormones, has been shown to improve mental health and quality of life for transgender and nonbinary (TNB) adolescents who desire this care. 1 –7 However, many gender-diverse youth face barriers to accessing this care. 8 Evidence suggests that a majority of youth who are interested in care are unable to receive it. 8 One notable barrier is the limited number of clinicians with expertise providing gender-affirming care for adolescents. 9 –11 These providers often practice in large urban academic centers, contributing to difficulties receiving care for youth living outside these areas. 12,13 Limitations in access to care are particularly concerning given the prevalence of depression, anxiety, and suicidality among TNB youth. 14,15
Electronic consultation (e-consult) refers to asynchronous, consultative, provider-to-provider communications within a shared electronic health record (EHR) or Web-based platform. 16 E-consult shows great promise as a way to provide primary care providers (PCPs) with timely, patient-specific consultative support. In adults, e-consults have been used to support the provision of care across multiple subspecialties and have been associated with improved PCP knowledge and decreased barriers to specialty care. 17,18 They have also been shown to be well received by PCPs. 16 Data in pediatric populations are more limited but suggest e-consults can help improve access to pediatric specialty care by reducing demand for in-person visits and allowing care to be delivered in the primary care setting. 19 –23
Studies of e-consult use among TNB patient populations is also limited, though there is some evidence from studies with TNB adults indicating that PCPs are largely satisfied with e-consult services and further suggest that this modality may help decrease referrals to specialty care. 24,25 Though emerging qualitative data suggest some pediatric PCPs desire to receive support from gender specialists via e-consult, 26 no prior studies have investigated their use in clinical practice. Understanding the impact of e-consult use among TNB adolescent-serving PCPs is important given the unique aspects of gender-affirming medical care for adolescents related to parental consent and confidentiality and how these issues may relate to access to care. To explore this, we designed and implemented an e-consult platform to support pediatric PCPs in caring for TNB adolescents. Following implementation of this platform, we conducted a study to (1) explore how access to the e-consult platform impacted PCP confidence caring for TNB adolescents and referral patterns to specialty care, (2) describe the content of clinical questions submitted by pediatric PCPs, and (3) evaluate pediatric PCP's perspectives regarding platform usability.
Methods
E-CONSULT PLATFORM DEVELOPMENT
A multidisciplinary team of stakeholders including three PCPs, three adolescent gender care specialists, and a member of the hospital health informatics team developed the structure of the e-consult platform in EpicCare Link, 27 a HIPAA-compliant, EHR-adjacent platform. The stakeholder team began by iteratively developing a series of prompts for PCPs to suggest information to include in the e-consult's free-text box (Fig. 1). Concurrently, the stakeholder team developed an instructional guide for e-consult submission. This guide included information about how to adjust EpicCare Link settings to send notifications via text or e-mail when a specialist response was submitted. After the prompts and guide were finalized and built into EpicCare Link, the system was pilot tested by two PCP stakeholders before implementation.
EpicCare Link platform prompts and instructions.
PLATFORM IMPLEMENTATION AND DATA COLLECTION
Once live in EpicCare Link, the e-consult platform was available to any community provider, regardless of the EHR used in their practice. The EpicCare Link platform was already used widely by community providers, with 4,526 users representing 341 practices across the region, at the time e-consult development began because it was used to submit subspecialty referrals. Providers without existing EpicCare Link access were required to request access to this platform to submit a e-consult.
Once logged in to EpicCare Link, PCPs generated a new order to submit an e-consult to a multidisciplinary team of providers (three medical providers and three social workers) who work in an adolescent specialty gender clinic. Once routed to the proper team member, a specialty provider responds to the PCP in EpicCare Link within three business days. During the study period, consults were not billed and could be submitted for any patient up to age 21, regardless of whether they were an established patient of Seattle Children's Hospital. The e-consult program was marketed both on the Gender Clinic website, during educational sessions and on provider e-mail listservs for providers across the Washington, Wyoming, Alaska, Montana, and Idaho region. Demographic information was collected from the EpicCare Link profiles of each PCP who submitted an e-consult. These data included provider type and zip code of primary practice location. Zip code was used to calculate distance from the PCP practice to the specialty gender clinic.
CONTENT ANALYSIS
To describe the content of the e-consult questions, each e-consult was reviewed, and an inductive content analysis was conducted by two authors (J.L.S. and K.B.). Each investigator reviewed each e-consult independently to develop a complete list of content codes. E-consults were then individually coded by the same two members of the research team (J.L.S. and K.B.) using the agreed-upon codes, with all disagreements adjudicated by a third author (G.M.S.). Because multiple e-consults included more than one clinical question, it was possible for each e-consult to include more than one code.
IMPACT AND USABILITY SURVEY
Between October 1, 2021, and December 31, 2022, after each e-consult submission, PCPs were e-mailed a 17-item follow-up survey focused on understanding (1) their perspectives regarding the impact of e-consult use on patient management and referral patterns, and (2) overall platform usability. Five survey items were identical to those used in a prior survey study evaluating an e-consult platform 24 for Canadian TNB adults. These items included how PCPs perceived its value to both them and their patient, outcomes of its use, and one open-ended item allowing them to provide feedback. Two additional investigator-derived items were included to understand how PCPs learned about the e-consult and if having access to the e-consult made PCPs feel more confident providing care to other TNB adolescents in their practice. The remaining items included the validated 10-item System Usability Scale (SUS) measure. 28 Survey data were analyzed descriptively, and human subject approval was obtained by the Seattle Children's IRB (STUDY00003529).
Results
A total of 20 unique PCPs submitted a total of 38 e-consults during the study period. A majority (n = 32, 84%) of consults were submitted by physicians. All were submitted by providers whose primary practice location was in Washington State. Nearly half (n = 16, 42%) of e-consults were submitted by providers practicing in zip codes located fewer than 10 miles from the specialty gender clinic, with the remaining consults submitted by providers practicing 10–99 miles (n = 12, 32%) and >100 miles (n = 10, 26%) away. The number of e-consults submitted per provider ranged from 1 to 7. Twelve providers submitted only one, six submitted two, and two providers submitted seven e-consults each. In terms of e-consults responses that were provided by the multidisciplinary gender clinic team, 28 consults (74%) were responded to by medical providers, 5 were responded to by a social worker only, and 5 were responded to by both a social worker and medical provider.
CONTENT ANALYSIS
With respect to the types of support PCPs were requesting through the e-consult (Table 1), question content was most frequently related to initiating puberty blockers (n = 10, 26%) or gender-affirming hormones (n = 10, 26%), followed by help determining which type of medical intervention, puberty blockers or gender-affirming hormones, would be appropriate to initiate for a patient (n = 7, 18%).
Frequency of Content Areas Identified in Submitted E-Consults
SURVEY RESULTS
A total of 26 follow-up surveys were completed. Survey responses indicated high overall value of the e-consult, both to PCPs themselves (median = 5/5) and the patient for whom the consult was submitted (median = 5/5). Every provider (100%) reported that having access to the e-consult platform made them feel more confident caring for other TNB adolescents (Table 2). A majority of providers (n = 21, 81%) felt they “got new advice for a new or additional course of action,” and for nearly one in five, a referral was originally contemplated but was able to be avoided because of the e-consult. Finally, the mean usability score on the SUS was 78.2 (highest possible score 100), indicating good overall usability. 29
Provider Responses to E-Consult Survey Items (n = 26)
Discussion
Results of this study suggest that pediatric PCPs caring for TNB youth find this EpicCare Link e-consult platform valuable. Importantly, PCPs reported that having access to the e-consult was not only valuable to the patient but also increased PCP confidence caring for other TNB youth in primary care. Overall, these findings provide support for e-consult platforms for pediatric gender-affirming care and suggest such platforms may be useful in increasing access to care for adolescents.
Given some of the unique aspects of care provision for adolescents compared to adults, such as parental consent, confidentiality, and use of medications like puberty blockers, understanding existing gaps in PCP knowledge in this area is critically important. Our content analysis suggests that a key gap in pediatric PCP education relates to the initiation of gender-affirming medications. These findings are in line with a prior qualitative study with pediatric PCPs which found that many desire to have access to more robust clinical decision support tools to facilitate the provision of gender-affirming medical care, such as protocols and algorithms that outline criteria for initiating gender-affirming medications. 30
The content areas we identified in the e-consults are also in line with other prior studies exploring gender-affirming care provision in adults, 24 indicating that support with medication management was a key topic of consultation for adult PCPs. Interestingly, questions about managing medications (i.e., interpreting laboratory values, making dose adjustments, and discussing delivery methods) were more common in the studies involving adult patients, while common questions in adolescent patients centered around medication initiation. This may be related to increased comfort on the part of adult PCPs to manage aspects of care that in pediatrics are often managed by specialists; however, future research is needed to better understand pediatric PCPs' perspectives regarding the management of gender-affirming care in the pediatric primary care setting.
Interestingly, the demographic characteristics of e-consult users in our study suggest that the platform is useful not only for providers located further from a specialty gender clinic but also for providers who work close to these clinics. These findings combined with the fact that pediatric PCPs reported increased confidence providing care to other TNB youth, suggest e-consult may be a key component of improving access to care, particularly for adolescents who experience barriers to receiving care in specialty care settings. 31
In line with the prior studies involving care for TNB adults, some providers in our study also indicated that being able to use the e-consult allowed them to avoid placing a referral to specialty care. This suggests that e-consult platforms could serve as a way to decrease the frequency of specialty referrals and support provision of care in the primary care setting. This is particularly important given many pediatric specialty clinics have long wait-times 32,33 and that the demand for gender-affirming care provision for adolescents in the United States far outweighs the number of specialty clinics currently providing these services. 13 Given the degree to which this limited capacity contributes to delays in adolescents being able to access this often time-sensitive care, developing strategies, like the e-consult, which can facilitate primary care-based provision may expand access and decrease demand for care in specialty settings.
Our findings should be interpreted within the context of the following limitations. First, this study was conducted by a multidisciplinary group of providers working in a single academic institution located in the Pacific Northwest. Furthermore, though e-consults were submitted by providers whose primary practice location varied with respect to the distance from the specialty gender clinic, all PCPs in our study practiced in Washington state. This may be because the platform is relatively new and therefore has not been widely advertised outside of Washington. Similarly, our study is limited by the small sample of completed e-consults submitted during the study period and limited ability to follow up with providers regarding the outcome the consult had on the subsequent management of their specific patient. Finally, though the SUS has been previously validated, the remaining survey items had not, however a majority of the remaining items had been used to evaluate an e-consult program for TGD adults. Thus, future studies of this and other e-consult platforms are needed to better understand the impact of e-consult use on both provider management and patient health outcomes.
Conclusion
Timely, consultative support via PCP-specialist platforms like this EpicCare Link e-consult platform has the potential to increase pediatric provider knowledge and confidence providing gender-affirming care. Our e-consult platform also shows promise in increasing primary care-based provision of gender-affirming care and decreasing referrals to specialty care. More widespread access to such platforms may help to decrease barriers to gender-affirming care for TNB adolescents, leading to improved health outcomes in this population.
Footnotes
Authors' Contributions
G.M.S., P.G.A. and N.F.K., and K.B. designed the data collection instruments, collected data, carried out the initial analyses, drafted the initial article, and critically reviewed and revised the article. J.L.S. designed carried out the initial analyses, drafted the initial article, and critically reviewed and revised the article. K.M.K. and J.K.H. conceptualized and designed the study, drafted the initial article, and critically reviewed and revised the article. D.A.C., L.P.R., and W.P. conceptualized and designed the study, coordinated and supervised data collection, and critically reviewed and revised the article for important intellectual content. All authors approved the final article as submitted and agree to be accountable for all aspects of the work.
Disclosure Statement
The authors have no conflicts of interest to disclose. G.M.S. serves as a consultant for Pivotal Ventures and the Fenway Institute.
Funding Information
This research was supported by the Seattle Children's Research Institute career development award (PI: G.M.S.) and the PEDSnet Learning Health Systems K12 award 5K12HS026393-03 (PI: Forrest) from the AHRQ.