Abstract
Introduction
Management of patients with complex medical problems should ideally be collaborative and supported by clear communication between primary care providers (PCPs) and specialists; however, this appears to be the exception rather than the norm in the United States. 1 Care fragmentation, financial constraints, and geographic barriers are a few of the many obstacles facing patients who need to obtain subspecialty care, particularly in rural or underserved areas. Even in well-resourced settings, cultural and structural differences between primary care and subspecialty services present challenges to healthcare coordination. 2 –6
Programs that foster primary care and specialty care collaboration have been proposed as a solution to improve access, quality, and primary–specialty integration. 7 One established model is The Extension for Community Healthcare Outcomes (ECHO), which combines mentorship of PCPs with subspecialty care delivery, without requiring a traditional face-to-face subspecialist visit. 8,9 Project ECHO originated in New Mexico in 2003 with the purpose of expanding care to rural and underserved patients with hepatitis C virus infection. Under subspecialist mentorship, PCPs produced clinical outcomes for hepatitis C virus on par with academic gastroenterologists located in tertiary care settings. 10 In addition to improved access to specialty clinical services, 11 PCPs who participated in ECHO reported improvements in clinical content mastery, professional satisfaction, and specialist communication. 12,13 The ECHO model has been implemented in the United States and internationally across dozens of specialties. 14
The United States Department of Veterans Affairs (VA) is the largest integrated healthcare system in the United States, and in response to increasing Veteran enrollment and a geographically dispersed patient population, VA actively expanded virtual clinical modalities. In 2011, the VA initiated a partner program, the Specialty Care Access Network–Extension for Community Healthcare Outcomes (SCAN-ECHO), to enhance high-performance patient-centered care for rural, homebound, or otherwise isolated Veterans. 15,16 SCAN-ECHO launched 11 regional hubs, each offering 4–8 distinct subspecialty programs. 17 In addition to expanding access to needed services, VA SCAN-ECHO has produced reported changes in participants' clinical practice and enhanced understanding of specialty care content. 18
In the VA's Northwest region, SCAN-ECHO conferences occur approximately once weekly for each of four subspecialties (Hepatology, Nephrology, Pulmonary, and Infectious Diseases). Each conference is staffed by a multidisciplinary team of experts, including a medical subspecialist and/or a specialty pharmacist, mental health provider, and a substance use disorders specialist depending on the clinical need. Conferences typically last 60–90 min and involve a 20–30-min subspecialty-focused didactic presentation followed by an interactive discussion of cases presented by the PCPs.
Despite the significant investment of telemedicine programs like SCAN-ECHO and participant time needed to continue implementation, the duration of participation needed to accrue optimal benefits is not known. We surveyed VA primary care-affiliated providers in the Pacific Northwest who participated in SCAN-ECHO for one or more offered subspecialties. We assessed whether those who participated in SCAN-ECHO for longer periods of time experienced improved benefits compared to those with briefer participation, specifically with respect to specialty care access, acquisition of new knowledge and skills, team integration, and overall job satisfaction.
Materials and Methods
We administered a voluntary e-mail survey to 159 VA-based providers in the Pacific Northwest (Alaska, Washington, Oregon, and Idaho) who participated in SCAN-ECHO in our region.
Survey Development
The survey instrument (Appendix A1) consisted of 47 items: 29 Likert Scale, 14 multiple choice (including 7 demographic items), and 4 open-ended items. Survey items were selected based on relevance to 10 domains (access; quality and coordination of care; impact on tertiary services and consults; force multiplication effect; clinician satisfaction, knowledge, competencies, and skills; clinician integration; workload; clinician burnout; facilitators of SCAN-ECHO participation; and barriers to SCAN-ECHO participation). Professional burnout was assessed using a single-item measure, which has been previously validated in VA healthcare professionals from diverse occupations. 19 The survey was pilot tested by a group of VA providers for content, clarity, and functionality before dissemination. Survey participation was not tied to financial or administrative incentives.
Data Sources and Collection
Survey data were collected and managed using Research Electronic Data Capture (REDCap) electronic data capture tools hosted at VA Puget Sound. 20 All data were obtained from participant self-report, except for duration of participation and provider rurality. Duration of participation was obtained from SCAN-ECHO attendance records. Providers were considered “rural” if they (1) practiced in a clinic designated by the VA Office of Rural Health (in fiscal year 2013) as a rural healthcare site, or (2) identified by the Office of Rural Health as working in a clinic that served at least 50% rural or highly rural patients.
An initial e-mail invitation was sent to each participant on April 29, 2014. During the following 6-week period, three additional REDCap e-mail reminders were sent, as well as a direct e-mail invitation to survey nonrespondents from a SCAN-ECHO team member who had no direct supervisory relationship with the respondent.
Human Subjects
Our project was performed as an approved quality improvement initiative under the auspices of the VA Office of Specialty Care. Per VHA Handbook 1058.05, we obtained approval of nonresearch status from the VA program office overseeing the initiative and, as such, did not require an Institutional Review Board application.
Study Outcomes
Our primary outcome was respondents' perception of change in specialty care access for patients. Secondary outcomes included perceived improvement in quality of care, improved coordination and integration of primary care and specialty services, expansion of practice (either through obtaining new skills or use of SCAN-ECHO knowledge for additional patients), and overall PCP job satisfaction.
Statistical Analyses
All statistical analyses were conducted using STATA v13.1 (College Station, TX). We assessed differences in respondent characteristics using chi-square tests for categorical variables or two-sample t-tests for continuous variables. The significance level was 0.05 for all tests.
The cut point for optimal duration of participation in SCAN-ECHO was determined using receiver operating characteristic (ROC) curve analysis examining the relationship between duration and our primary outcome, perceived specialty care access. The precise cut point was selected to maximize Youden's J index, calculated as sensitivity + specificity – 1. 21 We determined that the optimum duration of SCAN-ECHO participation needed to maximize perceived specialty care access was 1.1 years (416.5 days). The area under the ROC curve at this cut point was 0.72 with sensitivity 0.67 and specificity of 0.78, which represents fairly good discrimination. We decided to round the optimum duration of participation to 1 year for simplicity.
Logistic regression was used to calculate odds ratio (OR) comparing participants with ≥1 year of SCAN-ECHO participation to those with <1 year. Covariates for logistic regression were selected a priori based on known or suspected confounders for the relationship between SCAN-ECHO participation and selected outcomes. Covariates for adjustment included age, sex, race, professional background, years of primary care experience, and rural practice.
Results
We received a total of 88 survey responses (response rate 55%). Among those participating for ≥1 year (n = 44), the survey response rate was 70.4%. Among those participating for <1 year (n = 115), the survey response rate was 43.5%. Ten individuals reported no participation in the program at the time of the survey and were excluded from further analysis. One respondent submitted a partially completed survey and was included in the analysis when his or her response was available. Respondent characteristics are reported in Table 1.
Survey Respondent Characteristics
ARNP, Advanced Registered Nurse Practitioner; CNA, Certified Nursing Assistant; DO, Doctor of Osteopathic Medicine; HCV, Hepatitis C virus; LPN, Licensed Practical Nurse; MA, Medical Assistant; MD, Medical Doctor; PA, physician assistant; RN, Registered Nurse; SCAN-ECHO, Specialty Care Access Network–Extension for Community Healthcare Outcomes; VA, Veterans Affairs.
Duration of SCAN-ECHO participation ranged from 15 to 796 days (mean 305 days). A total of 28 (35.9%) participants had engaged in SCAN-ECHO for ≥1 year (27 of whom submitted a complete survey). The majority of participants were nurse practitioners (NPs) or physician assistants (PAs) (42.9%), physicians (19.5%), or pharmacists (19.5%). Most were white (82%) and female (73%). A large proportion (38%) practiced in a rural area. Most (55%) attended one to three SCAN-ECHO sessions per month, 21% attended four or more, and 24% attended less than one session per month. Participants were allowed to participate in more than one specialty, including Hepatology (56%), Nephrology (26%), Pulmonary (30%), and Infectious Diseases (14%).
In bivariate analysis (Table 2), providers who participated in SCAN-ECHO for ≥1 year were significantly more likely than others to strongly agree that it improved patients' access to specialty care (51.9% vs. 16.0%, p = 0.001), improved quality of care (59.3% vs. 30.0%, p = 0.012), and improved care coordination between PCPs and specialists (70.4% vs. 30.0%, p = 0.001). PCPs were not more likely to report improvement in overall job satisfaction. However, longer term participants were more likely than shorter term participants to strongly agree that SCAN-ECHO increased their knowledge and competencies (77.8% vs. 48.0%, p = 0.01) and led to expanded scope of practice (53.8% vs. 24.0%, p = 0.009).
Results of SCAN-ECHO Participant Survey (n [% with Strong Agreement])
One respondent with ≥1 year of participation submitted a survey missing data for all items in this table and therefore was excluded from total N.
Adjusted for age, sex, race, professional designation, primary care experience, rural practice.
AOR, adjusted odds ratio; CI, confidence interval; OR, odds ratio; PCP, primary care providers.
Our findings persisted after adjustment for age, race (white vs. nonwhite), sex, professional designation (NP/PA, MD/DO, pharmacist, RN/LPN, other), primary care experience (≥10 years vs. <10 years), and rurality (rural vs. nonrural) (Table 2). Engagement in telemedicine mentorship for ≥1 year was independently associated with perception of improved patient access to specialty care (adjusted odds ratio [AOR] = 9.3, p < 0.05) and usefulness in treating other patients (AOR = 3.7, p = 0.04). Participation ≥1 year was also associated with higher self-reported knowledge and competencies (AOR = 4.0, p = 0.03) and with perception of integration into a clinical team (AOR = 5.6, p = 0.01), but not with overall job satisfaction. We considered the possibility that SCAN-ECHO specialty (i.e., Hepatology, Nephrology, Pulmonary, and Infectious Diseases) might be related to perceived benefit, but exploratory analysis, including specialty as a covariate, produced no meaningful differences in outcomes compared to our original model.
Discussion
Telemedicine-based specialty mentorship programs are highly valued by primary care-based participants, and self-reported benefits accumulate beyond 1 year of participation. Respondents who engaged in SCAN-ECHO for at least 1 year were significantly more likely to strongly agree that it improved patient access to specialty care and was useful in treating other patients on their panels. Participation beyond 1 year was associated with increased self-reported knowledge and competencies and with perception of integration into a clinical team, but not with overall primary care job satisfaction. Our findings support prior studies highlighting that provider-to-provider mentorship programs enhance clinical practice, improve patient care, and create a professional community of practice. 12,18 Telemedicine mentorship programs require a significant time investment for PCPs, but our results suggest that participant benefits continue to accrue over time.
Several factors may explain the long-term benefits of SCAN-ECHO. First, PCPs report that their highest priorities for collaboration with specialists include developing personal relationships and gaining mutual respect. 22 Ease of primary–specialty communication appears to vary by specialty, 23 but collaborative relationships are likely to deepen with time. Second, adult learning theory suggests that self-efficacy is a crucial component of behavior change and developing confidence with managing complex conditions is a time-consuming process. Not surprisingly, PCPs report a sense of greater content mastery when engaging with SCAN-ECHO over longer time periods.
PCP job satisfaction is a topic of increasing concern due to high rates of PCP burnout in the VA and in the United States as a whole. 19,24 Although participants in our project reported several rewards related to SCAN-ECHO, duration of participation was not associated with overall job satisfaction. Nationally, causes of PCP burnout are formidable, including heavy workload burdens, documentation requirements, risk of medical liability, low relative pay compared to other specialties, and other concerns, such as professional isolation, especially in rural areas. 24,25 SCAN-ECHO is one potential means to increase satisfaction, but does not appear to be sufficient in the absence of other factors.
Our study benefitted from a high participant response rate for an e-mail survey. In addition, we determined duration of participation and rurality using attendance data rather than self-report. We included a diverse group of PCPs in terms of professional and geographic backgrounds. We considered the possibility that frequency of participation per month might be an important predictor of our outcomes, but this characteristic was evenly distributed across groups. We considered the possibility that those who participate longer in SCAN-ECHO might do so because they found value in the program and thus could be more likely to change their clinical practice. To counter this potential source of bias, we made a conservative assumption about duration of participation by “starting the clock” on the date of participants' first virtual clinic attendance, regardless of participation thereafter. The fact that we still found significant benefits to longer participation suggests that telementorship might confer even greater advantages if all participants attended consistently. The number of clinic sessions attended, and frequency of attendance needed to reach a clinically meaningful threshold, is an area for additional research.
Overall, our findings support the benefits of long-term participation in SCAN-ECHO, but several limitations deserve consideration. Our study was conducted in a VA-based sample of healthcare providers, although we have no reason to doubt the benefits of participation to be generalizable to other populations. Only providers who participated in SCAN-ECHO were included in our analysis, and they may differ systematically from other PCPs who were unable or unwilling to participate. However, since this study was not designed to assess the merits of SCAN-ECHO participation versus nonparticipation, we would not expect such factors to influence the effects of duration of participation. Although all specialty teleconferences were designed similarly (e.g., brief didactic combined with interactive case discussion), the curriculum and supervising specialists necessarily differed. Furthermore, the specialty teleconferences had staggered start times as they were set up one by one. For this reason, Hepatology was over-represented in the ≥1 year participation group. Nevertheless, we believe that specific specialty-based content should have minimal impact on the benefits of participation over time, given the common conceptual foundation of all the SCAN-ECHO programs. Last, it is possible that differences in organizational culture at various clinics may have enabled some PCPs to enter the program earlier than others. Our small sample size and the large number of partner sites involved in the analysis precluded adjustment for clustering by site, but future larger analysis should consider using this approach.
Further research is needed to ascertain the impact of the SCAN-ECHO model on specialty providers and on patients. A randomized trial of duration of participation in SCAN-ECHO would be helpful to definitively determine whether longer duration is associated with beneficial effects. It remains unknown whether the SCAN-ECHO model will be sustainable or cost effective in the long term. Future studies are planned to evaluate clinical outcomes, such as use of referral services, workflow efficiency, and quality of care. PCPs report that retention in rural practice is associated with opportunities to achieve professional goals. 26 Additional data are also needed to assess the effects of SCAN-ECHO on provider retention and persistence of benefits when PCPs end their SCAN-ECHO participation.
Footnotes
Acknowledgments
The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the Department of Veterans Affairs. This material is the result of work supported by resources from the VA Puget Sound Health Care System (Seattle, Washington) and the VA Portland Health Care System (Portland, OR). Funding support was provided by the VA Office of Patient Care Services, Specialty Care Services program office, and by the VA Office of Rural Health.
Disclosure Statement
No competing financial interests exist.
Appendix A1. SCAN-ECHO Participant Survey
