Abstract
Background
There is limited information about how on-line screening tools developed by integrated systems facilitated management of COVID-like illness patients.
Methods
Using the Kaiser Permanente Northwest (KPNW) Electronic Health Record, we identified adult plan members who accessed online COVID-19 screening e-visits and enumerated their subsequent medical encounters, tests for SARS-CoV-2, and test outcomes.
Results
Between May 2020 and December 2021, members completed 55,139 e-visits, with disproportionate representation among females (65% vs. 53% in the overall membership) and members aged <45 years (61% vs. 39%). Thirty percent of patients (16,953) were managed entirely through e-visits and 70% received subsequent in-person care. The percent of SARS-CoV-2 positive individuals was highest among the 1055 individuals triaged to inpatient care (17.9%), compared to 9.5% among those escalated to additional ambulatory care.
Conclusions
The e-visit on-line screening tool helped KPNW assist thousands of patients with COVID-19 symptoms, avoid unnecessary in-person patient encounters, and preserved KPNW infection control and pandemic surge capacity.
Background
A major shift in the use of telehealth was observed globally during the COVID-19 pandemic.1,2 In the United States, there was a 154% increase in telehealth visits during late March 2020 compared to the same time period in 2019. 2 Much of this unprecedented surge in telehealth visits could be attributed to managing symptoms associated with COVID-19.
Even prior to the COVID-19 pandemic, Kaiser Permanente NorthWest (KPNW), a large integrated healthcare delivery system in Oregon and Washington, was encouraging members to use telephone and video visits to improve accessibility for patients, reduce burden of office visits on healthcare staff, and reduce greenhouse gas emissions associated with in-person visits. 3 To further support these goals, KPNW began offering electronic visits (e-visits) to adults in 2019 facilitated by self-administered screening tools termed SmartExams™ 4 made available to members when they initiate an e-visit through the on-line patient portal. At the time, smartexams were available for birth control and emergency contraception; bladder infections; exposure to sexually transmitted infections; eye pain or irritation; and cold, sinus pain, allergy, or flu syndromes. In March 2020, KPNW incorporated COVID-19 into the respiratory screening tool and made it accessible in English and Spanish languages.
Since the rollout of the COVID-19 SmartExam™, limited information is available about who is using the tool and how clinicians review responses to make decisions about management of patients with COVID-19-like symptoms. Evaluating who is accessing available screening tools and how they are subsequently routed for clinical care may provide important context for understanding the utility of such tools in preserving pandemic surge capacity and the possible gaps in testing and care management when using virtual care. Prior publications have suggested that initiating care virtually is an effective way to streamline care for various conditions, however some research has pointed to disparities in accessing telehealth by age, insurance, and comorbid illnesses.5–7 Few studies have evaluated the use of virtual care specific to management of COVID-19-like illness. 8
Here we provide a description of members in a large integrated health system who initiated an e-visit with the COVID-19 SmartExam™ over a 20-month period and examine subsequent healthcare utilization, SARS-CoV-2 laboratory testing and rates of positive test results.
Methods
Population
We used the KPNW electronic health record (EHR) to identify health plan members aged 18 years and older, who completed the online COVID-19 SmartExam™ from May 3, 2020, through December 31, 2021, and contained the term COVID (in a medical record text note (e.g. chief complaint or clinician recommendations). Individuals could have more than one encounter in the evaluation period if encounters were separated by at least 15 days. Data presented here include a summary of all completed COVID-19 SmartExams™ in the evaluation period, as well as demographics on all individuals represented in the exams.
SmartExam™ encounters
The COVID-19 SmartExam™ (referred to as e-visit, hereafter) includes a standardized series of questions about symptoms, comorbid conditions, and other questions related to medical status, as well as free text fields to provide supplemental information. Questions are tailored as responses are given, and a clinician (a medical doctor in the Internal Medicine or Family Practice department) reviews the responses, makes a diagnosis and a recommendation to the patient regarding SARS-CoV-2 testing and follow-up care. If the patient responses do not provide sufficient detail to make a clinical recommendation, the clinician may choose to call the patient for additional information. Once the clinician makes a recommendation, it is then sent to the patient in a message in the patient portal as part of an “after visit summary.” Clinicians are expected to provide a recommendation within 1 hour of e-visit completion, however, the ability to meet that goal fluctuated with volume of completed e-visits.
Clinical care trajectory
For each completed e-visit with COVID-19 entered in the encounter note, we determined if there was subsequent follow-up care in the health care system. Encounters that occurred within 3 days of the initial e-visit were considered related follow-up encounters. We enumerated each encounter according to the type of follow-up care that was delivered, where each encounter was assigned to the most intensive type of care delivered, ranked as follows: inpatient (hospitalization or emergency room visit), ambulatory care (outpatient primary care, urgent care, radiology-only, other outpatient), virtual (additional telehealth visit), and laboratory (SARS-CoV-2 lab testing and no accompanying clinical encounter). Encounters were assigned to the most intensive type in an effort to most appropriately characterize the true severity of encounters initiated as e-visits.
SARS-CoV-2 testing
We further enumerated individuals who (1) received orders for SARS-CoV-2 lab testing; (2) had SARS-CoV-2 lab results available in the EHR; and (3) had a positive SARS-CoV-2 test result. An encounter was assigned as having a qualitative RNA (RT-PCR) lab order, where an order was placed within 3 days of the encounter date. A lab order was considered filled when there was a documented lab result in the EHR associated with the lab order. A lab order with no result was considered unfilled. Positive test results for identified lab orders were defined as results where COVID-19 was documented as ‘detected’ and excluded indeterminate results. Results did not have to occur within a specific window of time; only to be associated with the lab order that was placed within 3 days of the encounter date.
Demographics and comorbidities
Demographic and high-risk condition details about our evaluated population are person-based, where individuals with more than 1 episode are characterized according to the first e-visit date. Comparisons between our evaluated population and the overall KPNW population are made using standardized mean differences. The overall KPNW population included members as of January 2022, excluding anyone in the evaluated population.
Relevant comorbidities are based on the medical conditions outlined by the Centers for Disease Control and Prevention as placing a person at higher risk for severe outcomes from COVID-19. 9 Individuals with documented diagnostic codes for the included conditions, in the prior 12 months, were considered to have the high-risk condition. Diagnostic codes for several of these conditions were identified using the Elixhauser and/or Charlson Comorbid Indices,10,11 which specify appropriate ICD-9 and ICD-10 code lists by health conditions.
Statistical analysis
To assess factors associated with lab orders for SARS CoV-2 tests, patient characteristics and clinical factors were compared using two-sample t tests (for continuous variables) and chi-square tests (for categorical variables) between those with and without lab orders. The same process was used to assess factors associated with a SARS-CoV-2 positive test, restricted to those with documented SARS-CoV-2 lab results. Included episodes were restricted to the period of July 2020 through December 31, 2021, as availability of lab tests for SARS-CoV-2 did not become reliable to members until July 2020. Nonparametric tests were used when appropriate. A binomial model with logit link function was used to evaluate associations between these factors and each of the two binary outcomes: receiving a lab order and having a COVID-19 positive lab result. The generalized estimating equation (GEE) approach with exchangeable covariance structure was applied to account for the correlation among multiple e-visits per patient when estimating the odds ratios (OR) and the associated 95% confidence intervals (CI). Analyses were performed using SAS version 9.4 (SAS Institute, Cary, NC).
This evaluation was reviewed by the KPNW IRB and was determined to be non-research. This activity was reviewed by CDC and was conducted consistent with applicable federal law and CDC policy. a
Results
SmartExam™ (e-visit) encounters
Between May 3, 2020 and December 31, 2021, 42,474 individuals completed a total of 55,139 COVID-19 e-visits. In the period from May 2020 through March 2021, before COVID-19 vaccine was widely available, there were an average of 2600 e-visits per month, compared to an average of 1500 in the remaining months (Figure 1). There were three clear surges in e-visit use across the 20 surveillance months, where the average number of weekly e-visits exceeded 900: July to August 2020, October to December 2020, and September to October 2021. E-visits were lowest from February 2021 through July 2021, with an average of 379 weekly e-visits. An overview of the number of e-visits, plotted by week of completion, with corresponding proportions of associated lab orders, documented lab results, and COVID-19 positive status is displayed in Figure 1.
Weekly totals of COVID-19 e-visits with SARS-CoV-2 lab orders and results: May 2020–December 2021. Lab orders within 3 days of encounter date. Same-person encounters separated by minimum 15 days. EV: E-Visits.
Overall trends
The absolute standardized differences between those with an e-visit and the general KPNW membership suggest the two groups are comparable in the sample distributions of demographics variables including race, ethnicity, SES and education, and all high-risk conditions except hypertension, according to Cohen's rule. 12 Differences observed include those with e-visits were younger; 50% were aged 25 to 44 years, compared to 30% among general KPNW members, more likely to be female (65% vs. 53%), less likely to be, Medicare-insured (11% vs. 26%), and less likely to have hypertension (14% vs. 25%), compared to general KPNW members (Table 1).
Demographic comparison of patients accessing COVID-19 E-visits compared to general Kaiser Permanente Northwest health plan members.
E-visits from May 3, 2020-December 31, 2021.
*KPNW members who have had ≥1 encounter in 12 months prior to January 25, 2022, excluding the evaluated population.
Based on census data, from tract level of residence.
High-risk conditions based on https://www.cdc.gov/coronavirus/2019-ncov/need-extra-precautions/people-with-medical-conditions.html; defined according to Elixhauser comorbidity index using ICD codes from EHR.
Heart conditions such as heart failure, CAD, or cardiomyopathies.
Hypertension with and without chronic complications.
Immunocompromised state from blood or bone marrow transplant, immune deficiencies, HIV, use of corticosteroids (prolonged).
Clinical care trajectory
Among all e-visits, 47,315 (86%) were routed for follow up care within 3 days; 14% were not referred for additional follow-up. Of those who were routed for additional follow-up, 77% of episodes were triaged to an in-person ambulatory visit, 19% to an additional virtual visit, and 2% to the lab. Only 1055 (2%) episodes were referred to inpatient care (Figure 2). Across the evaluated population, there were 3 fatalities within 7 days of the initial e-visit. All 3 individuals were referred to an ED or Urgent Care on the same day as they completed the e-visit. Two were hospitalized on the same-day as the e-visit and the third presented to the ED 3 days following their e-visit. All 3 tested negative for SARS-CoV-2 during hospitalization and had a documented cause of death unrelated to respiratory illness. There were no deaths in the 7 days following the e-visit, among those not referred for follow-up care.
E-visit episodes and subsequent clinical care trajectory. May 3, 2020–December 31, 2021.
SARS-CoV-2 lab orders
More than 67% of e-visits had an associated lab order across the full 20-month evaluation period. This proportion fluctuated over time, with the highest proportion (79%) occurring during July through December 2020, a decline to 55% during January through July 2021, and up to 63% during August through December 2021 (Figure 1). Encounters that resulted in additional follow-up care usually received lab orders (77%) for SARS-CoV-2 testing, compared to only 9% of those who were not directed to additional visits (i.e. advised to convalesce at home) (Figure 2). During the full evaluation period, most lab orders (>83%) were filled (i.e. the individual followed through with testing), but an increasing proportion (24%) were left unfilled in July through December 2021.
Predictors of lab orders
A total of 50,999 episodes (92% of all episodes), from July 2020 through December 2021, were represented in the lab orders statistical model. KPNW members of all ages, known race and ethnic groups, and sex were equally likely to have lab orders placed. Factors associated with a decreased likelihood of having a lab order placed include, being a Medicaid or Medicare enrollee (OR (CI): 0.84 (0.78–0.0) and 0.71 (0.65–0.77), respectively), not having a college degree (OR (CI):0.98 (0.98–0.99)) or having two or more comorbid conditions (OR (CI): 0.90 (0.82–0.99)). Conversely, lower income levels and severe obesity were associated with increased likelihood of having a lab order placed (OR (CI): 1.04 (1.02–1.06) and 1.11 (1.02–1.2), respectively). Using July 2020 as the referent time point, e-visits that took place from August through December 2020 had an increased likelihood of having a lab order placed, where visits in any month in 2021 (except August) had a decreased likelihood (Table 2).
Factors associated with receiving a SARS-CoV-2 lab order and a SARS-CoV-2 lab result
*Adjusted for age, sex, race, Hispanic ethnicity, insurance status, FPL, college degree status, date of test, BMI, diabetes, and count of comorbid conditions.
Includes all e-visits from July 2020 through December 2021, excluding 292 episodes with missing income and/or education data based on census data, from tract level of residence.
^Includes all lab orders from July 2020 through December 2021, excluding 140 episodes with missing income and/or education data.
High-risk conditions based on ICD codes in electronic medical record data, as specified by Elixhauser and/or Charlson Comorbid Indices.
Predictors of positive SARS-CoV-2 test results
A total of 29,179 episodes with a documented SARS-CoV-2 result from July 2020 through December 2021 were included in this analysis. Male sex, black race, Hispanic ethnicity, and having either obesity or severe obesity were each associated with higher likelihood of having a SARS-CoV-2 positive test. Lower likelihood of having a positive test was associated with lower SES and having 1 or 2 comorbid conditions. Using July 2020 as the referent time point, e-visits that took place in most months from September 2020 through December 2021 were associated with positive test results. In particular, e-visits in April, August, or December 2021 were four-to-five times more likely to have a positive test outcome than tests in July 2020 (Table 2).
Discussion
Rapid mobilization of the e-visit tool led to triage of over 55,000 potential in-person health encounters to virtual care management across the full 20-month observation period. Almost one-third (30.7%) of those who initiated care through the SmartExam™ were clinically managed through exclusive use of telehealth services, further highlighting the value of telehealth for management of less severe disease and for reducing nosocomial disease transmission. The ability to transition screening to a virtual platform is a success only if it is effective in helping clinicians appropriately triage care. In this evaluation, we found that over two-thirds (68%) of presenting patients were routed for in-person care within 3 days of completing their e-visit, 92% of whom received SARS-CoV-2 lab orders. The screening tool offered clinicians the needed information to inform clinical decision making. The ability to quickly distinguish key symptoms of COVID-19 that could signal progression to severe disease was an important goal of early screening programs, particularly when there was limited understanding of who was most at-risk for severe clinical disease. 13 For this reason, we also assessed whether there were any fatal outcomes in 7 days following completion of the initial e-visit and found that the 3 identified fatalities all occurred during inpatient stays, and cause of death was unrelated to respiratory illness. This evaluation shows that individuals presenting with the most severe symptoms were routed to in-person care, and no deaths were observed in that 7-day window among those without additional follow-up.
Consistent with previous studies of telehealth usage, we observed that females and younger persons under age 45 were more likely to utilize e-visits, while those with hypertension or Medicare coverage were less likely to utilize e-visits. One prior evaluation of telehealth readiness in the United States found that individuals of older age and racial and ethnic minorities, as well as those with multiple comorbidities, were less likely to have access to computers and/or internet access than the general population, and thus at a disadvantage for telehealth visits.5–7,14 The younger age observed with e-visits explains the low rates of chronic medical conditions and Medicare insurance status. However, the slightly low representation of Blacks and Hispanics, while minimally lower, is important for considering ways to improve inclusivity through these telehealth tools. Identifying ways to increase use of telehealth opportunities, through use of smartphones or other devices, across all race/ethnic groups and among older age groups should be a focus of healthcare systems like KPNW.
The majority of patients (67%) who completed an e-visit were referred for SARS-CoV-2 lab tests, and only 9.6% yielded positive results. Weekly fluctuations show that clinicians may have become more targeted in their testing over time, as they went from ordering labs for 75% of e-visits in the earliest months, down to 54% in late 2021. Results from the model further support that clinician behavior was responsive to the dynamic COVID-19 situation, as more e-visits resulted in lab orders during periods of increased COVID-19 incidence. Even though most e-visits had associated lab orders, with no differences by race or ethnicity, people with lower education levels and those with either Medicare or Medicaid insurance were less likely to have a lab order placed. It is possible that these were appropriately triaged without the need for lab testing, however, given that older adults are already under-represented in e-visit use, more attention on this issue is needed. NIH guidance 15 states that testing should not be based on presence of high-risk conditions. Our data suggest that clinicians were generally following that guidance, as we did not observe a pattern of disproportionately high lab referrals for those with high-risk conditions.
In our evaluation, SARS-CoV-2 positive test results were associated with being male, as well as being either black or Hispanic, even though these populations were not over-represented in use of e-visits. One explanation of this may be that individuals in these groups might have been motivated to use the e-visit only when likelihood of infection is quite high (i.e. following direct exposure, or experiencing more symptom severity). One other published study reported higher hazards of COVID-19 test positivity among blacks (1.37, 1.35–1.40) and Hispanics (2.36, 2.33–2.38) in comparison to whites in the same study period, but in a more diverse healthcare setting. 16
There were several limitations to our evaluation. E-visits were only offered to adults, which required that we exclude children from this assessment. We also may have inadvertently excluded COVID-19 related e-visits that did not explicitly document COVID in the encounter notes, resulting in possible under-reporting. However, the pattern observed for uptake of e-visits mirrored the periodic case surges observed in Oregon 17 and confirms that use of these exams with diagnostic codes for COVID-19 are a reasonable proxy for measuring COVID-19-related telehealth encounters. The relative racial homogeneity of the population in this evaluation limited our ability to examine racial, and possibly other, disparities. We limited our review of clinical care trajectories to a three-day follow-up period, as we were interested in the immediate care following the e-visit, however there may have been additional, more intensive clinical encounters not represented in this summary, beyond the 3-day period. There may also have been fatalities outside of the 7-day window which we would not have captured. However, 86% of individuals completing an e-visit had some additional follow-up within the healthcare system, which demonstrates that the e-visit was most often the starting point for clinical management. Finally, we did not assess patient perspectives on the e-visit tool, either as facilitating or impeding access to care, thus cannot speak to whether patients were satisfied with how the SmartExam™ was used for clinical management.
Conclusion
The SmartExam™ supported appropriate triage of symptomatic patients and prevented or deferred over 55,000 symptomatic members from visiting the healthcare system; over 30% of whom were successfully managed without an in-person visit. If utilization of telehealth services is sustained after the pandemic period, healthcare systems will need to evaluate and improve on virtual screening tools in order to expand access and utility to all individuals in preparation for routine or future emergent medical situations.
Footnotes
Acknowledgements
The authors would like to acknowledge Dr Elizabeth G. Liles for her input on COVID-19 clinical care at Kaiser Permanente and to Lisa Fox, BFA for her assistance with graphics. The authors would also like to thank Stephanie Cooper for providing important information on the timing of availability of e-visits within the Kaiser Permanente system.
Author disclosures
ALN has received research funding from Pfizer and Vir Biotechnology for unrelated studies.
Declaration of conflicting interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Centers for Disease Control and Prevention (grant number 75D30120C07767).
