Adherence to Emergency Department Referral Criteria in a Direct-to-Consumer Telemedicine Center

Abstract

Background:

The safety of direct-to-consumer telemedicine (TM) is closely related to red flag detection and correct referrals. The adherence to referral criteria from current guidelines is not well quantified.

Objective:

To analyze the emergency department (ED) referral rate and adherence to referral guidelines in TM encounters of acutely ill patients calling a center that adopts stewardship protocols.

Methods:

This is a retrospective observational unicentric study, between March 2020 and March 2022, with patients who spontaneously sought direct-to-consumer urgent virtual medical assistance. A video-based teleconsultation was provided immediately after connection. Physicians managed situations according to their clinical judgment. Current guidelines, containing specific guidance for referral if red flags were identified, were available for consultation. Physicians' semiannual performance feedback was carried out. We analyzed the patterns for referral to immediate face-to-face medical evaluation and the agreement degree with the institutional guidelines.

Results:

A total of 232,197 patients were available, and 14,051 (6.05%) patients were referred to ED. A total of 8,829 (68.4%) referrals were based in specific guidelines according to the International Classification of Diseases hypothesis, and 8,708 (98.6%) were justified according to guidelines. Diarrhea had the highest guidelines' adherence to referral (97.6%), followed by COVID-19 (90%), headache (84.2%), and conjunctivitis (78.8%). Policies did not support 5,222 (31.6%) referrals, though 5,100 (97.6%) of these were justified according to the doctor's clinical judgment.

Conclusion:

TM doctors' assessment of acutely ill patients has high rates of adherence to guidelines regarding referral. Stewardship protocol adoption provides high rates of red flag description, even in the referral of nonpolicy diseases.

Introduction

Boosted by the COVID-19 pandemic, patients often seek direct-to-consumer telemedicine (TM) encounters as their first contact with the health care system, especially in acute situations.¹ The initial TM assessment comes along with a reduction of face-to-face evaluations.² The quality and safety of TM-discharged patients are closely related to guideline adherence.³ Virtual providers must discern red flags in each presented condition, whose presence implies immediate referral to the emergency department (ED).⁴

ED and TM centers usually have decision support algorithms that are easily available. The goal of clinical practice guidelines is to synthesize the best available evidence to support clinical decision making, which improves the quality of care, improves patient outcomes, and provides the most cost-effective care.⁵ However, the publication of guidelines does not guarantee their implementation or adherence to them by the clinician, and for this reason, protocol stewardship is critical.⁶

Increasing evidence shows that TM assessment is associated with high prescription adherence to current protocols. A recent study by our group found a high association between prescribing antimicrobials through TM and the current recommendations in the guidelines.⁷ To our knowledge, no previous studies have evaluated discharge or referral criteria after TM medical consultations.

We hypothesized that medical doctors follow current guidelines in referring their TM patients for face-to-face consultations if their center adopts stewardship protocols. This study aimed to analyze the guideline adherence of video TM encounters, defined as appropriate referral to the ED for face-to-face evaluation of acutely ill patients who spontaneously sought evaluation.

Methods

STUDY DESIGN AND PARTICIPANTS

This unicentric retrospective study was performed at the Telemedicine Center of Hospital Israelita Albert Einstein (São Paulo, Brazil). The trial protocol was approved by the institutional ethics board with registration number CAAE 58048422.9.0000.0071 and was named the Tele Red Flag study. All data can be accessed in the institutional digital records. Data were gathered and confidentially stored by TM physicians. All authors wrote the article's initial draft, had full access to all data, and reviewed the article.

No one who was not an author contributed to the writing of the article. All analyses were performed by the TM Center coordinating the study. All authors decided to submit the article for publication and attest to the data's integrity and accuracy and the trial's fidelity to the preset protocol.

The study population included patients >16 years old who spontaneously sought care at the virtual ED from March 2020 to March 2022. All patients presenting with any situation were included. Exclusion criteria comprised only connection problems that did not allow medical records to be created, as these participants did not complete a medical assessment, and there are no records of their visits in the data bank.

TM CONSULTATION

The TM consultation was performed using the internet, proprietary videoconference software, and electronic health records. All remote physicians were board certified and had TM and emergency medicine training. The TM Center provided easy access to current guidelines, did periodical analyses of consultation results, and gave semiannual feedback to all medical staff in a multiparameter stewardship manner. Medical notes were recorded into the electronic medical documentation, which had a specific field to fill in the International Classification of Diseases, 10th Revision main hypotheses and referral reasons.

RED FLAGS

Twenty-six guidelines were available for immediate consultation and were structured with highlighted red flags. Physicians were trained to refer the patient to the ED if at least one red flag was identified in the encounter.

DATA EXTRACTION

The percentage of patients referred to the ED was analyzed from a specific field in the electronic medical record. A software algorithm was used to identify whether the criteria entered in the appropriate field of the electronic medical record justifying referral were present in the institutional guidelines.

STATISTICAL ANALYSIS

We used IBM-SPSS for Windows version 22.0 software for statistical calculations. The statistic was only descriptive. Categorical variables are presented as counts and percentages. There were no missing data.

Results

From a total of 232,197 patients available, 14,051 (6.05%) patients were referred to the ED. The top 50 International Classification of Diseases diagnoses are shown in Figure 1, of these, those with a specific recommendation for referral are listed in Table 1. A total of 8,829 (68.4%) referrals were backed by specific guidelines according to the ICD hypothesis listed, and 8,708 (98.6%) referrals were justified according to guideline guidance (Fig. 2).

Fig. 1.

Top 50 ICD-10 diagnoses identified. ICD-10, International Classification of Diseases, 10th Revision.

Fig. 2.

Referrals backed by guideline.

Table 1.

Pooled International Classification of Diseases, 10th Revision and Corresponding Red Flags

ICD-10	FINAL DIAGNOSIS	RED FLAGS
K12	Recurrent oral aphthae	Fever >37.8°C for >3 days • Lesions >1 cm in diameter • Lesions recurring or lasting >10 days • Presence of decompensated comorbidities, immunosuppression, and autoimmune diseases
R51, G43, G44, G44.2	Headache	Abrupt-onset headache • Worst headache of life • New headache • Headache that causes nocturnal awakenings • Severe comorbidities, immunosuppression, neurological disease or history • Neurological symptoms or neurological physical examination changes • Headache triggered by physical exertion or sexual intercourse
N30	Cystitis	Comorbidities • Age >65 years • Symptoms of systemic involvement, fever, or significant low-back pain • Pregnant women • Massive hematuria • Recurrence in <30 days
S20, S30, S40, S50, S60, S70, S80, S90, T14	Contusions	Cut and blunt wound >1 cm • Joint hypomobility • Disabling pain • Extensive hematoma • Age >65 years
I10, I15, O10, O12, O13, O14, O16, R03	Hypertension	Decompensated cardiovascular comorbidities • Neurological symptoms or visual changes • BP >180/110 mmHg measured under appropriate conditions • Cardiovascular or renal symptoms, including lower limb edema • Urinary changes such as foamy urine, oliguria, or hematuria • Pregnant women
L20, L21, L23, L24, L25, L30, L30.9	Dermatitis	Large affected area • Significant desquamation • Mucosal involvement • Comorbidities
A09, K58, K59	Diarrhea	Severe abdominal pain or disproportionate to diarrhea • Jaundice • Blood or mucus in stool • Signs of dehydration or sepsis • Uncontrollable vomiting • Recent antibiotic use
N94, N94.6	Dysmenorrhea	Menstrual delay >48 h • Pain outside the midline • Pain without improvement with anti-inflammatory drugs • Presence of significant dyspareunia or purulent discharge (risk of inflammatory disease) • New dysmenorrhea with intensity >7 • Severe comorbidities • Allergy to dipyrone + NSAIDs
K30	Dyspepsia	Persistent pain in the upper floor in the presence of high cardiovascular risk • Severe, recurrent, cramping pain in the right upper quadrant region. • Comorbidities • Age >65 years • Fever, jaundice, pallor • Recurrent vomiting • Significant decline in general condition • Melena/fecal acholia • Pregnancy
M08, M16, M17, M18, M23, M24, M25, M25.5, M75, M77, M96, S53, S83, S93, Z96	Articular pain	Significantly limited range of motion • Joint instability • Arthritis or joint effusion • Disabling pain • Pain associated with trauma • Chills, fever, constitutional symptoms.
G71, M41, M62, M79.6, M95, M96, R25.2, R93, S86	Muscle pain	A significant limitation of joint range • Disabling pain • Suspected muscle tear • Pain associated with trauma • Significant swelling • Signs of neural impairment (paresthesias, reduced strength)
B86	Scabies	Comorbidities • Fever • Pervasive affected area • Absence of improvement with treatment already instituted
J02, J03, J06	Pharyngotonsillitis	Age >75 years • Comorbidities • Dyspnea/stridor/signs of airway obstruction • Signs of retropharyngeal abscess such as bulging palate or severe unilateral hypertrophy • Torticollis • Exaggerated prostration—signs of sepsis.
M10	Gout	Fever • First episode • Arthritis of any joint other than First MTP • Pregnancy • NSAID allergy • Age >65 years • Disabling pain • Vomiting
I84, K60.2, K61, K60.3, L05, O22, O87	Hemorrhoids	Presence of fever or systemic symptoms • Comorbidities • Pain refractory to analgesia • Heavy bleeding and presence of clots • Local painful bulging or nodulation
A60, B00, B02	Herpes simplex	Concurrent systemic infection • Lesion extending beyond the perilabral region • Signs of abscess or secondary infection • Comorbidities
B34.2, B97	COVID-19
M51, M54, S32, S33, S34	Back pain	Age >65 years • Fever • Weight loss • Trauma • Comorbidities • Neurological signs in lower limbs • Significant limitation • Urinary symptoms
L55, T20–T31, T95	Burn	More than 5% of total body area affected by second degree burn • Any third degree burn • Affects face, perineum, hand or foot (except minor superficial burns) • Crosses large joints • Circumferential • Suspected inhalation or high-voltage injury
J11	Flu syndrome	The groups with the highest chance of complications and death (1) Children <5 years old (especially <2 years old); (2) elderly from >60 years (especially >65 years); (3) patients with chronic diseases: (a) respiratory: for example, asthma, COPD, interstitial diseases; (b) cardiovascular: for example, heart failure (except isolated SAH); (c) kidneys: for example, chronic kidney dysfunction or failure; (d) liver: for example, cirrhosis and liver failure; (e) hematological: for example, sickle cell anemia; (f) endocrinological: for example, diabetes (especially when using insulin); (g) neurological: epilepsy, previous stroke. (4) low immunity (e.g., by drugs, chemotherapy, HIV/AIDS); (5) pregnant women and postpartum women (up to 2 weeks postpartum); (6) nursing home residents; (7) grade 4 obesity (BMI ≥40); (8) indigenous population (Native Americans). Warning signs: (1) worsening the clinical status after initial improvement (a disease in two phases). Suggestive of bacterial pneumonia. Increased cough, sputum, yellowish discharge, dyspnea, chest pain, and return of fever; (2) persistence of high fever, dyspnea, and cyanosis, suggestive of viral pneumonia. (3) Rare complications: myositis and Guillain–Barré.
J01	Sinusitis	Age >75 years • Comorbidities • Dyspnea • Persistent fever, especially if associated with productive cough • Periorbital edema, facial erythema, proptosis, vision changes
G2.3, M43.6	Torticollis	Cervical trauma • Neurological symptoms (vertigo, gait disturbance, bladder dysfunction) • Paresthesia in shock on cervical flexion • Fever • Comorbidities • Age >65 years • Anterior neck pain • Severe headache
R05	Cough	Age >75 years • Comorbidities • Dyspnea • Presence of fever >37.8°C for >3 days (or >38.5°C once) • Productive cough >5 days (or progressive increase in sputum), wheezing
L50, L56	Urticaria	Respiratory symptoms • Concurrent angioedema • Mucosal lesions • Vomiting, abdominal pain, fainting • Age >65 years • Comorbidities
A88, H81	Vertigo	Age >65 years • Vertical nystagmus • Neurological signs • Gait instability • Any additional neurological changes • Vomiting, important limitation
N77, N77.1, B37.3	Vulvovaginitis	Presence of fever or systemic symptoms • Comorbidities • Risk of pregnancy • Dyspareunia (pain with intercourse) or significant pelvic pain • Overtly purulent discharge

ICD-10, International Classification of Diseases, 10th Revision; NSAID, Non-Steroidal Anti-Inflammatory Drug.

Diarrhea diagnosis resulted in the highest rate of referral adherence, observed in 97.6% of cases, followed by COVID-19 (90%), headache (84.2%), and conjunctivitis (78.8%) (Fig. 3). A total of 5,222 (31.6%) patients were referred for diagnoses who did not have specific institutional guidelines warranting referral, but 5,100 (97.6%) of these were justified according to the doctor's clinical judgment (Fig. 2).

Fig. 3.

The ICD-10-based guideline adherence rate.

Discussion

TM has emerged as a vital tool in providing medical care in multiple scenarios and became more popular during the COVID-19 pandemic.⁸ This study collected data from the first pandemic year and two subsequent years when mass immunization was in effect. During the first COVID-19 waves, ∼25% of our ED referrals were of suspected or confirmed COVID-19 patients.² Including all acute illnesses during the study period, our center referred only 6% of patients to the ED. These data suggest our doctors are highly resolute in their TM encounters of low-risk patients and strictly follow the scarce direct scientific evidence.⁹

With the rapid adoption of virtual care, it is crucial to ensure that the same standards of care and guidelines are maintained in TM as in traditional in-person consultations.¹⁰ Guideline adherence refers to how health care providers adhere to established clinical guidelines and protocols in their practice. This helps to mitigate the risk and ensure that the best delivery of care is related to quality and safety.¹¹ The lack of physical examination and limited access to patient information are the main challenges in a TM encounter. This can make it difficult for providers to make an accurate diagnosis and prescribe the appropriate treatment.¹²

In-person consultations frequently do not follow guideline therapy recommendations.¹³ Mounting evidence shows that TM assessment has been characterized by high guideline adherence regarding issues such as correct and safe prescriptions.¹⁴ This success is ensured through complete control of the data produced by the virtual care routine and frequent feedback and staff training. Our center avails itself of 26 internationally adapted guidelines to support assistance with TM consultations.

Although most diseases have a low risk of poor prognosis, some red flags may be present, and their recognition is a cornerstone of safety management.¹⁵ There is no previous study addressing TM quality referral based on red flag identification. A stewardship protocol is an essential element in the provision of TM services. This is a set of guidelines that govern the conduct of health care professionals when providing remote medical consultation.

Beyond prescription, administrators must check other features of the visit, such as correct diagnosis and referral. TM centers should consider developing protocols and guidelines for virtual consultations that consider the limitations of TM and the propensity of TM doctors to make ED referrals. Our center has a predominance of low-complexity patients, and the referral criteria were determined to be very sensitive.

Conclusion

TM assessment of acutely ill patients has high rates of adherence to emergency-referral guidelines. Stewardship protocol adoption provides highly accurate red flag identification, even when referring patients for nonpolicy diseases. TM should be reinforced as a strategy to provide first access to the health system associated with high resolution rates for low-complexity conditions and high adherence to guideline-directed referral to the ED.

Footnotes

Authors' Contributions

Conception, planning, analysis, and interpretation of data were contributed by T.A.D.A., F.T.M., and A.A.E. The writing of the article or its critical intellectual review was carried out by T.A.D.A., F.T.M., K.F.K., and K.D.A.L. Responsibility for the final approval for publication was on R.A.M. and C.H.S.P.

Disclosure Statement

No competing financial interests exist.

Funding Information

No competing financial interests exist.

References

Hollander

, Carr

. Virtually perfect? Telemedicine for Covid-19. N Engl J Med, 2020; 382(18):1679–1681.

Accorsi

TAD

, Amicis

, Brígido

ARD

, et al. Assessment of patients with acute respiratory symptoms during the COVID-19 pandemic by telemedicine: Clinical features and impact on referral. Einstein (Sao Paulo), 2020; 18:eAO6106.

Hersh

, Helfand

, Wallace

, et al. A systematic review of the efficacy of telemedicine for making diagnostic and management decisions. J Telemed Telecare, 2002; 8(4):197–209.

Sperling

, Andretta

CRL

, Basso

, et al. Telehealth for supporting referrals to specialized care during COVID-19. Telemed J E Health, 2022; 28(4):544–550.

Lopes

MACQ

, Oliveira

GMM

, Ribeiro

ALP

, et al. Guideline of the Brazilian Society of Cardiology on Telemedicine in Cardiology—2019. Arq Bras Cardiol, 2019; 113(5):1006–1056.

Fernández Mondéjar

. Considerations on the low adherence to clinical practice guidelines. Med Intensiva, 2017; 41(5):265–266.

Pedrotti

CHS

, Accorsi

TAD

, De Amicis Lima

, et al. Antibiotic stewardship in direct-to-consumer telemedicine consultations leads to high adherence to best practice guidelines and a low prescription rate. Int J Infect Dis, 2021; 105:130–134.

Kato-Lin

, Thelen

. Telemedicine for acute conditions during COVID-19: A Nationwide Survey Using Crowdsourcing. Telemed J E Health, 2021; 27(7):714–723.

Sri-Ganeshan

, Mitra

, Soldatos

, et al; Southeast Region Virtual Emergency Department (SERVED) Research Group. Disposition of patients utilising the virtual emergency department service in southeast region of Melbourne (SERVED-1). Emerg Med Australas, 2023; 35(4):553–559.

10.

Grimshaw

, Russell

. Effect of clinical guidelines on medical practice: A systematic review of rigorous evaluations. Lancet, 1993; 342(8883):1317–1322.

11.

Grol

, Grimshaw

. From best evidence to best practice: Effective implementation of change in patients' care. Lancet, 2003; 362(9391):1225–1230.

12.

Sher

, Semciw

, Jessup

, et al. Structured evaluation of virtual emergency department triage model of care: A study protocol. Emerg Med Australas, 2022; 34(6):907–912.

13.

Cabana

, Rand

, Powe

, et al. Why don't physicians follow clinical practice guidelines? A framework for improvement. JAMA, 1999; 282(15):1458–1465.

14.

Kulchar

, Chen

, Moon

, et al. Telemedicine, safe medication stewardship, and COVID-19: Digital transformation during a global pandemic. J Interprof Educ Pract, 2022; 29:100524.

15.

Alshahrani

, Katbi

, Bahamdan

, et al. Frequency, causes, and outcomes of return visits to the emergency department within 72 hours: A Retrospective Observational Study. J Multidiscip Healthc, 2020; 13:2003–2010.