McIsaac score for group A streptococcal infection: Comparison of electronic visits versus face-to-face visits

Abstract

Introduction

Acute sore throat is a common complaint traditionally completed with an in-person visit. However, non-face-to-face telemedicine visits offer greater access at reduced cost. We evaluated patient/caregiver asynchronous text-based electronic visits (eVisits) for acute sore throat and whether there was concordance for individual components and total McIsaac score compared to a clinician’s assessment. eVisits were completed by patients and/or their caregivers via a secure patient portal.

Methods

In this retrospective study, we manually reviewed charts between February 2017 and July 2019 of patients who had an eVisit, in-person visit and group A streptococcal (GAS) test performed on the same day for an acute sore throat. We calculated a McIsaac score for eVisits and in-person visits, and compared each component and total score using Cohen’s kappa agreement statistic.

Results

There were 320 instances of patients who had an eVisit, in-person visit and GAS testing done on the same day. Approximately a third of eVisits were missing at least one McIsaac component, with the physical examination elements missing most commonly. Individual score congruence was moderate for cough (0.41), fair for fever (0.34) and slight for tonsillar swelling/exudate and lymphadenopathy (0.17 and 0.08, respectively), with total congruence being slight to fair (0.09–0.37). A McIsaac score of ≤1 showed moderate agreement (0.44). Visits with complete individual score components demonstrated improved congruence: substantial for cough (0.64), moderate for fever (0.57), fair for tonsillar swelling (0.3) and slight for lymphadenopathy (0.13).

Discussion

Overall agreement for individual score components was better for symptoms than it was for examination components, and was improved when data were complete. A McIsaac score of 1 or 0 had moderate agreement and thus could reasonably be safely used to exclude patients from GAS testing.

Keywords

Electronic visits telemedicine pharyngitis telehealth virtual consult

Introduction

Sore throat is a common reason for patients seeking health care,¹ and most cases of acute pharyngitis are caused by either group A streptococcal (GAS) or viral infections.^2,3 Current Infectious Diseases Society of America guidelines recommend treatment for GAS be based on a positive rapid antigen detection test and/or culture.¹ To help delineate which patients should be tested for GAS and which can be managed without testing or treatment, the Centor Score was developed for use in adult patients.⁴ Subsequently, the Modified Centor Score (McIsaac score) incorporated patient age for use in children three years of age or older.⁵ The McIsaac score includes a point for temperature >38°C, lack of cough, tender anterior cervical adenopathy, tonsillar swelling or exudate and being aged 3–14 years, with 0 points for being aged 15–44 years and –1 point for being aged >45 years (Supplemental Table S1). These scoring systems were validated with in-person provider visits.

Non-face-to-face (non-f2f) care is increasing in an effort to improve patient convenience, help contain health-care costs and promote access.⁶ Non-f2f care can be done via telephone calls, asynchronous text-based electronic visits (eVisits) or synchronous video-based telemedicine visits, and these have been evaluated for a number of acute concerns.^7–10 Since acute sore throats are rarely due to life-threatening entities,^11,12 with the majority of viral origin,¹³ acute pharyngitis may be amenable to evaluation without an in-person visit. In non-f2f care for acute pharyngitis, using the McIsaac score to help stratify patients would require patients (or their caregivers) to report the physical examination findings of tender enlarged lymph nodes and tonsillar swelling and/or exudate. Yet, the accuracy of patient/caregiver reported physical examination findings is not known, nor is the accuracy of utilising patient/caregiver reports to calculate a McIsaac score.

Our institution offers eVisits to established patients. These eVisits are asynchronous text-based visits via an online secure patient portal in which a patient/caregiver responds to a series of questions. For an acute sore throat eVisit, the questions include the components to calculate a McIsaac score. Once completed, the eVisit is forwarded to a certified nurse practitioner for review. The nurse practitioner then assesses whether home care is recommended. Alternatively, the patient can be referred to either an in-person appointment or GAS test collection by a nurse. Some patients end up having an eVisit and in-person visit on the same day as their sore throat assessment. This can occur if the nurse practitioner performing the eVisit recommends an in-person visit be performed. Additionally, patients or their caregivers can also choose to follow up on the same day for an in-person visit, even if the nurse practitioner does not make a recommendation to do so. In-person appointments at our institution are performed by physicians, nurse practitioners and physician assistants.

The primary aim of our study was to assess the agreement of individual McIsaac components as reported by patients/caregivers during eVisits to those found by health-care providers who saw the patient on the same day at an in-person visit. Our secondary aims were to evaluate the degree of concordance of McIsaac score totals between the eVisits and in-person visits and compare the percentage of positive GAS tests for McIsaac scores calculated from patient/caregiver eVisit responses to those calculated from an in-person visit with a clinician.

Methods

We retrospectively reviewed charts of patients who had an eVisit, in-person visit and GAS test done on the same day for acute sore throat between 13 February 2017 and 10 July 2019. These visits occurred at local outpatient Mayo Clinic sites in Kasson and Rochester, Minnesota. Demographic data for age, sex and race were collected. Two authors (J.L.P. and N.E.M.) manually abstracted the McIsaac score components as reported by the patient/caregiver during the eVisit and the clinician during the in-person visit. Total McIsaac scores and the individual scoring components were compared using Cohen’s kappa agreement statistic. Degree of agreement for various kappa scores was based on Cohen’s reference (Supplemental Table S2).¹⁴ We performed an initial analysis for degree of agreement for all patients between their eVisit McIsaac score components and their in-person score components which included encounters with missing scores at the eVisit and/or in-person visit. In this analysis, any missing score component (in the eVisit, the in-person visit or both) was considered to not be concordant (Table 1). We then performed the same analysis excluding any components with missing scores, that is, only score components where both the patient/caregiver (at the eVisit) and the clinician (at the in-person visit) had entries were evaluated for concordance (Table 2). We also reviewed the percent positive GAS tests with 95% confidence intervals for each McIsaac score as reported by the patient/caregiver (during their eVisit) or the clinician (during their in-person visit) for all encounters. Confidence intervals (95%) were calculated in order to compare percent positive GAS tests for each McIsaac score between providers and patients/caregivers. Patients younger than three years of age and those older than 75 years of age were excluded. JMP Pro v14.1.0 was used for statistical analysis. This study was evaluated and approved by the Mayo Clinic Institutional Review Board (#19-005282).

Table 1.

Frequency and comparison of reported McIsaac criteria and McIsaac scores by patients/caregivers and providers for all 320 patient encounters.

McIsaac score (total)	Provider reported (in-person visit), n (%)	Patient/caregiver reported (eVisit), n (%)	Kappa (95% CI)	Level of agreement
Absence of cough	198 (62%)	252 (79%)	0.41 (0.32–0.49)	Moderate
Fever present	163 (51%)	111 (35%)	0.34 (0.28–0.41)	Fair
Tonsillar swelling/exudate	182 (57%)	178 (55.6%)	0.17 (0.12–0.22)	Slight
Lymphadenopathy present	207 (64.7%)	164 (51%)	0.08 (0.03–0.02)	Slight
McIsaac score 0	20 (6.2%)	19 (5.9%)	0.37 (0.17–0.58)	Fair
McIsaac score 1	50 (15.6%)	41 (12.8%)	0.30 (0.16–0.44)	Fair
McIsaac score 2	70 (21.9%)	94 (29.4%)	0.09 (–0.02 to 0.20)	Fair
McIsaac score 3	95 (29.7%)	110 (34.4%)	0.13 (0.02–0.24)	Slight
McIsaac score 4	85 (26.6%)	56 (17.5%)	0.33 (0.21–0.44)	Fair
McIsaac score ≤1	70 (21.9%)	60 (18.7%)	0.44 (0.32–0.56)	Moderate

Missing score components at in-person and/or eVisit considered discordant.

CI: confidence interval.

Table 2.

Frequency and comparison of reported McIsaac criteria by patients/caregivers and providers for patients without missing data.

	Provider reported (in-person visit), n (%)	Patient/caregiver reported (eVisit), n (%)	Kappa (95% CI)	Level of agreement
Absence of cough, N = 266	198 (74%)	209 (79%)	0.64 (0.52–0.75)	Substantial
Fever present, N = 233	109 (47%)	131 (56%)	0.57 (0.47–0.68)	Moderate
Tonsillar swelling/exudate, N = 200	135 (67%)	173 (86%)	0.3 (0.17–0.43)	Fair
Lymphadenopathy present, N = 222	155 (70%)	162 (73%)	0.13 (–0.006 to 0.27)	Slight

Coding rubric used

eVisits

The eVisit platform asks patients/caregivers to report on the McIsaac score components allowing answers of ‘yes’ or ‘no’ or ‘I don’t know’ for all components except for cough for which only ‘yes’ or ‘no’ answers are allowed. Lymphadenopathy was asked as ‘Do you have sore “lumps” along your neck or under your chin (e.g. swollen lymph nodes)?’ For tonsillar findings, patients are asked if their tonsils were ‘larger than normal’ and/or had ‘white spots on them’. Patients/caregivers were also asked if they had had a fever and, if so, what the maximum temperature was. For this study, patients were assigned a point if their maximum reported temperature was >100.4°F. Answers of ‘I don’t know’ for fever, lymphadenopathy and tonsillar findings were counted as ‘missing’, and no points were given for that McIsaac component.

In-person visits

As no specific documentation flow sheet is used for clinical documentation for in-person visits, we used pre-specified criteria to count McIsaac score components as present or absent (Supplemental Table S3). Any criteria not specifically documented at the in-person visit was counted as ‘missing’ and was not included as a point for the total McIsaac score.

Results

During the study time period, there were 316 unique patients who had a total of 320 eVisits and in-person visits with GAS testing for acute sore throat on the same day. The average age of patients on the day of their encounters was 22.8 years. There were 132 (41%) patients who were between 3 and 14 years old, 164 (51%) who were between 15 and 44 years old and 24 (7%) who were between 45 and 75 years old. Sex distribution was 65% females, and 92% of patients identified race as white, 4% as other or choose not to disclose, 2% as black and 2% as Asian.

Provider notes were missing historical data for cough (16.9%) and fever (9.7%), but only rarely missing criteria for physical examination findings (1.9% each for lymph node and tonsillar examination). During the eVisit, patients/caregivers were unable to report fever (22%), tonsillar exudate/swelling (35.9%) and lymphadenopathy (29.4%), but all reported on the presence or absence of a cough which was required by the eVisit questionnaire.

For individual score components, kappa coefficients indicated moderate agreement between patients/caregivers and clinicians for the absence of cough (0.41). However, there was only fair (0.34) agreement between presence/absence of fever and only slight agreement (0.17 and 0.08, respectively) for tonsillar swelling/exudate and lymphadenopathy (Table 1). To account for missing data in provider notes and/or ‘I don’t know’ answers in eVisits, kappa coefficients were also generated for each McIsaac criterion in which the individual component was available for both the provider in-person visit and the patient/caregiver eVisit (Table 2). When excluding components with missing data, agreement was substantial for absence of cough (0.64), moderate for fever (0.57), fair for tonsillar swelling/exudate (0.3) and slight for lymphadenopathy (0.13).

Comparing McIsaac scores 0–4 for patients/caregivers compared to clinicians resulted in slight to fair agreement (0.09–0.37; Table 1). Agreement for a McIsaac score of ≤1 was moderate (0.44). The percentage of positive GAS tests for each McIsaac score for providers and patients/caregivers is shown in Table 3. The overlap in 95% confidence intervals between provider and patients/caregivers for percentage of positive GAS tests for each McIsaac score suggests there are no statistically significant differences between the two groups in our study in predicting a positive GAS test for each McIsaac score.

Table 3.

Percent of positive group A strep (GAS) tests for each McIsaac score for patient/caregivers (in eVisits) and providers (in-person visits).

	n positive/total N (% positive) [95% CI] GAS
McIsaac 0
Provider	2/20 (10%) [2.8–30.1%]
Patient/caregiver	2/19 (10.5%) [2.9–31.4%]
McIsaac 1
Provider	13/50 (26%) [15.9–39.5%]
Patient/caregiver	10/41 (24.4%) [13.8–39.3%]
McIsaac 2
Provider	21/70 (30%) [20.5–41.5%]
Patient/caregiver	39/93 (41.9%) [32.4–52.1%]
McIsaac 3
Provider	46/95 (48.4%) [38.6–58.3%]
Patient/caregiver	53/110 (48.2%) [39.1–57.4%]
McIsaac 4
Provider	47/84 (55.9%) [45.3–66.1%]
Patient/caregiver	25/56 (44.6%) [32.4–57.6%]
McIsaac ≤1
Provider	15/70 (21.4%) [13.4–32.3%]
Patient/caregiver	12/60 (20%) [11.8–31.8%]

Of the 12 eVisits with a positive GAS test and a patient/caregiver-derived McIsaac score of ≤1, all were missing score data points, with two eVisits missing one element, five eVisits missing two components and five eVisits missing three elements. There were only five eVisits with no missing data and a McIsaac score of ≤1. None of these five eVisits had a positive GAS test.

Discussion

The advantages of telemedicine with respect to cost and patient access make non-f2f encounters desirable. However, research supporting non-f2f evaluation of acute sore throat is sparse. Hertzog et al. found that the diagnostic accuracy of eVisits and in-person encounters for low acuity visits such as pharyngitis were comparable, but this only included 10 eVisits for pharyngitis and did not include a McIsaac score evaluation.¹⁵ When patients/caregivers were able to assess the components of the McIsaac score, our study found substantial and moderate amount of congruence between patients/caregivers and providers for cough and presence of fever but only fair and slight congruence for tonsillar and lymph node findings. Given that many or most patients/caregivers probably lack medical training, it is unsurprising that there is less agreement for tonsillar and lymph node findings than there is for fever and cough. However, the relative lack of concordance with fever compared to cough is more surprising. Possibly, this discrepancy is due to clinicians utilising the temperature taken by the nurse rather than relying solely on patient history. Or perhaps this relates to lack of patient clarity on what temperature level constitutes a fever. Interestingly, twice as many eVisits were unable to report on the presence or absence of fever compared to missing documentation for fever in the provider visits. This finding is a bit odd, as missing data in an eVisit meant the patient/caregiver answered ‘I don’t know’ to whether they had a fever. Yet, later that day, half of those who did not know were able to report to the examining provider whether they had a fever. Potential explanations include the patient/caregiver checking for fever after the eVisit questionnaire, fever developing in the intervening time period, the patient/caregiver noting the rooming vitals by the nurse or different caregivers for the eVisit versus the in-person visit. Finally, we were not able to ascertain whether caregivers always completed the eVisit with their child present, which might also explain why they answered ‘I don’t know’ in some cases.

Despite the varying agreement for the McIsaac components, there was fair congruence for the total scores for McIsaac scores 0–2 and 4 (and slight congruence for a score of 3) between providers and patients/caregivers. As a low McIsaac score is often used to identify patients who can avoid testing for GAS, we reviewed the level of agreement for low McIsaac scores of ≤1 and found moderate congruence between patient/caregiver and provider assessment.¹⁶ This suggests that scores of ≤1 may be used to triage patients away from needing an in-person visit or GAS test.

Our study found a higher (though similar) prevalence of positive GAS tests in low McIsaac scores in both patient/caregiver-derived scores and provider-derived scores compared to the McIsaac study.⁵ However, results for positive GAS tests in the low McIsaac scores were closer to a 2012 study by Fine et al. looking at number of positive GAS tests by McIsaac score in a large sample size (>200,000), which showed 8% and 14% positive tests for scores of 0 and 1, respectively.¹⁷ Fine et al. only included patients who had complete McIsaac data, whereas our study includes missing data in encounters, which might account for part of the difference. Additionally, previous studies have shown that younger patients are more likely to utilise eVisits, and^18,19 given that younger patients are more likely to be streptococcus positive as well as be streptococcus carriers,²⁰ this difference could play a role for the higher rates of GAS that we found. This suggests that further studies evaluating demographic differences for populations utilising eVisits and rates of positive GAS tests for acute sore throat may be beneficial.

Studies specifically comparing patient/caregiver report to clinician evaluation in primary care for the McIsaac criteria are lacking. Lindgren et al. compared patient/caregiver McIsaac scores to physician-derived scores for children aged 3–21 years seeking care in the emergency department for a sore throat. Similar to our findings, their study also showed moderate agreement between patients/caregivers and clinicians for low McIsaac scores (0–1). Additionally, our finding of 21% of patient/caregiver-derived McIsaac scores of ≤1 having a positive GAS test is very similar to that found by Lindgren et al., who found an 18% streptococcus-positive rate for patients with a patient/caregiver-derived McIsaac score of ≤1. However, their study did not indicate if patients were given the option to choose ‘I don’t know’ for any of the score components. Lindgren et al. also found a similar level of congruency (slight) for lymph node findings but a lower congruency of agreement for cough and a higher congruency of agreement for fever and tonsillar findings.²¹ The Lindgren study design used pictures to illustrate oral and lymph node findings, and this may have improved the congruency for tonsillar findings, though it did not appear to improve congruency for lymph node findings.

Allowing the inclusion of photos of the oropharynx and the cervical lymph node region to be included as a component of the eVisit could potentially improve ability to assess examination findings during eVisits. For example, utilising simple diagrams of the oral cavity, Xu et al. found modestly improved agreement of history and physical examination findings compared to our results.²² However, their study included only adult patients, with the patients completing the information at the clinic right before a provider visit. So it is unclear how generalisable their results are. Using video technology, Akhtar et al. found fair agreement for tonsillar size (κ = 0.251) and pharyngeal exudate (κ = 0.298) and moderate agreement for assessing tenderness of the superficial cervical lymph nodes (κ = 0.431) when comparing clinician video visits to in-person visits.²³ Akhtar et al. compared video visits with a provider compared to in-person clinician visits. Whether provider video visits improve accuracy compared to patient/caregiver report remains to be established.

Limitations

This was a retrospective study in which only patients having an eVisit and in-person visit with a GAS test on the same day were included. Due to the retrospective methodology, patients (and providers) may have self-selected or been referred for GAS testing based on other factors, such as a known streptococcus contact or a history of recurrent ‘strep throat’, rather than McIsaac score, which may have accounted for a higher percentage of positive GAS tests at lower McIsaac scores compared to the original McIsaac study. Additionally, in the analysis that included patients with missing score components, many of the lower McIsaac scores may have reflected inability of patients/caregivers to evaluate their score components or, in the case of provider derived scores, incomplete charting.

The difference in patient/caregiver versus provider findings along with the lack of temporally consistent examination means that it is conceivable that an examination could have evolved between the time of the eVisit and the actual provider encounter. However, that seems unlikely, as only patients who had an eVisit and an in-person visit on the same day were included in our study. A future study might consider looking at the time difference between the eVisit and the encounter to see if there is any temporal relationship between the findings.

Conclusion

Patients/caregivers were unable to report on tonsillar findings or lymphadenopathy in approximately one third of eVisits, and agreement of score components was better for symptoms than examination components. Overall, a McIsaac score of 1 or 0 had moderate agreement, and there were no positive GAS tests in encounters with a patient/caregiver score of ≤1 without any missing score elements. These results suggest that eVisits with a McIsaac score of 1 or 0 can safely be used to exclude patients from GAS testing.

Supplemental Material

sj-pdf-1-jtt-10.1177_1357633X21990999 - Supplemental material for McIsaac score for group A streptococcal infection: Comparison of electronic visits versus face-to-face visits

Supplemental material, sj-pdf-1-jtt-10.1177_1357633X21990999 for McIsaac score for group A streptococcal infection: Comparison of electronic visits versus face-to-face visits by Nathaniel E Miller, Teresa B Jensen Leah M Nigon Kristine S Penza, Martha A Murray, Beckie J Kronebusch and Jennifer L Pecina in Journal of Telemedicine and Telecare

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship and/or publication of this article.

ORCID iDs

Nathaniel E Miller

Kristine S Penza

Beckie J Kronebusch

Jennifer L Pecina

Supplemental material

Supplemental material for this article is available online.

References

Shulman

Bisno

Clegg

, et al. Clinical practice guideline for the diagnosis and management of group A streptococcal pharyngitis: 2012 update by the Infectious Diseases Society of America. Clin Infect Dis 2012; 55: 1279–1282.

Bisno

. Acute pharyngitis. N Engl J Med 2001; 344: 205–211.

Huovinen

Lahtonen

Ziegler

, et al. Pharyngitis in adults: the presence and coexistence of viruses and bacterial organisms. Ann Intern Med 1989; 110: 612–616.

Centor

Witherspoon

Dalton

, et al. The diagnosis of strep throat in adults in the emergency room. Med Decis Making 1981; 1: 239–246.

McIsaac

White

Tannenbaum

, et al. A clinical score to reduce unnecessary antibiotic use in patients with sore throat. CMAJ 1998; 158: 75–83.

Courneya

Palattao

Gallagher

JM.

HealthPartners’ online clinic for simple conditions delivers savings of $88 per episode and high patient approval. Health Aff (Millwood) 2013; 32: 385–392.

Penza

Murray

Myers

, et al. Management of acute sinusitis via e-visit. Telemed J E Health. Epub ahead of print 10 June 2020. DOI: 10.1089/tmj.2020.0047.

Penza

Murray

Myers

, et al. Treating pediatric conjunctivitis without an exam: an evaluation of outcomes and antibiotic usage. J Telemed Telecare 2020; 26: 73–78.

Penza

Murray

Pecina

, et al. Electronic visits for minor acute illnesses: analysis of patient demographics, prescription rates, and follow-up care within an asynchronous text-based online visit. Telemed J E Health 2018; 24: 210–215.

10.

Murray

Penza

Myers

, et al. Comparison of eVisit management of urinary symptoms and urinary tract infections with standard care. Telemed J E Health 2020; 26: 639–644.

11.

Ridgway

Parikh

Wright

, et al. Lemierre syndrome: a pediatric case series and review of literature. Am J Otolaryngol 2010; 31: 38–45.

12.

Ames

Ward

Tranter

, et al. Adult epiglottitis: an under-recognized, life-threatening condition. Br J Anaesth 2000; 85: 795–797.

13.

Pham

L-L

Bourayou

Maghraoui-Slim

, et al. Laryngitis, epiglottitis and pharyngitis. Infectious Diseases 2017; 229–235.e1.

14.

Cohen

A coefficient of agreement for nominal scales.

Educ Psychol Meas 1960; 20: 37–46.

15.

Hertzog

Johnson

Smith

, et al. Diagnostic accuracy in primary care E-Visits: evaluation of a large integrated health care delivery system's experience. Mayo Clin Proc 2019; 94: 976–984.

16.

Choby

BA.

Diagnosis and treatment of streptococcal pharyngitis. Am Fam Physician 2009; 79: 383–390.

17.

Fine

Nizet

Mandl

KD.

Large-scale validation of the Centor and McIsaac scores to predict group A streptococcal pharyngitis. Arch Intern Med 2012; 172: 847–852.

18.

Adamson

Bachman

JW.

Pilot study of providing online care in a primary care setting. Mayo Clin Proc 2010; 85: 704–710.

19.

Mehrotra

Paone

Martich

, et al. Characteristics of patients who seek care via eVisits instead of office visits. Telemed J E Health 2013; 19: 515–519.

20.

Shaikh

Leonard

Martin

JM.

Prevalence of streptococcal pharyngitis and streptococcal carriage in children: a meta-analysis. Pediatrics 2010; 126: e557–e564.

21.

Lindgren

Neuman

Monuteaux

, et al. Patient and parent-reported signs and symptoms for group A streptococcal pharyngitis. Pediatrics 2016; 138: e20160317.

22.

Schwartz

Monsur

, et al. Patient–clinician agreement on signs and symptoms of ‘strep throat’: a MetroNet study. Fam Pract 2004; 21: 599–604.

23.

Akhtar

Van Heukelom

Ahmed

, et al. Telemedicine physical examination utilizing a consumer device demonstrates poor concordance with in-person physical examination in emergency department patients with sore throat: a prospective blinded study. Telemed J E Health 2018; 24: 790–796.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.07 MB