Abstract
Introduction
Treatment of acute ischemic stroke patients largely depends on achieving reperfusion of the brain tissue. 1 Mechanical thrombectomy (MT) has recently been shown to have a high success rate in achieving reperfusion and improving functional outcome in patients with acute stroke secondary to large vessel occlusion (LVO). 2 –8 MT is primarily available in thrombectomy-capable stroke centers (TSCs), which include large primary stroke centers, and comprehensive stroke centers (CSCs). Under the current system of care, patients presenting to community hospitals are evaluated through a telestroke network for eligibility for tissue plasminogen activator, and are transferred to the nearest TSC for thrombectomy evaluation when an LVO is suspected. Suspicion of LVO largely depends on the National Institute of Health Stroke Scale (NIHSS) and the presence of cortical signs. However, this approach is not unified and often lacks specificity, which results in unnecessary transfers of patients without LVO to TSC (false transfer). In a case series of 157 patients who were transferred for thrombectomy evaluation, only 46% underwent MT, 9 and the most common reason for false transfer was the absence of LVO. False transfer does not only increase TSC volume workload but also remarkably increases the cost of patients' care and complications secondary to transfer. 10
Multiple efforts have been made to develop clinical scales that could identify patients with LVO in a prehospital setting with promising results. 11,12 However, none of these scales have been validated in telestroke setting. In this study, we aimed to investigate the accuracy of LVO scores in identifying LVO in telestroke patients and compare it with the NIHSS.
Methods
SETTING
The Medical University of South Carolina (MUSC) telestroke program was established in 2008 to provide evidence-based stroke care to patients in rural South Carolina. MUSC telestroke network works in a hub (MUSC main hospital–the CSC) and spoke (community hospitals) paradigm. 13 The number of participating spoke sites covered by the MUSC telestroke network has increased from 5 sites in 2008 to 26 sites in 2017. 14 MUSC telestroke program was detailed in previous report. 13
STUDY POPULATION
We performed a retrospective chart review of all consecutive telestroke patients who were transferred to MUSC hub for suspected LVO during study period from May 2014 to March 2018. Exclusion criteria were patients with hemorrhagic stroke on initial computed tomography (CT) scan at spoke center (before transfer), patients who received tissue plasminogen activator (tPA) at spoke before transfer, and patients who did not receive vessel imaging after arrival to MUSC. We excluded tPA-treated patients because tPA could lead to recanalization in 20–50% of LVO and subsequently could have led to a false negative CT angiogram ultimately resulting in result bias. 15 The following data were collected from the charts: age on presentation, gender, race, vascular risk factors, NIHSS before transfer, whether patient had LVO on CT angiography (CTA) after arrival to the TSC, and the location of LVO on cerebral vessel imaging.
LVO was defined as an occlusion of the proximal or distal internal carotid artery, first or second segment of the middle cerebral artery (M1 and M2), first segment of posterior cerebral artery (P1), first segment of anterior cerebral artery (A1), vertebral artery, and basilar artery.
LVO RECOGNITION SCALES
We used NIHSS that was calculated and documented by the consulting stroke neurologists during the telestroke consults to calculate the following scales: 3-item stroke scale (3I-SS), 16 Rapid Arterial Occlusion Evaluation Scale (RACE), 17 Cincinnati Prehospital Stroke Severity Scale (CPSSS), 18 Prehospital Acute Stroke Severity Scale (PASS), 19 and the Field Assessment Stroke Triage for Emergency Destination (FAST-ED) stroke scale. 20 We used the cutoffs that were suggested by the initial publication for each scale (Table 2).
STATISTICAL ANALYSIS
Demographic variables for the study participants were described as mean and standard deviation (SD) for quantitative variables or as frequency and percentage for categorical variables.
We calculated the false-positive (1-specificity) and false-negative (1-sensitivity) rates, sensitivity, specificity, accuracy, positive, and negative predictive values of each score. We used the published cutoffs to first determine these values. Secondarily, we considered all possible cutoff values for a given scale to identify the cutoff that maximized the sum of sensitivity and specificity. This study was approved by the Institutional Review Board of MUSC.
Results
During the study period, a total of 1,130 patients were transferred to the TSC. Of whom, 511 patients fit the inclusion criteria (Supplementary Fig. S1). Seventy-two (14%) patients were excluded due to missing NIHSS in their record, leaving 439 patients to be included. Study flowchart is in the supplemental material. Baseline characteristics of our patients are detailed in Table 1. Mean (SD) age was 66.7 years (15.3), and 47.3% of patients were female. Mean (SD) NIHSS was 14.2 (7.9). Two-hundred thirteen patients (48.5%) had LVO confirmed on CTA upon arrival to the TSC. Among patients with confirmed LVO, 194 (91%) patients had occlusion of the anterior circulation arteries, and 19 (9%) patients had occlusion of the posterior circulation arteries.
Baseline Characteristics of Study Cohort
Values are n (%).
Values are in mean (SD).
Afib, atrial fibrillation; CAD, coronary artery disease; DM, diabetes mellitus; HLD, hyperlipidemia; LVO, large vessel occlusion; NIHSS, National Institute of Health Stroke Scale; SD, standard deviation.
Table 2 presents the overall accuracy, false negative rate (FNR), and false positive rate (FPR) for each of the stroke scales. FNR represents the proportion of patients with LVO that would not have been transferred to the hub despite LVO (missed LVO), in contrast FPR represents the proportion of patients without LVO who could have been transferred to the hub (false transfer).
False Negative Rate, False Positive Rate, and Overall Accuracy of Each Score in Identifying Large Vessel Occlusion
3I-SS, 3-item stroke scale; CPSSS, Cincinnati Prehospital Stroke Severity Scale; FAST-ED, Field Assessment Stroke Triage for Emergency Destination; FNR, false negative rate; FPR, false positive rate; LL, likelihood ratio; PASS, Prehospital Acute Stroke Severity; RACE, Rapid Arterial Occlusion Evaluation Scale.
RACE score (using cutoff of 5) had the highest accuracy (78%). If RACE score was used to triage patients, 11% of patients would not have been transferred despite LVO. In contrast, using NIHSS of 6 would have resulted in only 3% of missed LVO but 73% of false transfer. Table 3 and Figure 1 depict the comparative performance of all included LVO scores using suggested cutoff values.

Comparative analysis of six scales in identifying LVO among patients transferred to TSCs for suspected LVO. LVO, large vessel occlusion; ROC, receiver operating characteristic; TSCs, thrombectomy-capable stroke centers.
Performance of Scales in Identifying Large Vessel Occlusion Using Published Cutoff
NPV, negative predictive value; PPV, positive predictive value.
In addition, we studied all possible cutoff points for each score to determine the cutoff with the highest sum of sensitivity and specificity. An NIHSS score of 11 had the highest sum of sensitivity and specificity. RACE, CPSS, FAST-ED, and PASS had the highest sum of sensitivity and specificity at the same cutoff values as those reported in the initial publication of each score. 17 –20 3I-SS had a higher accuracy and sum of sensitivity and specificity when using cutoff point of ≥3 instead of the suggested cutoff point of ≥4 that was suggested in the initial publication (Supplementary Table S1). 16
Most of the patients transferred had findings consistent with acute stroke on brain imaging. Stroke mimics were seen in 27 patients (6.2%) (Supplementary Fig. S2). The most common stroke mimic seen was postictal paralysis, which was seen in 12 patients. Other stroke mimics noted were sepsis in three patients, migraine with aura in two patients, recrudescence of old stroke in two patients, posterior reversible encephalopathy syndrome in two patients, sedating medications in two patients, hyperglycemia in one patient, hypertensive emergency in one patient, intracerebral hemorrhage that was not seen on CT at the spoke site in one patient, and subarachnoid bleeding in one patient.
All LVO scores had a poor accuracy in identifying stroke mimics (24–68%) (Table 4). 3I-SS had the highest accuracy (68%), however, FPR was 30%.
False Negative Rate, False Positive Rate, and Overall Accuracy of Each Score in Identifying Stroke Mimics
Discussion
This is the first study to investigate the accuracy of LVO recognition scales in telestroke patients who were transferred to TSCs for suspected LVO. We included RACE, PASS, CPSS, FAST-ED, and 3I-SS scores, along with NIHSS. We found that using LVO scores to triage telestroke patients could result in missing at least 10% of acute stroke patients with LVO. The only exception was when using NIHSS with a score ≤6, which had FNR of 3% but FPR of 73%. In our study, RACE score (cutoff of 5) had the highest accuracy, which accords with previous studies in the prehospital setting. 21
One of the most challenging problems in telestroke is making the decision regarding transfer to a TSC versus no transfer. Since the publication of the five pivotal thrombectomy trials in 2015, a remarkable emphasis has been placed on developing tools that could identify LVO in patients with suspected ischemic stroke. 22 Los Angeles Motor Scale (LAMS), FAST-ED, 3I-SS, CPSS, PASS, and RACE were the most widely used and studied clinical scales. In a comparative study of LVO scores in tPA patients, accuracy, sensitivity, and specificity were in the range of 0.74–0.78, 50–69%, and 83–89%, respectively. 19 Another comparative study of eight scales in ED patients with cerebral ischemia showed lower accuracy (0.62–0.7) and sensitivity (42–64%) than the previously mentioned study, but higher specificity (81–96%). Discrepancy of the results could be attributed to different inclusion criteria and different LVO definitions. 12 The largest comparative study to date evaluated the performance of 13 different clinical stroke scales in 1,004 ischemic stroke patients. In that study, the author found that using a published cutoff point for the LVO scores was associated with FNR of at least 20%. 21
It should not be surprising that sensitivity (ranging from 47% to 97%) in our study was higher than that of previously published studies. 11,21 This finding has multiple potential explanations. First, we excluded patients with hemorrhagic stroke and included only patients who were transferred to the hub for suspected LVO. Second, we considered M2, P1, and A1 as LVO due to accumulating evidence, suggesting the benefit of thrombectomy in these groups of patients. 23,24 Third, all scales were derived from NIHSS that was documented by a stroke neurologist.
Identifying stroke mimics is challenging in telestroke setting and relies mainly on the examiner expertise and available brain imaging. In our study, 6.2% of the patients had stroke mimics, which is lower than that reported in the literature (20–40%). 25,26 The low stroke mimics rate in our study stems from the fact that all patients included in our study were evaluated by stroke neurologist through telestroke and were deemed to have possible LVO before transfer. Even though multiple efforts have been made to develop scores that can be used to differentiate stroke mimics for ischemic stroke, to date there is no reliable scale that is routinely utilized. 25,26 In our study, all studied LVO scores did poorly in identifying stroke mimics.
LVO recognition scores generally rely on motor deficit with or without cortical findings to predict LVO. It is not uncommon for non-LVO stroke patients to present with severe hemiparesis or cortical findings mimicking LVO strokes. 27 –29 Moreover, up to 7% of patients with LVO could present with NIHSS of 0. 30 In a study of 565 ambulance-initiated stroke alerts, 38% of anterior circulation LVO patients presents with atypical presentation. 11 LVO scores were able to identify anterior circulation LVO in <20% LVO patients with atypical presentation versus ≥95% of LVO patients with typical presentation. 11
After the extension of thrombectomy eligibility with the publications of DAWN and DEFUSE trials, 7,8 patients transfer will likely remarkably increase, likewise, false transfers will probably increase. A study in the United States by Sonig et al. evaluating 1,311,511 inpatient stroke admissions from 2008 to 2010 revealed that transferred patients undergoing MT and tPA had an average expenditure of $27,000 more than nontransferred patients. 10 In addition to increasing cost, false transfer increases workload in the TSC and patient/family discomfort. Thus, a more sensitive and specific tool is urgently needed to improve telestroke triage system and decrease false transfer.
CT angiogram and perfusion are now increasingly incorporated in telestroke. 31 A simulation study by Boulouis et al. evaluated 508 patients transferred from spoke sites for thrombectomy evaluation and found that doing vascular imaging at spoke sites can be a useful triaging method and can avoid 20% of futile transfers. 32 In addition, obtaining vascular imaging at spoke before transfer can help in not only identifying intracranial occlusions but also assessing collateral blood vessels, which may further provide additional information to determine thrombectomy candidacy. 33,34
Several experimental technologies have been proposed for detection of LVO. The volumetric impedance phase shift spectroscopy (VIPS) device is a noninvasive device capable of detecting an asymmetric of bioimpedance between two hemispheres that could result from ischemic brain injury. 35 A pilot study investigating the accuracy of VIPS device in detecting LVO in patients transferred from satellite hospitals to CSCs for suspected LVO demonstrated a sensitivity of 86% and a specificity of 77% for LVO. 35
Our study has multiple limitations that should be considered while interpreting our results. First, LVO recognition scales were scored based on NIHSS, which did not allow us to evaluate other LVO recognition scores that require elements not included in NIHSS such as grip strength. However, in previous studies evaluating LVO scores, RACE and FAST-ED had the highest accuracy, therefore, including other LVO scores would unlikely alter our study results. Second, retrospective design is a methodological shortcoming. Notably, our data including NIHSS were collected prospectively and updated periodically as part of our telestroke registry, so while the records were retrospectively reviewed, the NIHSS score was collected at the time of patient evaluation. Finally, patients who received tPA were not included in the study, which could be a source of bias given that tPA patients may have higher likelihood of having “real stroke” and, arguably, may have higher likelihood of having LVO. However, tPA is associated with considerable rate of recanalization, which may alter the results of the analysis.
Conclusion
Our study showed that LVO scores should not be used to triage telestroke patients, and until we have better tools, all patients with clinical examination concerning for LVO should be transferred to TSCs.
Footnotes
Acknowledgment
The project described was supported by the NIH National Center for Advancing Translational Sciences (NCATS) through Grant Number UL1 TR001450.
Statement of Compliance
Authors confirm that the study is an observational minimal risk study and no consent is required per the Medical University of South Carolina Institute policy. Our study was approved by the institutional review board of the Medical University of South Carolina.
Authors' Contributions
M.A. performed study concept and design, data interpretation, and article writing; E.A. was involved in data collection, data interpretation, and article writing; A.H.W. carried out statistical analysis; C.A.H. contributed to critical revision of the article for important intellectual content and study supervision.
Disclosure Statement
No competing financial interests exist.
Supplementary Material
Supplementary Figure S1
Supplementary Figure S2
Supplementary Table S1
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
