Efficacy of Telemedicine for Stroke: Pooled Analysis of the Stroke Team Remote Evaluation Using a Digital Observation Camera (STRokE DOC) and STRokE DOC Arizona Telestroke Trials

Introduction

National recombinant tissue plasminogen activator (rt-PA) administration rates for acute ischemic stroke continue to be low in spite of U.S. Food and Drug Administration approval over a decade ago. rt-PA treatments are noted to be particularly low in small hospitals and those located in less densely populated areas. ¹ A significant proportion of the American population lives in rural areas. Current data report that only 55% of Americans have access to primary stroke centers within 60 min. ² Rural patients may not have immediate access to acute stroke expertise and may not be afforded rt-PA therapy. ³ Acute ischemic stroke patients presenting to rural emergency departments are approximately 10 times less likely to receive rt-PA than those presenting to urban primary stroke centers. ³

There is a demonstrable gap in access to stroke specialists and specialty care in rural communities. ¹ Telemedicine may be one method to minimize this gap in immediate and appropriate access to care. ³ Telemedicine's reliability for stroke syndrome assessments has been published. ^4,5 Two randomized assessments of stroke telemedicine efficacy for decision-making and long-term outcomes have been conducted and published: the Stroke Team Remote Evaluation Using a Digital Observation Camera (STRokE DOC) trials, in 2008 and 2010. ^6,7 The National Institutes of Health (NIH)–funded STRokE DOC trial was developed to assess acute decision-making efficacy in stroke telemedicine. ⁶ The primary outcome measure of correctness of treatment decision was adjudicated at three levels of data availability and showed a significant benefit of telemedicine over telephone (Primary Outcome—Level 2b Blinded Adjudication: Correctness of Decision-Making, 98% versus 82%; odds ratio [OR] 10.9; 95% confidence interval [CI] 2.7–44.6; p=0.0009). This pattern also held for the rt-PA subset of subjects (97% versus 76%; OR 7.4; 95% CI 1.03–53.2; p=0.0466). The trial reported a high rt-PA utilization rate for both modalities at 25% overall (28% telemedicine, 23% telephone). Safety of telemedicine- and telephone-guided rt-PA assessments was apparent with no difference in overall post-consult intracranial hemorrhage (ICH) rate (7% versus 8%; OR 0.8; 95% CI 0.1–6.3; p=1.0). Outcome measures were not statistically different for 90-day Barthel Index (BI) (p=0.13), 90-day modified Rankin Scale (mRS) (p=0.09), or 90-day mortality (p=0.27). The rt-PA subgroup mortality was not significantly different after adjusting for baseline NIH Stroke Scale (NIHSS) score severity (p=0.17).

The STRokE DOC Arizona (STRoke DOC AZ) trial was a planned expansion of the original STRokE DOC trial, and its objectives were twofold: To determine the feasibility of establishing a single-hub, multi–rural spoke hospital telestroke network in Arizona by building upon and replicating the first STRokE DOC trial. We sought to determine whether telemedicine or telephone was superior for decision-making in acute stroke consultations in a different state among different hospitals, clinicians, and investigators. Identical to the original STRokE DOC trial's design, the STRokE DOC AZ trial was a prospective, single-hub, multi-spoke, randomized, blinded outcome trial of two consultative techniques for acute stroke evaluations. The population was adult stroke patients. The intervention was telemedicine versus telephone-only. The outcomes were correct rt-PA decision (compared with adjudication committee), rt-PA rate of use, 90-day outcomes, and ICH. Fifty-four patients were enrolled (27 in telemedicine, 27 in telephone). The rt-PA utilization rate was 30% (30% telemedicine, 30% telephone). The overall proportion of correct treatment decision was 87% (85% telemedicine, 89% in telephone; p>0.99). Telephone outperformed telemedicine but without statistical significance in the 90-day achievement of BI (95–100) (59% telemedicine, 58% telephone; p=0.77), 90-day achievement of mRS (0–1) (46% telemedicine, 38% telephone; p=0.61), 90-day mortality (4% telemedicine, 11% telephone), and post-rt-PA ICH (4% telemedicine, 0% telephone; p>0.99). The trial concluded that it was feasible to extend the STRokE DOC protocol to a new state and establish an operational multi-spoke telestroke research network in Arizona. Whether by telemedicine or telephone, there were appropriate treatment decisions, high rt-PA utilization rates, and low ICH complications.

Figure 1 shows the STRokE DOC telestroke hub and spoke network dynamic in neighboring states of California and Arizona.

OPEN IN VIEWER

Fig. 1.

Stroke Team Remote Evaluation Using a Digital Observation Camera telestroke hub and spoke network dynamic in California and Arizona.

A planned pooled analysis of the two identical acute stroke protocols—STRokE DOC (positive trial) and STRokE DOC AZ (negative trial)—was performed to assess the best estimate of the direction and magnitude of rt-PA eligibility decision-making with telemedicine compared with telephone-only consultations in a larger sample size of acute stroke syndrome patients presenting to emergency departments at remote hospitals without neurologists on-call.

This research has been previously presented as an oral platform presentation at the 2010 International Stroke Conference and published only in abstract form. ⁸

Subjects and Methods

In the original STRokE DOC trial (University of California San Diego) (n=222) acute stroke patients were prospectively randomized via a secure Specialized Programs of translational Research in Acute Stroke Web site to telemedicine or telephone consultation from four spoke centers in California. The STRokE DOC AZ trial (Mayo Clinic) (n=54) independently replicated the original trial design with its own two spoke centers in Arizona. For each trial, the primary outcome measure was correctness of treatment decision, as determined by central blinded adjudication. Details of the design and statistical methods for each of the two trials have been published. ⁹ Common data elements were pooled to assess for correctness of thrombolysis decision-making. Secondary outcomes included rt-PA use rate, 90-day functional outcome, hemorrhage, and data completeness.

All analyses were prespecified and based on the intent-to-treat population. Baseline and demographics characteristics between studies (University of California San Diego versus Mayo Clinic) and between treatment arms (telemedicine versus telephone) were compared using Wilcoxon rank sum tests for continuous variables and Fisher's exact tests for categorical variables.

Results from the two trials were combined using a center-stratified Cochran–Mantel–Haenszel estimate of the OR and 95% CI. The Cochran–Mantel–Haenszel test, stratified according to the participating center, was used to compare the primary outcome—correct decision rates at Level 2 adjudication—between the telemedicine and the telephone groups. Homogeneity of ORs across centers was assessed using the Breslow–Day test.

A Fisher's exact test was used to compare the other correct decision rates, rates of thrombolytic use, the rate of ICH, mortality rates, and 90-day mRS score between treatment groups, whereas the Wilcoxon rank sum test was used for the 90-day BI comparisons.

All statistical analyses were done using the statistical software R version 2.7.0. ¹⁰ All the analyses were two-sided, and the significance level was set at a two-tailed p<0.05. No adjustment for multiple comparisons was made for the secondary outcomes.

Results

There were no differences for baseline characteristics or risk factors between STRokE DOC and STRokE DOC AZ except for ethnicity and elements driven by data collection “unknowns.” There were no differences in baseline stroke severity, glucose measurements, or baseline computed tomography (CT) scan findings. Overall, there were only minimal differences (insignificant heterogeneity) between the two studies. Therefore, a pooled analysis was justified and performed.

Two hundred seventy-six combined patients were prospectively evaluated. Mean age was 69±14.5 years. Fifty-one percent were female. Pooled baseline characteristics were consistent with the original STRokE DOC trial (Table 1).

Mean NIHSS score was 9.1 (10.6±8.37 telemedicine, 7.7±6.89 telephone; p=0.006). Similar to the original STRokE DOC trial, increased baseline stroke severity was noted in the telemedicine arm (NIHSS score, 10.6 versus 7.7). Increased abnormal baseline CT scans were noted in the telemedicine arm. Neither the telemedicine patients' increased NIHSS nor increased abnormal baseline CT scans were adjusted for because these abnormalities may have been an artifact of improved data collection and direct viewing of images in the telemedicine arm of the trial (Table 2).

Telemedicine consults took 8 min longer, on average, than telephone consults (duration, 35.4 min versus 27.1 min) but resulted in improved decision-making (Table 3).

Correctness of decision-making was found to significantly favor telemedicine in this pooled analysis (96% telemedicine, 83% telephone; OR 4.2; 95% CI 1.69–10.46; p=0.002) (Table 4).

Intravenous rt-PA use rate was 26% overall (29% telemedicine, 24% telephone; OR 1.27; 95% CI 0.71–2.25; p=0.41). The 90-day outcomes were not different for BI (95–100) (46% telemedicine, 55% telephone; p=0.17), for mRS, dichotomized 0–1 (36% telemedicine, 45% telephone; p=0.20), or for mortality (16% telemedicine, 12% telephone; p=0.49). There was no difference in post-rt-PA ICH (8% telemedicine, 6% telephone; p>0.999) (Table 4). There was a notable difference in amount of incomplete data (3% telemedicine, 11% telephone; p=0.004).

Telemedicine resulted in safe rt-PA eligibility assessments and administration with post-thrombolysis ICH in 7% overall. The long-term outcomes were acceptable with mRS (0–1) achieved in 40% and mortality of 14% overall. Among acute stroke syndrome patients at participating network spoke hospitals who received a telemedicine consultation, 29% were determined to be eligible for and received rt-PA.

The rt-PA subgroup outcomes were favorable and not different between groups. These results were consistent with those of the published literature and, taken together, provide reasonable reports of telemedicine outcome expectations for rt-PA-treated patients (comparison from the published literature: BI [95–100], 45–47%, mRS [0–1], 38–43%, and mortality, 11–20%). ¹¹

Discussion

One STRokE DOC trial was positive, and the second was negative. The second trial, STRokE DOC AZ, may have resulted in a different outcome because it was not designed to have sufficient power to reveal a significant difference between the two treatment arms should a true difference have existed. If, in fact, the null hypothesis is false and telemedicine is superior to telephone, then an underpowered trial may not have revealed a statistically significant difference by chance alone, resulting in a Type II error. If the null hypothesis is true and telemedicine is not superior to telephone, then there remains a very small probability that the STRokE DOC trial resulted in a Type I error. Most likely some factor(s) may have differed between the two trials, resulting in a different outcome. For example, the STRokE DOC AZ telemedicine consultations may have underperformed compared with the original STRokE DOC trial because of the relative telemedicine inexperience of the hub and spoke healthcare personnel and the high proportion of technical observation witnessed by the telemedicine consultations. ⁷ The STRokE DOC AZ telephone-only consultations may have overperformed compared with STRokE DOC for the following reasons: The STRokE DOC AZ TIME trial authorized a 1-month telephone-only run-in experience before randomization, and the STRokE DOC AZ TIME trial vascular neurology consultants devoted approximately the same time to telephone consultations as they did to telemedicine consultations. ⁷ Consultants regularly reacquired the history directly by telephone from patient, family members, and witnesses in addition to requesting repeat neurological assessments from emergency department nurses and physicians. ⁷ A testament to the thoroughness of the telephone consultations was the comparable completeness of data collection between telephone and telemedicine modalities. ⁷

Telemedicine can make a significant impact upon stroke diagnosis, treatment, and quality of outcomes. Telemedicine provides patients with stroke immediate access to remote neurology practitioners in circumstances and communities lacking local neurology support. Telemedicine is superior to telephone-only for establishing an accurate diagnosis of stroke. A summary of the published experience of 12 telestroke networks demonstrated that the thrombolysis administration rate can increase to 23%, approximately 10-fold from baseline, without jeopardizing protocol violation or hemorrhagic complication rates. ¹² Good 6-month outcomes have been reported for stroke patients evaluated by telemedicine, with 18% mortality and 86% of survivors residing at home. ¹³ Compared with telephone-only assessments, lower mortality and trends toward lower morbidity and dependency on institutionalized care have been reported for stroke patients who have undergone emergency telemedicine consultations. ¹⁴

Telemedicine for stroke is practical and affordable and will be supported by physicians, hospitals, and insurance companies across the nation. Despite any barriers to adoption, already 100 telestroke programs in 43 states have been identified, and the mean number of spoke hospitals each hub serves has doubled in 2 years, from four to eight. ¹⁵ Therefore, an estimated 800 spoke hospitals in the United States are currently supported by telestroke networks. Existing barriers ¹⁶ to telestroke network implementation are gradually being addressed. On-call stipends can help to incentivize telemedicine practitioners. Cost of equipment, formerly formidable, is becoming more affordable. Students in all clinical health sciences are beginning to learn, at a formative stage of their education, the critical role telemedicine plays. Redundancy and duplicity of credentialing and privileging have been successfully addressed by the new Centers for Medicare and Medicaid Services rule that took effect in July 2011. ¹⁷ The rule results in streamlining of credentialing and privileging of telemedicine providers who deliver services to Medicare hospitals. A broad range of successful business models for telestroke has been constructed and studied. Emerging evidence from national and global health economic research suggests that telestroke practice is highly cost-effective, which should serve well to motivate government and non-government insurers to adjust their positions on reimbursement. ¹⁸

In the United States, 40% of inhabitants reside in counties without a hospital actively engaged in acute stroke care. ¹ The American Heart Association/American Stroke Association recommends that whenever local or on-site acute stroke expertise is insufficient to provide around-the-clock coverage for a healthcare facility, telestroke systems should be deployed. ¹⁹ Although NIHSS-telestroke examinations using high-quality videoconferencing is granted a Class I, Level A recommendation, providing an opinion in favor or against the use of intravenous rt-PA is classified as Class I, Level B. The original STRokE DOC trial was included in this evidence analysis, but STRokE DOC AZ and STRokE DOC pooled analysis was completed after the time points of included data for those recommendations. In the two published randomized controlled trials, STRokE DOC and STRokE DOC AZ, and the STRokE DOC pooled analysis multiple populations have been evaluated in two states and in two telestroke networks. Telemedicine is clearly more effective than telephone-only consultation for the determination of thrombolysis eligibility. Effective replication of the hub and spoke telestroke network supports the concept of generalizability and implementation in broader settings. Structured telephone-only consultations, as used in STRokE DOC trials, may provide an adequate backup plan in instances when technology fails but have proven to have poor sensitivity for thrombolysis eligibility determination compared with adjudication committee standard. ²⁰ In effect, with telephone-only, the stroke specialist is more likely to erroneously judge a stroke patient ineligible for rt-PA. The favorable effect or influence of a supportive telestroke network, found in both arms of the STRokE DOC trials, was unmeasured but not ignored. Factors such as the stroke network community, stroke systems of care, around-the-clock availability of stroke specialists, education, mentorship, follow-up, and feedback may all have served to elevate the level of spoke hospital stroke care for both arms of the trials. In a mature hub and spoke telestroke network, any treatment effect exerted by consultative mode may be even more difficult to detect.

A limitation of the pooled analysis is that it was underpowered to detect differences in 90-day functional outcome. Using the rates of 90-day outcomes published in the two STRokE DOC trials, for a power of 0.80 and alpha of 0.05, a pooled analysis should include approximately 340, 240, and 610 subjects per group, respectively, to detect a difference in BI, mRS, and overall mortality.

For those areas where telemedicine is not possible to implement, are there any lessons from the trials that could improve upon the quality and effectiveness of telephone-only stroke consultation? Telephone-only stroke consultations can serve as an adequate ancillary, adjunctive, supplemental, or emergency backup modality for a telestroke network. ²¹ To support communities for which telemedicine is not yet feasible, it is advisable to implement a rigorous telephone stroke algorithm. Telephone consultations for stroke can be improved with advanced community, emergency medical services, and hospital-directed emergency stroke education modules and evidence-based stroke guideline adoption and electronic order sets, care plans, and pathways. Hub neurology practitioners are encouraged to first visit remote community hospitals that they intend to later support by telephone. When emergency nurses and physicians, in a remote hospital, are trained or certified in NIHSS assessments, their neurological examination observations are more reliable and easier to communicate. Paper or electronic telephone stroke algorithms and sequential step-by-step flow charts can aid health practitioners during emergency situations. Adopting a common thrombolysis eligibility checklist is advisable, particularly for remote or rural hospital staff who might ordinarily be infrequently engaged in acute stroke care.

Other carefully designed research studies, such as the National Stroke Association–sponsored prospective tri-cohort study, will examine for differences in acute stroke care delivery and 90-day functional outcomes among control community hospitals, telestroke network spoke community hospitals, and telestroke network primary stroke center–certified hub hospitals in multiple states (at ClinicalTrials.gov see Trial Registration Number NCT01226862, Study Protocol—Advancing Telestroke Care: A Prospective Observational Tri-Cohort Study).

Conclusions

STRokE DOC pooled analysis data support the primary hypothesis that stroke telemedicine consultations, compared with telephone-only, result in more accurate medical decision-making. Replication of the hub and spoke infrastructure supports telemedicine's generalizability into broader settings. Poor sensitivity of telephone determination of thrombolysis eligibility suggests that telephone assessments may result in stroke consultants ruling out patients who should have been treated with rt-PA.

The larger patient sample size remains insufficient to demonstrate differences in 90-day patient outcomes. The superior decision-making, high rt-PA utilization rate, low ICH complication rate, and acceptable outcome support telemedicine's efficacy and usefulness as a tool for acute stroke evaluation. A larger prospective trial specifically assessing long-term outcomes in telemedicine-evaluated acute stroke patients is still warranted.