Patient-specific prediction of functional recovery after stroke

Introduction

Stroke is the most common cause of adult physical disability and the third most common cause of death worldwide.^1–3 It is estimated that 25–74% of the 50 million stroke survivors worldwide require some assistance or are dependent on caregivers for activities of daily living.^4–6

Despite the introduction of effective treatments for acute stroke, early rehabilitation and secondary prevention, the majority of stroke survivors have medical comorbidities, physical, and/or cognitive impairments that require ongoing active assessment and management. ³ Clinical prediction of the recovery is an important medical aim for the many people who survive stroke. First, it could help early planning of rehabilitation and longer term management. Second, it might provide early warnings or triggers for those who fail to recover as predicted. Third, knowing in advance when a poor health outcome might occur would allow survivors and their carers to plan accordingly. Therefore, a practical and accurate tool to assist in predicting functional recovery after stroke at patient level could provide significant aid for patient management and cost of health delivery.^7,8 Furthermore, a prediction model of functional recovery trajectory can be useful as a tool for clinical research, health economics policy making, and clinical decision support. ⁹

Recent published prognostic tools for stroke functional recovery include the ASTRAL (90% accuracy for classifying modified Rankin Scale (mRS) > 2 at three months), ⁹ the PLAN score (89% accuracy for predicting mortality and mRS 5–6 combined at discharge), ¹⁰ six simple variables model (84% accuracy for mortality and mRS < 3 combined at six months), ¹¹ and the iScore (79% accuracy for mortality or mRS 3–5 combined at 30 days). ¹² However, these studies used predictions for one specific time point which does not allow for the complexity of changes of functional recovery over time or for the variety of confounders that may alter the functional recovery trajectory. Furthermore, they do not distinguish between whether higher functional ability is due to an overall improved functional ability that is already present at the early stage, or due to improved functional recovery over time, or due to both. Thus, they have limited utility for clinical applications which require ongoing active assessment and monitoring of stroke patients.

Stroke recovery is a complex process in its nature, and individual functional recovery patterns differ across subpopulations and time. ¹³ Current practice makes use of clinicians’ expert judgments based on a number of factors at the onset such as age, stroke severity, consciousness, and stroke subtype. A recovery curve predictive model could potentially provide an equivalent prognostic tool that could support decision-making process at any time after stroke. Detailed longitudinal data and methods may be more appropriate to accurately capture stroke recovery with the aim of designing a tool that could assist in planning immediate and midterm care for individual patients.^14,15 Several studies have hypothetically suggested that recovery of body function and activities could be predictable after stroke.^14–16 In this study, we developed and validated a novel patient-specific predictive model of functional stroke recovery trajectories over time and evaluated its clinical utility.

Materials and methods

Results

Of the 462 patients in the development cohort, 113 died (24.4%) during the study. The study population consisted of 252 females (50.9%). Average age was 71 years (range 22–98 years). The external temporal validation cohort has similar characteristics (Table 1).

OPEN IN VIEWER

Table 1.

Characteristics of stroke patients

	Development cohort (2002–2004)	Validation cohort (2005–2011)
Number of patients	495	1049
Number of observations	2773	2159
Age group, n (%)
0–64	144 (29.1)	368 (35.1)
65–74	129 (26.1)	241 (23.0)
75–84	144 (29.1)	278 (26.5)
85+	78 (15.8)	162 (15.4)
Sex, n (%)
Female	252 (50.9)	490 (46.7)
Ethnicity, n (%)
White	362 (73.1)	690 (65,8)
Black	94 (19.0)	275 (26.2)
Other/unknown	39 (7.9)	84 (8.0)
Prestroke disability, n (%)
Severe/moderate (BI: 0–14)	41 (8.3)	67 (6.6)
Mild (BI: 15–19)	80 (16.2)	142 (14.0)
Independent (BI: 20)	359 (72.5)	809 (79.5)
Glasgow coma scale, n (%)
Severe	42 (8.5)	105 (10.0)
Moderate	70 (14.1)	147 (14.1)
Mild	364 (73.5)	796 (75.9)
NIHSS
Median score (IQR)	6 (4–11)	7 (3–13)
Subtype, n (%)
TACI	78 (15.8)	126 (12.0)
PACI	126 (25.5)	373 (35.6)
POCI	66 (13.3)	128 (12.2)
LACI	146 (29.5)	300 (28.6)
Hemorrhage	67 (13.3)	122 (11.6)
Service provision, n (%)
Hospital (nonstroke unit)	90 (18.2)	132 (12.6)
Stroke unit	379 (76.6)	850 (81.0)
Unknown/ nonhospital	26 (5.3)	67 (6.4)

The predictive recovery curves showed an excellent fit and prediction. In the internal cross-validation, predictive error RMSD over all time points was 3 BI points, mean absolute error 2 BI points, and a R² of 77%. Predictive accuracy of the model was also good in the external validation cohort with a RMSD of 2.83 BI points, mean absolute error of 1.72 BI points, and a R² of 83%. Figure 1 presents the average predicted recovery patterns with comparison to the average observed BI after stroke. OPEN IN VIEWER

Average recovery was characterized by a steep increase in BI over the first eight weeks followed by a more gradual period of recovery reaching its apex at around 12 weeks. Depending on patient characteristics considered in the model, predictive curves showed peaks ranging from 8 to 32 weeks. Recovery was inversely associated with NIHSS and age, and had a positive association with GCS. Subtype had a variable influence on recovery, with POCI and LACI showing the most immediate (∼8 weeks) recovery with the highest, and roughly equivalent, peaks. Hemorrhage and PACI showed the next quickest and complete recovery, by 12 weeks. TACI showed longer spontaneous neurological recovery, peaking at around 32 weeks and a marked decline subsequently. Figure 2 presents recovery patterns after stroke stratified by age group, stroke subtype, GCS, and NIHSS. OPEN IN VIEWER

Recovery curve models were further evaluated to define a set of rules with appropriate sensitivity and specificity to usefully identify patients with poor recovery patterns. Figure 3 presents discrimination and calibration analyses of predicting functional outcomes at three and 12 months in the temporal external validation cohort. The sensitivity and specificity in the external validation at three months were excellent: 71 [58.8–81.3] and 96 [93.6–97.4] for poor outcome, 88 [84.3–90.7] and 82 [73.7–89.0] for good outcome, and 80 [74.9–83.7] and 83 [77.2–87.9] for community independence. OPEN IN VIEWER

The model has shown also potential utility, expressed by predictive values for the risk of poor outcomes. The negative predictive values of recovery curves were over 90% for poor outcome, 96% [93.6–97.4] and 93% [90.8–95.3] at three and 12 months, respectively. Table 2 presents predictive values and likelihood ratios for classifying each functional outcome of interest.

OPEN IN VIEWER

Table 2.

Prognostic performance and clinical utility for recovery outcomes at three and 12 months in the validation cohort

	Measure	3 months	12 months
Poor outcome (cutoff = 8)
Prevalence	Prevalence (BI < 8)	13%	11%
Overall prognostic performance
	Overall performance (Brier)	6%	7%
	Discrimination (AUC)	94% [90.4–97.0]	89% [84.1–93.5]
	Calibration (H–L test)	chi2 = 1.5 (p = 0.908)	chi2 = 1.2 (p = 0.879)
Prognostic performance at a cutoff (<8)
	Sensitivity	71 [58.8–81.3]	47 [34.3–59.8]
	Specificity	96 [93.6–97.4]	96 [93.4–97.2]
Clinical utility at cutoff (<8)
	PPV	71 [58.8–81.3]	58 [43.2–71.3]
	NPV	96 [93.6–97.4]	93 [90.8–95.3]
	LR+	17 [10.8–26.8]	11 [6.5–17.2]
	LR−	0.3 [0.21–0.44]	0.6 [0.44–0.70]
Good outcome (cutoff = 12)
Prevalence	Prevalence (BI > 12)	80%	82%
Overall prognostic performance
	Overall performance (Brier)	8%	9%
	Discrimination (AUC)	91% [87.4–94.3]	87% [82.5–90.8]
	Calibration (H–L test)	chi2 = 1.9 (p = 0.762)	chi2 = 0.6 (p = 0.904)
Prognostic performance at a cutoff (BI > 12)
	Sensitivity	88 [84.3–90.7]	87 [84.1–90.3]
	Specificity	82 [73.7–89.0]	73 [62.9–81.2]
Clinical usefulness at cutoff (BI > 12)
	PPV	95 [92.8–97.2]	94 [91.0–95.8]
	NPV	62 [53.5–70.0]	55 [46.4–64.1]
	LR+	5 [3.3–7.4]	3 [2.3–4.4]
	LR−	0.1 [0.11–0.19]	0.2 [0.13–0.23]
Community independence (cutoff = 17)
Prevalence	Prevalence (BI > 17)	62%	62%
Overall prognostic performance
	Overall performance (Brier)	13%	17%
	Discrimination (AUC)	87% [83.4–89.9]	77% [72.7–80.7]
	Calibration (H–L test)	chi2 = 5.6 (p = 0.235)	chi2 = 0.8 (p = 0.858)
Prognostic performance at a cutoff (>17)
	Sensitivity	80 [74.9–83.7]	79 [73.9–82.8]
	Specificity	83 [77.2–87.9]	66 [58.8–71.9]
Clinical usefulness at cutoff (>17)
	PPV	89 [84.5–91.9]	79 [73.9–82.8]
	NPV	71 [64.8–76.6]	66 [58.8–71.9]
	LR+	5 [3.4–6.4]	2 [1.9–2.8]
	LR−	0.2 [0.20–0.31]	0.3 [0.26–0.41]

The net benefit as a function of a threshold probability of poor outcomes at three and 12 months was illustrated in Figure 4. The gray line was drawn to reflect the strategy of “assume all patients have poor outcomes” (i.e. recommend intervention for all), and the black line was drawn to reflect the strategy of “assume all patients would have good outcomes” (i.e. do not recommend any intervention). As expected, these two lines cross at the prevalence of poor outcome, 13% for three months and 11% for 12 months. The net benefit was maximized by the recovery curves predictive model (dash line) with threshold probabilities of 5–71% at three months and 7–57% at 12 months. For higher thresholds (>71% for three months or >57% for 12 months) where the concerns are more about unnecessary interventions than missed prognosis, the option to not intervene was preferred. In this case, strategies aided by recovery curves prediction have no value. OPEN IN VIEWER

Discussion

In this study, we have developed and validated a dynamic patient-specific prognostic tool of functional recovery poststroke in a population-based cohort. The proposed patient-centered model is the first that enables recovery to be predicted using a continuous score, derived from a combination of individual patient characteristics, and makes available adjusted functional trajectories up to 12 months. At predefined time points, the accuracy of this model has been shown to be superior compared to other exiting prediction models.^9–12

Patterns of functional recovery in a general unbiased stroke population demonstrate sharp improvements in the first weeks after stroke and then plateau. Patients with varying severity, different stroke subtype, and risk factors experience different recovery courses that plateaued at different levels and time. Although the general pattern of recovery from stroke, including functional and neurological impairments, has already been discussed and illustrated at population level in previous studies,^14,27 our proposed model is patient specific and has additionally provided the ability to predict accurately recovery trajectories up to one year poststroke. This could be useful for early rehabilitation and discharge planning, by predicting whether a patient is likely to be dependent, require some assistance, or be independent, at a certain time poststroke. To give a practical example, when considering functional recovery shortly after admission, the predictive recovery curves show similarities between hemorrhage and TACI stroke but large differences three months later; conversely, dissimilarities were observed between hemorrhage and PACI at the baseline but recovery after three months was comparable. This could be explained by the different rate of recovery between stroke types and reconfirms that survivors of hemorrhagic stroke have a greater chance for recovering function than those who suffer ischemic stroke as suggested in previous studies. ²⁸ Another case that we have tested is when patients have different, or combinations of risk factors, clearly distinct trajectories with different recovery rates can be expected and we can begin to predict poor functional outcomes. For instance, we have shown that changes in recovery do not vary between different age groups. There was a sharp initial increase in recovery which plateaued earlier in younger patients compared to older ones. But despite the slow recovery rate observed in old patients, we would expect improvement to the same level as that of younger patients. This would suggest that specialized stroke rehabilitation may be beneficial for all ages. It also confirms that older patients may need longer rehabilitation and are less likely to be discharged earlier. Interestingly, the change in recovery rate in severe stroke (poor GCS or NIHSS at onset) showed inverse relationship with recovery. This suggests that apart from old age, factors such as onset stroke severity should also be taken into consideration when planning interventions and rehabilitation after stroke. Therefore, this patient-specific recovery curve pattern prediction could potentially inform tailored discharge planning for stroke survivors including those with “hidden” impairments that have not been recognized by the medical team, for example in patients with communication problems. It could also be used to provide early warning of the development of stroke sequelae, such as medical complications, poorly managed risk factors, or inadequate rehabilitation. Evidence from a recent systematic qualitative review ²⁹ showed that rehabilitation could be improved by fostering patients’ autonomy through genuinely patient-centered care and more effective communication and information. Recovery curve predictions may play an important role not only to risks of poor recovery identification, but also to inform a joint physician–patient decision making for effective delivery of preventive and therapeutic interventions.

Generally, subjects’ stratification is often based on one characteristic variable at a specific time point, such as stroke severity at the onset of stroke, which may limit the clinical utility of their predictiveness. We have shown that using a dynamic (time-dependent) multivariate patient-specific predictive model, we can make individual recovery profiles and accurately classify future risks of outcomes. By maintaining this accuracy in differing patient populations, we believe that the recovery curve model is sufficiently robust and DCAs suggest it may be used clinically.

Although the general pattern of recovery from stroke, including functional and neurological impairments, has already been discussed and illustrated in previous studies,^14,27 our proposed model has additionally provided the ability to predict at any time the recovery of individuals up to one year poststroke. These patient-specific recovery curves predictions could allow more insight into both spontaneous and directed neurological recovery after stroke. Practically, this could offer the potential to use recovery curves to chart an individual’s recovery and identify when and why there is significant variance from the normal population curve in much the same way as normal growth curves are used to monitor the development of children. This prognostic information is important for clinicians, stroke survivors, and their carers.

In clinical research, this could also be applied as an aid in assessing the beneficial effects of evidence-based interventions and care settings. As a research tool, this could be used to test novel interventions or to identify enriched samples, reducing the reliance on the need for expensive and often impractical randomized controlled trials. This predictive enrichment strategy is of importance for designing future trials as it enables the enrollment of the most suitable patients thereby permitting the use of a smaller study population. Another potential application could be to derive a set of preliminary cost weights on resource uses which could help to build personalized patient care and funding models.

A key strength of the current study is that the models were built and externally validated using a prospective unbiased population-based cohort of first ever strokes. This is preferable over a hospital-based population, which may result in case-mix-specific models, or over aggregate data from clinical trials, which usually represent selected patients, and thus nonrepresentative populations. Notwithstanding, the study could be strengthened further if the model were to be validated in a completely independent holdout population in different poststroke care setting, preferably from another country. A limitation of this study is the effectiveness and usefulness of recovery curves as a low-cost predictive-based intervention still needs to be confirmed. An impact study needs to be conducted in a randomized control trial (RCT) setting to confirm whether being able to predict recovery and the resulting intervention could make a difference to the patient. Nevertheless, the current pilot study has now allowed us to recognize how recovery curves could be implemented and planned for use in stroke care. Estimates and variations from this study will serve to inform the design of the RCT study on how recovery curves and its software as a management tool could improve functional recovery.

Conclusion

Prognostic risk classification based on recovery curves can be clinically useful. This tool makes prognostic information available which could support the development of more tailored management and assist the implementation of more effective care models. Recovery curves tool could provide a useful framework for clinical and public health practice.