Abstract
Dear Editor:
With great interest we read the recent study by Walker and colleagues on the development of a prognostic model to predict long-term functional outcomes for adult patients with moderate and severe traumatic brain injury (TBI). 1 The authors used a large prospective multi-center cohort of patients with TBI receiving inpatient rehabilitation, representative for clinical practice in the United States. Prognostic modeling for outcome after TBI in the rehabilitation setting could help set expectations and plan treatments in those patients who are in inpatient rehabilitation after sustaining a TBI. We noted, however, several methodological shortcomings that necessitate a cautious interpretation of findings from this study.
First, the authors seem to have excluded or removed patients with the outcomes death or vegetative state from the analysis, arguing that including these would not have added much significant information. Obviously, leaving out these patients introduces some bias toward better outcome. Moreover, it is unknown in advance which patients will die or remain vegetative, and hence use of the model in clinical practice is impossible.
Second, the authors performed a complete case analysis by removing all patients with missing Glasgow Outcome Scale scores or a missing covariate from the analysis. Systematic differences between patients with missing data and patients with complete data could cause bias. A solution for this problem that is now widely implemented in clinical research is a multiple imputation procedure, where missing values are substituted with plausible values based on correlations with covariates and with outcome variables. 2
Third, the authors claim that a decision tree model is the best method to define a prognostic model in this context. Thorough methodological research has shown quite suboptimal performance of decision trees for modeling prognosis in TBI and other medical domains, however. 3,4 Studies comparing different modeling strategies concluded that logistic regression analysis is the preferred method to develop a prognostic model for outcomes of TBI. 3 A key prognostic characteristic such as age is then dealt with in a natural, continuous way rather than creating artificial groupings.
Fourth, the authors state that they demonstrated a reasonable predictive accuracy of the model. Indeed, a random split sample is an independent test for the model, but cannot be considered as external validation. To assess generalizability of the model, validation is required with meaningful geographic or temporal splitting. 5
Remarkably, the authors cite a systematic review that includes all the above mentioned recommendations for improvement of methodological quality in prognostic models in TBI. 6 Moreover, promising prognostic models for functional outcome after moderate and severe TBI have been developed over the last decade, including the International Mission for Prognosis and Analysis of Clinical Trials (IMPACT) and Corticoid Randomisation After Significant Head injury (CRASH) models. 7,8 Relevant admission characteristics included in these models, such as pupillary reactivity and extracranial injury, unfortunately were not incorporated in the current analyses.
In conclusion, we observe multiple methodological shortcomings in both development and validation of the proposed prognostic tool. In addition, important advances in prognostic modeling in TBI over the last decade should be considered. Application of the proposed model in patients with TBI in inpatient rehabilitation can only be recommended after satisfactory performance is shown in fully independent external validation studies with adequate design.