Machine Learning Web Application for Predicting Functional Outcomes in Patients With Traumatic Spinal Cord Injury Following Inpatient Rehabilitation

Abstract

Accurately predicting functional outcomes in patients with spinal cord injury (SCI) helps clinicians set realistic functional recovery goals and improve the home environment after discharge. The present study aimed to develop and validate machine learning (ML) models to predict functional outcomes in patients with SCI and deploy the models within a web application. The study included data from the Japan Association of Rehabilitation Database from January 1, 1991, to December 31, 2015. Patients with SCI who were admitted to an SCI center or transferred to a participating post-acute rehabilitation hospital after receiving acute treatment were enrolled in this database. The primary outcome was functional ambulation at discharge from the rehabilitation hospital. The secondary outcome was the total motor Functional Independence Measure (FIM) score at discharge. We used binary classification models to predict whether functional ambulation was achieved, as well as regression models to predict total motor FIM scores at discharge. In the training dataset (70% random sample) using demographic characteristics and neurological and functional status as predictors, we built prediction performance matrices of multiple ML models and selected the best one for each outcome. We validated each model's predictive performance in the test dataset (the remaining 30%). Among the 4181 patients, 3827 were included in the prediction model for the total motor FIM score. The mean (standard deviation [SD]) age was 50.4 (18.7) years, and 3211 (83.9%) patients were male. There were 3122 patients included in the prediction model for functional ambulation. The CatBoost Classifier and regressor models showed the best performances in the training dataset. On the test dataset, the CatBoost Classifier had an area under the receiver operating characteristic curve of 0.8572 and an accuracy of 0.7769 for predicting functional ambulation. Likewise, the CatBoost Regressor performed well, with an R² of 0.7859, a mean absolute error of 9.2957, and a root mean square error of 13.4846 for predicting the total motor FIM score. The final models were deployed in a web application to provide functional predictions. The application can be found at http://3.138.174.54:8501. In conclusion, our prediction models developed using ML successfully predicted functional outcomes in patients with SCI and were deployed in an open-access web application.

Introduction

Traumatic spinal cord injury (SCI) is a devastating neurologic condition with substantial socioeconomic effects on patients and their caregivers. Patients with SCI also experience a high rate of unemployment and a decreased quality of life.¹ Therefore, functional recovery is one of the main goals for patients with SCI and their families. Accurate prediction of functional outcomes can help clinicians and patients set realistic functional recovery goals and improve the home environment after discharge. Moreover, accurate prediction may help stratify patients in interventional trials.^2,3 Several studies have reported prognostic models for patients with SCI using traditional statistical methods such as linear or logistic regression using patient characteristics as predictors.^4

–9

Machine learning (ML) is a mathematical model approach that finds patterns within large amounts of sample data called “training data” and makes predictions. In certain cases, ML models outperform traditional statistical methods,¹⁰ given their ability to detect nonlinear relationships and interactions between variables. Although the use of ML models to generate clinical predictions is promising, one drawback is that clinicians cannot readily access and use these models, unlike simpler tools such as risk scores and nomograms. A well-performing and more powerful ML model may be less attractive if it cannot be practically accessed by healthcare professionals.

The present study aimed to develop and validate an ML model for predicting independent ambulation and the total motor Functional Independence Measure (FIM) score in patients with SCI utilizing data from patients enrolled in the Japan Association of Rehabilitation Database (JARD). We also aimed to provide healthcare professionals with an open-access web application that effectively communicates functional outcomes of SCI patients predicted by the ML models. In addition, we sought to identify factors associated with good functional outcomes using the ML approach.

Methods

Patients

The study was approved by the Institutional Review Board of Chiba University Graduate School of Medicine. The study included data from the JARD from January 1, 1991, to December 31, 2015. JARD enrollment included both patients with SCI who were transferred to a participating post-acute rehabilitation hospital after receiving acute treatment and patients admitted to a participating SCI center immediately after injury and subsequently underwent post-acute rehabilitation (Fig. 1). Thus, this database contains a mixture of acute and subacute patients with SCI. The database project itself was approved by the Institutional Review Board of JARD. The requirement for informed patient consent was waived, as retracing is not possible because the data are anonymized.

FIG. 1.

Three-quarters of patients received acute care at one hospital before being transferred to a rehabilitation hospital. In this case, the number of days from injury to admission was calculated as the length of the hospital stay during the acute phase. One quarter of patients were admitted to a spinal cord injury center immediately after the injury and went directly from acute treatment to post-acute rehabilitation. In this case, the number of days from injury to admission was counted as one.

Demographic data, including age, sex, occupation, educational background, marital status, and comorbidities (hypertension, cardiovascular disease, diabetes, chronic kidney disease, etc.) were collected. The following characteristics of the SCI injury were recorded: cause of injury; presence of vertebral fracture or dislocation; surgically treated or not; other associated injuries; received a blood transfusion or not; number of days from injury to admission; length of stay in rehabilitation hospital; neurological level of injury; and, at admission and discharge from the rehabilitation hospital, scores on each item in the FIM, the American Spinal Cord Injury Association (ASIA) Impairment Scale (AIS), and the ASIA motor and sensory assessments.¹¹ Surgical indications and timing of surgery were not standardized due to the multi-center nature of the study. We excluded patients with non-traumatic injuries, those who expired during hospitalization, and patients presenting with an AIS E. For the total motor FIM score prediction model, we further excluded patients with missing FIM motor scores at discharge. For the functional ambulation prediction model, we additionally excluded patients who had a FIM locomotion score of 6 or more, indicating their ability to walk independently at the time of admission.

Rehabilitation setting

Rehabilitation programs for patients with SCI focused on gait and exercises related to activities of daily living. The gym exercise program comprised 40-80 min of physical therapy 5-7 days per week. The program included a range of motion, muscle strengthening, and basic motion exercises (e.g., rolling over, standing up, and walking).

Outcome variables

The FIM score is an established index of disability severity that is widely used in rehabilitation settings.¹² The instrument comprises 18 items, each assessed on a 7-point ordinal scale; higher scores correlate to higher levels of independent activity. Motor FIM consists of 13 activity items (eating, grooming, bathing, upper body dressing, lower body dressing, toileting, bladder management, bowel management, bed-to-chair transfer, toilet transfer, tub/shower transfer, locomotion [walk/wheelchair], and stairs), with scores ranging from 13 (totally dependent) to 91 (totally independent). Cognitive FIM consists of five items (comprehension, expression, social interaction, problem solving, and memory).

The primary outcome in this study was functional ambulation at discharge from the rehabilitation hospital, evaluated by the FIM locomotion (walk/wheelchair) score. “Functional ambulation” was defined as a score of 6 (modified independence) or 7 (complete independence) where the mode of ambulation was either walking or walking and using a wheelchair equally. “Not functional ambulation” was defined as a score of 1 to 5 for any mode of locomotion.¹³ The secondary outcome was the total motor FIM score at discharge from the rehabilitation hospital.

Machine learning models

We used PyCaret version 2.3.10 (https://pycaret.org) to apply ML to our dataset. PyCaret is an open-source, low-code ML library in Python that automates ML workflows. All ML algorithms were implemented in Python 3.8 using Visual Studio Code version 1.77.3 (Microsoft Corporation, WA, USA).

The list of features input into the model is displayed in Table 1. Proportions for missing values for each feature are shown in Table 2 and any features that had more than 30% of the values missing were removed. Iterative imputation was used for missing values. LightGBM and Random Forest were set as the numeric iterative imputer and the categorical iterative imputer, respectively.

Table 1.

List of Features Input into the Machine Learning Model

Features
Patient background
Sex
Age
Occupation
Educational background
Marital status
Living with housemate or not
Driver's license status
Treated with ndustrial injury insurance or not
Hypertension
Cardiovascular disease
Stroke
Diabetes
Liver disease
Respiratory disease
Kidney disease

Characteristics of spinal cord injury
Cause of injury
Presence of vertebral fracture or dislocation
Surgically treated or not
Other associated injuries
Received a blood transfusion or not
Number of days from injury to admission
Presence of OPLL or OLF

Neurological status at admission
AIS
Neurological level of injury
ASIA motor score for each key muscle
Total ASIA motor score
ASIA sensory score (light touch)
ASIA sensory score (pin prick)
Total FIM score
Total motor FIM score
Scores for each of the 18 FIM items

OPLL, ossification of the posterior longitudinal ligament; OLF, ossification of the ligamentum flavum; AIS, American Spinal Cord Injury Association Impairment Scale; ASIA, American Spinal Cord Injury Association; FIM, Functional Independence Measure.

Table 2.

Baseline Characteristics at Admission of Patients (N = 3827) Who Were Included in the Prediction Model for the Total Motor FIM Score

Characteristics	Values	Missing data, n (%)
Sex (male), n (%)	3211 (83.9)	1 (0.0)
Age at the time of admission, mean (SD)	50.4 (18.7)	15 (0.4)
Number of days from injury to admission, mean (SD)	82.1 (348.8)	26 (0.7)
Admitted on the day of injury, n (%)	951 (24.8)
Severity of neurological deficit, n (%)		202 (5.3)
AIS A	1266 (33.1)
AIS B	386 (10.1)
AIS C	921 (24.0)
AIS D	1052 (27.5)
Neurological level of injury, n (%)		137 (3.6)
C1-C4	853 (22.3)
C5-T1	1795 (46.9)
T2-T7	260 (6.8)
T8-L1	548 (14.3)
L2-S5	234 (6.1)
ASIA motor score, mean (SD)	47.9 (27.8)	80 (2.1)
ASIA sensory score (light touch), mean (SD)	64.0 (33.2)	305 (8.0)
ASIA sensory score (pin prick), mean (SD)	60.3 (34.3)	315 (8.2)
Total FIM score, mean (SD)	63.0 (24.2)	49 (1.3)
Total motor FIM score, mean (SD)	30.5 (22.3)	58 (1.5)
Spinal surgery, n (%)	2524 (66.0)	93 (2.4)

AIS, American Spinal Cord Injury Association impairment scale; ASIA, American Spinal Cord Injury Association; C, cervical; FIM, Functional Independence Measure; L, lumbar; S, sacral; SD, standard deviation; T, thoracic.

Using the Boruta algorithm,¹⁴ the feature dimensions were reduced to eight features in the binary classification model and 12 features in the regression model. Boruta is a Python package designed to take the “all-relevant” approach to feature selection.

After preprocessing the dataset, we compared the ML models using the compare_models function in PyCaret. This function trains all the models in the model library with default hyperparameters and evaluates performance metrics using a 10-fold cross-validation in the training dataset. We used binary classification models to predict whether functional ambulation was achieved, as well as regression models to predict total motor FIM scores at discharge. The models used in this step are listed in Table 3 and Table 4 for functional ambulation and total motor FIM score, respectively. Then, we selected the best performing model from the training dataset and optimized the hyperparameters using the tune_model function in PyCaret.

Table 3.

Comparing the Performance of Binary Classification Models for Predicting the Achievement of Functional Ambulation

Model	Accuracy	AUC	Recall	Prec.	F1
CatBoost Classifier	0.7615	0.8407	0.8053	0.7763	0.7901
Gradient Boosting Classifier	0.7574	0.8358	0.8012	0.7731	0.7864
Random Forest Classifier	0.7592	0.8319	0.8102	0.7704	0.7894
Extra Trees Classifier	0.7524	0.8276	0.7979	0.7677	0.7820
Logistic Regression	0.7524	0.8273	0.7905	0.7731	0.7808
Ada Boost Classifier	0.7496	0.8251	0.8028	0.7623	0.7815
Linear Discriminant Analysis	0.7478	0.8246	0.7930	0.7658	0.7783
Light Gradient Boosting Machine	0.7478	0.8196	0.7864	0.7679	0.7764
Extreme Gradient Boosting	0.7423	0.8126	0.7774	0.7651	0.7706
Quadratic Discriminant Analysis	0.7378	0.8027	0.8102	0.7431	0.7749
Naïve Bayes	0.7350	0.7963	0.7881	0.7502	0.7680
K Neighbors Classifier	0.7304	0.7824	0.7799	0.7476	0.7631
Decision Tree Classifier	0.6842	0.6789	0.7256	0.7133	0.7185

AUC, area under the receiver operating characteristic curve; Prec., precision.

Table 4.

Comparing the Performance of Regression Models for Predicting the Total Motor FIM Score

Model	MAE	RMSE	R²
CatBoost Regressor	9.6077	13.8682	0.7635
Light Gradient Boosting Machine	9.4766	13.8700	0.7628
Random Forest Regressor	9.7890	14.0630	0.7566
Gradient Boosting Regressor	10.0200	14.0988	0.7558
Extra Trees Regressor	10.0354	14.2877	0.7495
Extreme Gradient Boosting	10.2312	14.7828	0.7302
K Neighbors Regressor	10.9222	15.6786	0.6980
Linear Regression	12.8517	16.6195	0.6610
Ridge Regression	12.8517	16.6194	0.6610
Least Angle Regression	12.8517	16.6195	0.6610
Bayesian Ridge	12.8735	16.6251	0.6608
Elastic Net	12.9474	16.6562	0.6597
Lasso Regression	12.9437	16.6585	0.6596
Huber Regressor	12.6746	17.0803	0.6404
AdaBoost Regressor	15.0951	17.5351	0.6227
Orthogonal Matching Pursuit	14.9305	19.0257	0.5567
Decision Tree Regressor	12.9298	20.1217	0.5035

FIM, functional independence measure; MAE, mean absolute error; RMSE, root mean square error.

Model explainability

Shapley additive explanation (SHAP) values were computed for the top-performing models to allow for model explainability. SHAP, which is characterized as a “game-theoretic technique for understanding the output of any ML model,” was used to assess the impact of each variable on the model.¹⁵

Model deployment to web application

The final algorithms for independent ambulation and total motor FIM score prediction were deployed within an open-access web application. Utilizing Streamlit (https://streamlit.io), an open-source app framework for ML and data science projects, the application was deployed on an Amazon Web Services (AWS) Elastic Compute Cloud (EC2) server.

Statistical analysis

The dataset was randomly split 7:3 into training and testing subsets. The training set was used for model training to solve binary classifications (n = 2185) and regression problems (n = 2678). Then, we trained each model using 10-fold cross-validation in the training set, and the model performance was assessed on the test set (n = 937 and n = 1149 for binary classifications and regression problems, respectively).

Accuracy, area under the curve (AUC), recall, precision, and F1 values were calculated to evaluate the binary classification models. R², mean absolute error (MAE), and root mean square error (RMSE) were used to compare performance of the regression models. All analyses were conducted using the PyCaret version 2.2.3 in Python 3.8.

Results

Baseline patient characteristics

Of the 4181 patients in JARD, 3827 patients met the study inclusion criteria for the total motor FIM score prediction model and 3122 patients were included for the functional ambulation prediction model. A flowchart of the patient selection process is presented in Figure 2. Table 2 shows baseline characteristics of the 3827 patients who were included in the prediction model for the total motor FIM score; of these, 1266 (33.1%) were AIS A, 386 (10.1%) AIS B, 921 (24.0%) AIS C, and 1052 (27.5%) AIS D at admission to inpatient rehabilitation. There were 951 (24.8%) patients who were admitted on the day of injury.

FIG. 2.

Flowchart of patient selection. AIS, American Spinal Cord Injury Association impairment scale; FIM, Functional Independence Measure.

Evaluation metrics

The mean ± SD FIM locomotion (walk/wheelchair) score was 2.4 ± 2.1 at admission and 5.0 ± 2.1 at discharge. The mean ± SD total motor FIM score was 30.5 ± 22.3 at admission and 58.4 ± 28.8 at discharge. The average ± SD length of stay was 168.2 ± 138.9 days.

Model performance

Table 3 shows the results of compare_models in PyCaret with a summary of the accuracy, AUC, recall, precision, and F1 score for each binary classification model predicting the achievement of functional ambulation. In the training dataset, the CatBoost Classifier showed the highest AUC (i.e., 0.8407) and accuracy of 0.7615 for predicting functional ambulation. On the test dataset, the tuned CatBoost Classifier had an AUC of 0.8572, accuracy of 0.7769, recall of 0.8529, precision of 0.7645, and F1 score of 0.8063. The receiver operating characteristic curve and confusion matrix of the CatBoost Classifier for predicting functional ambulation are shown in Figure 3A and 3B, respectively.

FIG. 3.

The performance matrix of the best model for predicting the achievement of functional ambulation. (A) ROC curves. (B) Confusion matrix of predictions for achieving functional ambulation. 1 = functional ambulation; 0 = not functional ambulation. AUC, area under the curve; ROC, receiver operating characteristic curve.

Table 4 shows the results of compare_models in PyCaret with a summary of the MAE, RSME, and R² of each regression model predicting the total motor FIM score. In the training dataset, the CatBoost Regressor exhibited the second lowest MAE (i.e., 9.4766) and the lowest RMSE (i.e., 13.8682) and the highest R² (i.e., 0.7635) for predicting the total motor FIM score. On the test dataset, the tuned CatBoost Regressor had an MAE of 9.2957, RMSE of 13.4846, and R² of 0.7859. The prediction error plot of the best model for predicting the total motor FIM score at discharge from the rehabilitation hospital is shown in Figure 4.

FIG. 4.

Prediction error plot of the best model for predicting the total motor FIM score. FIM, Functional Independence Measure.

Model explainability

ASIA motor score, age, total FIM score, number of days from injury to admission, and neurological level of injury had high SHAP values and were identified as factors that strongly influence the model's output. SHAP values in the functional ambulation and total motor FIM score models are shown in Figure 5A and 5B, respectively.

FIG. 5.

SHAP values for prediction models. (A) Impact of features on predicting functional ambulation. (B) Impact of features on predicting the total motor FIM score. Red and blue colors represent high and low levels of each predictor, respectively. The x-axis represents the SHAP value. A positive SHAP value indicates the patient is likely to acquire functional ambulation or a higher total motor FIM score; a negative value means they are unlikely to acquire functional ambulation or a high total motor FIM score. For example, age is a negative predictor since blue falls on the positive SHAP value. Occupation_12.0, within the model, denotes unemployment status where ‘1’ indicates unemployment, and ‘0’ indicates employment. Please note that students and homemakers were not considered in the unemployment classification. FIM, Functional Independence Measure; SHAP, Shapley additive explanations.

Model deployment to web application

The final prediction models are available as a web application at http://3.138.174.54:8501 (Fig. 6).

FIG. 6.

Web application interface for the two final algorithms. Left panel, input tab for multiple features; main frame, prediction results.

Discussion

In the present study, we showed that using a large multi-center dataset from JARD, ML models could successfully predict functional outcomes. Further, we found that functional outcomes predicted by the model are strongly influenced by certain factors, including ASIA motor score, age, total FIM score, number of days from injury to admission, and neurological level of injury. We deployed these ML models in an open-access web application.

Our ML models could stratify the prognosis of walking ability and predict total motor FIM score in patients with SCI based on neurological and functional status and demographic data at admission to inpatient rehabilitation. Existing studies on the application of ML to predict functional outcomes in SCI are limited. Four studies had sought to predict neurological or functional outcomes of patients with SCI using an ML approach. Among them, Inoue and colleagues analyzed data from 165 patients with SCI and applied a two-class discrimination model using XGBoost to predict AIS (A, B, C or D, E) 6 months after SCI with an accuracy of 81.1% and an AUC of 0.867.¹⁰ Facchinello and colleagues applied a regression tree algorithm for predicting the long-term functional outcome of 172 patients using the Spinal Cord Independence Measure following traumatic SCI.¹⁶ Regression tree models demonstrated R² values of 0.517 and 0.632 for the simplified and complete models, respectively.¹⁶ The prediction models from the first two studies may be less stable given the small sample size of the datasets. Belliveau and colleagues used an artificial neural network (ANN) and logistic regression to predict walking recovery following SCI. The AUC for determining who could walk 150 ft at 1 year after hospital discharge was 0.8801 and 0.8754 for ANN and Logistic Regression, respectively.¹⁷ DeVries and colleagues applied unsupervised ML models to predict functional ambulation one year after injury using FIM locomotion from a dataset of 862 SCI patients.¹³ The proposed unsupervised ML model demonstrated an AUC of 0.86, showing no differences compared to the earlier conventional models.

More recently, studies have combined novel imaging and ML approaches to predict functional outcomes after SCI. McCoy and colleagues demonstrated that injury volume of the spinal cord as observed in magnetic resonance images derived from a deep learning segmentation model significantly correlated with motor scores at admission and discharge.¹⁸ Okimatsu and colleagues reported that the combination of deep learning radiomics and random forest algorithms could distinguish between five grades of the ASIA impairment scale 1 month after injury with an accuracy of 0.715.¹⁹

Despite the complexity of SCI pathology and the variability in our patient cohort, we achieved a favorable AUC for predicting functional ambulation and a high R² for predicting the total motor FIM score. Regarding the prediction models for functional ambulation in patients with SCI, Van Middendorp and colleagues used a logistic regression model based on 492 patients, and they had an excellent AUC of 0.956.² A similar AUC (0.939) was achieved by external validation of Van Middendorp and colleagues' model on 184 patients by Van Silfhout and colleagues.²⁰ Hicks and colleagues reported a logistic regression model based on 278 patients that achieved an AUC of 0.889.²¹ Another logistic regression model by Phan and colleagues based on 675 patients achieved AUCs ranging from 0.516 to 0.730 for each AIS A to D at admission.²² The AUC of the present ML model for ambulation was slightly inferior to those previously reported.^2,20,21 This may be due to the heterogenous cohort in the present study, which includes both acute and subacute patients. Moreover, having a higher proportion of patients with AIS A or AIS D could inflate the model's predictive accuracy,²¹ as it is intuitive to understand that patients with AIS A experience unfavorable gait outcomes and patients with AIS D experience good gait outcomes. The proportion of patients with AIS A or AIS D at admission was 57% in our classification model, 71% in Van Middendorp and colleagues' model,² 68% in Hicks and colleagues' model,²¹ and 76% in Phan and colleagues' model.²²

Regarding prediction models for the total motor FIM score, Wilson and colleagues reported a linear regression using four predictors 1 year after injury with an R² value of 0.52.⁷ An R² value of 0.72 was reported by Abdul-Sattar for a linear regression between the motor FIM score and five predictors in the acute phase.⁶ Another linear regression model was described by Post and colleagues, reporting an R² value of 0.49 for model training.²³ The R² of 0.7859 in our predictive model for total motor FIM score was better than those previously reported.^6,7,23 This may be because ML can uncover relationships within complicated datasets that standard linear regression may overlook.

The model we built here identified ASIA motor score, age, total FIM score, number of days from injury to admission, and neurological level of injury at admission as important features for predicting functional outcomes in patients with SCI. These results are consistent with findings presented previously. Recent systematic reviews^24
-26 suggest that the initial severity (based on AIS) of a traumatic SCI was the main predictor of functional outcomes. The ASIA motor score also was a significant predictive factor of functional outcomes because it directly correlates with AIS grade.²⁷ However, the ASIA motor score does not reflect neurologic sacral evaluation, which is considered a crucial factor in neurological and functional recovery.²⁸ Age is also consistently found to affect long-term functional status.^24,25,29 The final functional score was correlated with baseline functional status at discharge from acute care or at admission to the rehabilitation hospital.²⁴ This is consistent with the final functional outcome correlating with the total FIM score on admission in the present study. A shorter stay in acute care hospitals was similarly associated with improved functional recovery after inpatient rehabilitation.^6,8 Shorter length of stay in acute care may also reflect underlying factors such as fewer complications, comorbidities, or associated injuries that could lead to functional recovery.²⁴

In addition, we identified the number of days from injury to hospitalization as an important feature that may be considered for use in algorithms stratifying patients into acute and subacute phases. Neurological level of injury was another important predictor of functional recovery, especially when patients were divided into groups based on quadriplegia and paraplegia,⁶ or when considering cases with the same severity of injury.^30
-32 While the trend toward early surgical management for SCI is recognized, there is uncertainty regarding the impact of the timing of surgical decompression following SCI,^4,8,33 and the timing of surgery was not recorded in the JARD.

This study created a modern ML model with the largest dataset to date of patients with SCI to predict functional outcomes, and this model was made publicly available in the form of a web application. An early and precise prediction of functional outcomes in patients using ML models can help healthcare professionals promote efficient care, optimize treatments, set realistic goals, and assess the effect of cutting-edge therapies in clinical trials.³⁴ Given these benefits, ML is expected to be an essential tool for making personalized medicine more common.

Limitations

There are several limitations in this study. First, the final motor FIM measurement was taken at discharge, and discharge timing varied between patients. Nonetheless, our study population, based in a country with low patient hospitalization costs, often stays hospitalized for extended rehabilitation and discharges around 6 months in average when neurologic and functional outcomes are known to plateau.³⁵ Second, the patients included in this study were a mixture of those in the acute and post-acute phases, making it difficult to perform a uniform initial evaluation at the time of admission. However, the number of days from injury to admission was included as a factor in our model, and the model could be applied to both acute and post-acute phases, which makes the model available for a larger population range. Third, by excluding deceased cases from the analysis, our model may not accurately predict the prognosis of cases that could potentially result in death. Fourth, some of the injury details were unavailable in the database, such as the timing of surgery and certain MRI characteristics of the spinal cord. Although these elements could have further enhanced the prediction accuracy of our ML model, this data was not available due to the nature of the retrospective study using a database.

Conclusions

To conclude, in the present study we showed that prediction models using ML could successfully predict functional outcomes, including ambulation and total motor FIM score. Further, our ML models identified ASIA motor score, age, total FIM score, number of days from injury to admission, and neurological level of injury as factors that strongly impact functional outcomes. We deployed the proposed ML models in an open-access web application with a user friendly interface for healthcare professionals.

Footnotes

Acknowledgments

We acknowledge the Japan Association of Rehabilitation Database for establishing the Japan Rehabilitation Database, which served as a core resource for this study.

Disclaimer: The views presented here are those of the authors and do not necessarily represent the views of the Japan Association of Rehabilitation Database. The registration data is not a representative sample of rehabilitation in Japan as well as rehabilitation in the web application user's country because most of the facilities that participated in the Japan Association of Rehabilitation Database were actively engaged in the rehabilitation of SCIs.

Authors' Contributions

Satoshi Maki had full access to all the data in the study and took responsibility for the integrity of the data and the accuracy of the data analysis.

Concept and design: Satoshi Maki.

Acquisition, analysis, or interpretation of data: All authors.

Drafting of the manuscript: Satoshi Maki.

Critical revision of the manuscript for important intellectual content: All authors. Statistical analysis: Satoshi Maki.

Obtained funding: Satoshi Maki.

Administrative, technical, or material support: All authors.

Supervision: Seiji Ohtori.

Funding Information

This work was supported by a research grant funded by the JOA-Subsidized Science Project Research 2020-1 and JSPS KAKENHI Grant Number JP20K18052.

Author Disclosure Statement

No competing financial interests exist.

References

Hiremath

S V

, Hogaboom

, Roscher

, et al. Longitudinal prediction of quality-of-life scores and locomotion in individuals with traumatic spinal cord injury. Arch Phys Med Rehabil, 2021; 98(12):2385–2392; doi: 10.1016/j.apmr.2017.05.020

Van Middendorp

, Hosman

, Donders

ART

, et al. A clinical prediction rule for ambulation outcomes after traumatic spinal cord injury: a longitudinal cohort study. Lancet, 2011; 377(9770):1004–1010; doi: 10.1016/S0140-6736(10)62276-3

Tanadini

, Hothorn

, Jones

LAT

, et al. Toward inclusive trial protocols in heterogeneous neurological disorders: prediction-based stratification of participants with incomplete cervical spinal cord injury. Neurorehabil Neural Repair, 2015; 29(9):867–877; doi: 10.1177/1545968315570322

Kaminski

, Cordemans

, Cernat

, et al. Functional outcome prediction after traumatic spinal cord injury based on acute clinical factors. J Neurotrauma, 2017; 34(12):2027–2033; doi: 10.1089/neu.2016.4955

Wilson

, Cadotte

, Fehlings

. Clinical predictors of neurological outcome, functional status, and survival after traumatic spinal cord injury: a systematic review. J Neurosurg Spine, 2012; 17(1 Suppl):11–26; doi: 10.3171/2012.4.aospine1245

Abdul-Sattar

. Predictors of functional outcome in patients with traumatic spinal cord injury after inpatient rehabilitation: in Saudi Arabia. NeuroRehabilitation, 2014; 35(2):341–347; doi: 10.3233/NRE-141111

Wilson

, Grossman

, Frankowski

, et al. A clinical prediction model for long-term functional outcome after traumatic spinal cord injury based on acute clinical and imaging factors. J Neurotrauma, 2012; 29(13):2263–2271; doi: 10.1089/neu.2012.2417

Denis

, Feldman

, Thompson

, et al. Prediction of functional recovery six months following traumatic spinal cord injury during acute care hospitalization. J Spinal Cord Med, 2018; 41(3):309–317; doi: 10.1080/10790268.2017.1279818

Zoerner

; Blanckenhorn

W U

; Dietz

, et al; EM-SCI Study Group; Armin Curt. Clinical algorithm for improved prediction of ambulation and patient stratification after incomplete spinal cord injury. J Neurotrauma, 2010; 27(1):241–252; doi: 10.1089/neu.2009.0901

10.

Inoue

, Ichikawa

, Ueno

, et al. XGBoost, a machine learning method, predicts neurological recovery in patients with cervical spinal cord injury. 2020; 1:8–16; doi: 10.1089/neur.2020.0009

11.

Maynard

, Bracken

, Creasey

, et al. International Standards for Neurological and Functional Classification of Spinal Cord Injury. American Spinal Injury Association. Spinal Cord, 1997; 35(5):266–274; doi: 10.1038/sj.sc.3100432

12.

Keith

, Granger C

, Hamilton

, et al. The functional independence measure: a new tool for rehabilitation. Adv Clin Rehabil, 1987; 1:6–18.

13.

DeVries

, Hoda

, Rivers

, et al. Development of an unsupervised machine learning algorithm for the prognostication of walking ability in spinal cord injury patients. Spine J, 2020; 20(2):213–224; doi: 10.1016/j.spinee.2019.09.007

14.

Kursa

, Rudnicki

. Feature selection with the boruta package. J Stat Softw, 2010; 36(11):1–13; doi: 10.18637/jss.v036.i11

15.

Lundberg

, Lee

S-I

. A Unified Approach to Interpreting Model Predictions. In: Advances in Neural Information Processing Systems. ( Guyon

, Luxburg

U V

, Bengio

, et al. eds). Curran Associates, Inc.; 2017.

16.

Facchinello

, Beauséjour

, Richard-Denis

, et al. Use of regression tree analysis for predicting the functional outcome after traumatic spinal cord injury. J Neurotrauma, 2021; 38(9):1285–1291; doi: 10.1089/neu.2017.5321

17.

Belliveau

, Jette

, Seetharama

, et al. Developing artificial neural network models to predict functioning one year after traumatic spinal cord injury. Arch Phys Med Rehabil, 2016; 97(10):1663–1668.e3; doi: 10.1016/j.apmr.2016.04.014

18.

McCoy

, Dupont

, Gros

, et al. Convolutional neural network–based automated segmentation of the spinal cord and contusion injury: deep learning biomarker correlates of motor impairment in acute spinal cord injury. Am J Neuroradiol, 2019; 40(4):737–744; doi: 10.3174/ajnr.A6020

19.

Okimatsu

, Maki

, Furuya

, et al. Determining the short-term neurological prognosis for acute cervical spinal cord injury using machine learning. J Clin Neurosci, 2022; 96:74–79; doi: 10.1016/j.jocn.2021.11.037

20.

Van Silfhout

, Peters

AEJ

, Graco

, et al. Validation of the Dutch clinical prediction rule for ambulation outcomes in an inpatient setting following traumatic spinal cord injury. Spinal Cord, 2016; 54(8):614–618; doi: 10.1038/sc.2015.201

21.

Hicks

, Zhao

, Fallah

, et al. A simplified clinical prediction rule for prognosticating independent walking after spinal cord injury: a prospective study from a Canadian multicenter spinal cord injury registry. Spine J, 2017; 17(10):1383–1392; doi: 10.1016/j.spinee.2017.05.031

22.

Phan

, Budhram

, Zhang

, et al. Highlighting discrepancies in walking prediction accuracy for patients with traumatic spinal cord injury: an evaluation of validated prediction models using a Canadian multicenter spinal cord injury registry. Spine J, 2019; 19(4):703–710; doi: 10.1016/j.spinee.2018.08.016

23.

Post

MWM

, Dallmeijer

, Angenot

ELD

, et al. Duration and functional outcome of spinal cord injury rehabilitation in the Netherlands. J Rehabil Res Dev, 2005; 42(3 Suppl 1):75–85; doi: 10.1682/jrrd.2004.10.0133

24.

Richard-Denis

, Beauséjour

, Thompson

, et al. Early predictors of global functional outcome after traumatic spinal cord injury: a systematic review. J Neurotrauma, 2018; 35(15):1705–1725; doi: 10.1089/neu.2017.5403

25.

Wilson

, Cadotte

, Fehlings

26.

AlHuthaifi

, Krzak

, Hanke

, et al. Predictors of functional outcomes in adults with traumatic spinal cord injury following inpatient rehabilitation: a systematic review. J Spinal Cord Med, 2017; 40(3):282–294; doi: 10.1080/10790268.2016.1238184

27.

Kirshblum

, Waring

, Biering-Sorensen

, et al. Reference for the 2011 revision of the International Standards for Neurological Classification of Spinal Cord Injury. J Spinal Cord Med, 2011; 34(6):547–554; doi: 10.1179/107902611X13186000420242

28.

Kirshblum

, Botticello

, Lammertse

, et al. The impact of sacral sensory sparing in motor complete spinal cord injury. Arch Phys Med Rehabil, 2011; 92(3):376–383; doi: 10.1016/j.apmr.2010.07.242

29.

Wilson

, Davis

, Kulkarni A

, et al. Defining age-related differences in outcome after traumatic spinal cord injury: analysis of a combined, multicenter dataset. Spine J, 2014; 14(7):1192–1198; doi: 10.1016/j.spinee.2013.08.005

30.

Horn

, Smout

, Dejong

, et al. Association of various comorbidity measures with spinal cord injury rehabilitation outcomes. Arch Phys Med Rehabil, 2013; 94(4 SUPPL.):S75–S86; doi: 10.1016/j.apmr.2012.10.036

31.

Ozelie

, Gassaway

, Buchman

, et al. Relationship of occupational therapy inpatient rehabilitation interventions and patient characteristics to outcomes following spinal cord injury: the SCIRehab Project. J Spinal Cord Med, 2012; 35(6):527–546; doi: 10.1179/2045772312Y.0000000062

32.

Teeter

, Gassaway

, Taylor

, et al. Relationship of physical therapy inpatient rehabilitation interventions and patient characteristics to outcomes following spinal cord injury: the SCIRehab project. J Spinal Cord Med, 2012; 35(6):503–526; doi: 10.1179/2045772312Y.0000000058

33.

Grassner

, Wutte

, Klein

, et al. Early decompression (< 8 h) after traumatic cervical spinal cord injury improves functional outcome as assessed by spinal cord independence measure after one year. J Neurotrauma, 2016; 33(18):1658–66; doi: 10.1089/neu.2015.4325

34.

Sharif

, Yassar

, Ali

. Outcome prediction in spinal cord injury: myth or reality. World Neurosurg, 2021; 140:574–590; doi: 10.1016/j.wneu.2020.05.043

35.

Ditunno

. The John Stanley Coulter Lecture. Predicting recovery after spinal cord injury: a rehabilitation imperative. Arch Phys Med Rehabil, 1999; 80(4):361–364; doi: 10.1016/s0003-9993(99)90270-8