Post-Concussion Symptoms Rule: Derivation and Validation of a Clinical Decision Rule for Early Prediction of Persistent Symptoms after a Mild Traumatic Brain Injury

Abstract

Mild traumatic brain injury (mTBI) is a common problem. Depending on diagnostic criteria, 13 to 62% of those patients develop persistent post-concussion symptoms (PPCS). The main objective of this prospective multi-center study is to derive and validate a clinical decision rule (CDR) for the early prediction of PPCS. Patients aged ≥14 years were included if they presented to one of our seven participating emergency departments (EDs) within 24 h of an mTBI. Clinical data were collected in the ED, and symptom evolution was assessed at 7, 30 and 90 days post-injury using the Rivermead Post-Concussion Questionnaire (RPQ). The primary outcome was PPCS at 90 days after mTBI. A predictive model called the Post-Concussion Symptoms Rule (PoCS Rule) was developed using the methodological standards for CDR. Of the 1083 analyzed patients (471 and 612 for the derivation and validation cohorts, respectively), 15.6% had PPCS. The final model included the following factors assessed in the ED: age, sex, history of prior TBI or mental health disorder, headache in ED, cervical sprain and hemorrhage on computed tomography. The 7-day follow-up identified additional risk factors: headaches, sleep disturbance, fatigue, sensitivity to light, and RPQ ≥21. The PoCS Rule had a sensitivity of 91.4% and 89.6%, a specificity of 53.8% and 44.7% and a negative predictive value of 97.2% and 95.8% in the derivation and validation cohorts, respectively. The PoCS Rule will help emergency physicians quickly stratify the risk of PPCS in mTBI patients and better plan post-discharge resources.

Introduction

Mild traumatic brain injury (mTBI), commonly known as concussion, is a frequent problem. The Public Health Agency of Canada recently reported more than 5 million emergency department (ED) visits for head injuries between 2002 and 2017, 70% of which were due to concussions or mTBI.¹ This situation is similar elsewhere; in the United States, approximately 2 million people suffer a mTBI each year^2
-4 and worldwide, mTBI represents approximately 70-80% of all TBIs.⁵ Patients with mTBI often suffer from post-concussion symptoms such as headache, dizziness, and concentration problems, which can impede the return to normal function.⁶

Although post-concussion symptoms are usually transient, prevalence rates of persistent post-concussion symptoms (PPCS) have been reported in between 13 and 62% of patients, depending on diagnostic criteria.^{2,3,7

–21} Prolonged symptoms lead to time lost from work, social costs, and adverse impact on quality of life.^20,22 Early interventions, individualized neuropsychological interventions, and interdisciplinary interventions seem to reduce the frequency, intensity, and duration of PPCS^23
-25 and may facilitate the recovery of daily activities and return to work.^26

–29 Unfortunately, predicting long-term mTBI symptoms is challenging in the ED since emergency physicians’ diagnostic accuracy for predicting full-recovery or PPCS at 3 months post-injury is no better than chance.³⁰

Prior studies have identified some specific organic and psychogenic factors that seem to be associated with PPCS. The most frequently reported characteristics are sex (female),^{2,10,31

–41} prior TBI,^{3,10,31,32,35,42} history of psychiatric disorders,^{8,10,11,14
-16,34,40,43
-45} level of education,^10,32 comorbidities,^32,46,47 dizziness in ED,^15,48 headaches in ED,^{11,12,15,16,32} concomitant injuries,^3,49 alcohol intoxication,^16,40 hemorrhage on computed tomography (CT) scan,^{11,12,32,38,50,51} cervical sprain or strain,^39,52,53 and litigation.^19,54 Some symptoms have also been associated with PPCS during early follow-up³²: light sensitivity,^3,55 fatigue,^32,56 sleep disturbance,^8,57,58 severe symptoms on the Rivermead Post-Concussion Questionnaire (RPQ),^32,59 pain,^32,60 depressive symptoms,^59,61,62 anxious symptoms,⁴⁵ difficulty concentrating,⁵⁵ and abnormal eye movements.^63,64 It seems that patients who sustained sport-related mTBI may experience less post-concussion symptoms.⁶⁵ However, the association with some factors remains controversial with contradictory results in the literature, such as age,^{8,15,17,32

–36,38,45} loss of consciousness,^{2,14,15,18,29,33,42,61,66} post-traumatic amnesia^{2,33,42,45,55,59,61,67} and Glasgow Coma Scale <15 on arrival in ED.^29,32,39,40 Some biomarkers, combined with clinical factors, may be useful to predict the diagnosis and prognosis of mTBI. It has also been reported that S100ß, neuron-specific enolase (NSE), glial fibrillary acidic protein (GFAP), and cleaved-Tau (C-Tau) could be associated with brain damage.⁶⁸ However, their ability to predict PPCS is still unclear.⁶⁹ Finally, even if other biomarkers like diffusion tensor imaging⁷⁰ could be potentially predictive, they are expensive and difficult to measure among all mTBI patients in the ED.

To better identify patients who would most benefit from follow-up, some predictive models have been proposed,^{10,11,13,32,39,71
-73} but to our knowledge, none were externally prospectively validated and no evidence-based guidelines have been proposed. Consequently, there is a pressing need for ED clinicians to have access to a fast and reliable clinical decision rule (CDR) to predict PPCS in adults who sustained a mTBI.

The main objective of this study was to derive and validate a CDR, called the PoCS Rule (Post-Concussion Symptoms Rule), to predict PPCS 90 days after a mTBI. The secondary objectives were to validate the PoCS Rule for the prediction of PPCS, using other definitions and non-return to work/school at 90 days.

Methods

Study design and setting

This prospective multi-center cohort study was conducted across seven Canadian academic EDs (Level I and Level II trauma centers). Patients in the derivation cohort were recruited between July 2013 and October 2016, and those in the validation cohort were recruited between February 2017 and September 2018. This study was approved by each participating institution's research ethics board.

Study population

Patients were included if they had a documented mTBI within 24 h of ED visit and were age ≥14 years. We defined mTBI using the World Health Organization Task Force criteria⁷⁴: a patient with a head trauma and a Glasgow Coma Scale (GCS) score of 13-15 ≥ 30 min post-injury and at least one of the following: confusion or disorientation, loss of consciousness ≤30 min, post-traumatic amnesia ≤24 h, and/or other transient neurological abnormalities such as focal signs, seizure, and intracranial lesion not requiring surgery. Patients were excluded if they were hospitalized, were unable to consent, or were not able to communicate in English or French.

Data collection

Assessment in the ED

Eligible patients were identified by emergency physicians and research assistants at each participating site. During the initial ED visit, the emergency physicians used a standardized data collection form (Supplementary File S1) to collect relevant sociodemographic and clinical data, including the time of trauma. Informed consent was provided during the initial visit before the blood samples were taken. The blood samples were taken for all patients in our cohort for research purposes. The decision to perform a head CT scan was left to the emergency physician, and patients were included whether they had a CT or not.

Patient follow-up

Trained research assistants administered phone interviews at 7, 30, and 90 days post-mTBI (Supplementary File S2). During the 7-day interview, research assistants collected detailed information about the trauma and the medical history of patients to optimize data completeness and accuracy (Supplementary File S3). The Rivermead Post-Concussion Symptoms Questionnaire (RPQ), which is frequently used in the clinical and research settings,^{2,3,6,9,13,16,21,52,75,76} was used to evaluate the patient's symptoms at each interview. This simple validated questionnaire assesses the severity of PCS by having patients rate 16 symptoms on a scale of 0 (not experienced at all) to 4 (a severe problem), with the total score being the sum of all RPQ items.^77
-79

Blood sampling and analysis

The exact time of blood sample collection was recorded for each participant. These samples were taken by the ED nurse ≤24 h after the trauma in most of the study's recruiting sites (4/7). This was done only on weekdays by a research nurse for the other participating sites. The samples were sent to the laboratories of each hospital to be centrifuged, put into three small aliquots and frozen at -20°C until they were sent on dry ice to the study's central laboratory (CHU de Québec-Université Laval, Quebec City), where they were stored at -80°C until batch tested.

The blood concentrations of S100ß, NSE, GFAP, and C-Tau were analyzed by enzyme-linked immunosorbent assay (ELISA). For each biomarker, different cut-offs were tested during our statistical analyses, including those most often described in the literature: S100B ≥ 0.10 mcg/L⁸⁰ and ≥0.20 mcg/L,⁸¹ NSE ≥1.0 mcg/L,⁸² and C-Tau ≥1.5 mcg/L.⁸³ For GFAP, no cut-off point has been established related to PPCS,⁸⁴ we have therefore tested ≥0.045 mcg/L (detection limit of our ELISA assays) and ≥0.1 mcg/L (another limit of detection described in the literature).⁸⁵

Primary outcome measure

Until recently,⁸⁶ there was no consensus in the literature regarding the diagnostic criteria of PPCS.⁸⁷ Therefore, we used a nominal croup technique to define our primary outcome, PPCS at 90 days. This technique allows participants to give their opinions during a structured group interaction and comprises four key stages: silent generation, round-robin, clarification, and voting (ranking or rating).⁸⁸ In our study, PPCS was defined as the presence of spontaneously described, severe, persistent symptoms that have an impact on patient's life. Symptoms were considered spontaneously described if patients mentioned symptoms following a standardized interview prompt (“I would like to know whether you still have any symptoms from your head trauma?’’ The research assistant completed the RPQ as per standard questionnaire administration guidelines if patients answered positively. If patients responded negatively, their symptoms were not considered spontaneously described. Symptoms were considered severe if patients answered that they had at least one symptom of 4 points (severe problem) or two symptoms of 3 points (moderate problems), as scored on the RPQ. Symptoms were considered persistent if they lasted at least 90 days. To ensure that the symptoms were not newly present at 90 days, patients had to spontaneously answer that they had symptoms at 7 and 30 days.

As no specific measure of impact was available in the derivation cohort, symptoms were considered as having an impact if patients who were working/in school at the time of trauma did not return to work/studies as before the trauma. In the validation cohort, the measure of the impact on the patient's life was obtained by asking the following questions: “Did you return to your normal activities just as before the head trauma? If not, is this due to your trauma to the head?” A positive answer to the latter question was considered as having an impact on the patient's life.

Secondary outcome measures

Secondary outcome measures were PPCS at 90 days according to four other sets of criteria used in the literature:

Delphi definition: “Any post-concussion symptom appeared within hours of mTBI which is still present every day 90 days after the trauma and has an impact on at least one sphere of life.”⁸⁶

PPCS with a documented impact: impact on normal activities due to mTBI, according to the patient.

Non-return to work: patients working/in school at the time of trauma and did not return to work/studies as before trauma.

Moderate-to-severe symptoms: the presence of at least three symptoms of 3 or 4 points on the RPQ, indicating a moderate-to-severe intensity level.

Sample size

Derivation cohort

Based on the literature, the proportion of patients with PPCS at 90 days was estimated to be around 13-62%.^{2,3,7

–21} Assuming a middle range of 35%, 16 potential candidate predictors, a precision of 5% and at least 10 events per covariable, we estimated that 468 observations were required to develop the model.

Validation cohort

As the prevalence of PPCS at 90 days was 15.6% in our derivation cohort, for an accurate estimation of regression coefficients, assuming 10 events per covariable and a final model including eight predictors in the ED, we estimated that a minimum of 550 observations were required to validate the model. For the second step, five predictors at follow-up), assuming a prevalence of 18% of PPCS in that sub-group, a precision of 5% and at least 10 events per covariable, we estimated that a minimum of 275 observations were required.

Statistical analysis

We used the methodological standards^89,90 to develop and validate our CDR. Descriptive statistics were computed for baseline characteristics. The prevalence of PPCS has been reported as the proportion of participants corresponding to each definition. Univariate analyses were performed to measure the association between each potentially predictive factor and the primary outcome (odds ratio [OR] with 95% confidence intervals [CIs]). Multiple imputation was used to handle missing data for potential predictors in the ED: the chained equations method was used to simulate missing values, and 20 imputed datasets were generated for the following variables: cervical sprain, history of mental health disorders, prior multiple TBI and prior TBI <1 year.

A two-step bootstrap sampling with replacement was used to select variables of interest.⁹¹ For each step, 500 random samples were generated for each imputation and variables that were statistically significant in more than 40% of samples (alpha = 0.157) were considered relevant to be tested in the model. Backward selection was used to validate candidate predictors before using them in the final logistic regression model (alpha = 0.157). Selected predictors were screened for multi-collinearity. Coefficient rating was standardized using the smaller value of the coefficient.⁹² Thirty-day follow-up data were used if they were missing for patients who were not reached at the 7-day follow-up (simple imputation). Sensitivity analyses were performed to test different categories for the following variables: age, number of prescribed medications, headache intensity (out of 10), loss of consciousness duration, amnesia duration, RPQ, and different blood levels for all four biomarkers. Some symptoms on the RPQ at the 7-day follow-up were group tested using subtypes classification suggested by some authors^93,94: cognitive symptoms, oculomotor symptoms, headache/migraine, vestibular symptoms, and anxiety/mood symptoms.⁹⁴ Risk groups were defined using the most appropriate thresholds from the classification performance for the following two steps: the ED evaluation and the follow-up evaluation 7 days after the trauma. Model performance was evaluated by measuring sensitivity, specificity, negative predictive value (NPV), and positive predictive value (PPV). Discrimination was evaluated for each step and is presented using area under the curve (AUC). For each step of the model, a calibration curve of expected versus observed risk was assessed. Variance Inflation and Condition Index were used to evaluate the possibility of collinearity. All analyses were performed with the Statistical Analysis System (SAS Institute, Cary, NC, version 9.4).

Patient and public involvement

Patients were involved as research partners during our nominal group technique, particularly when discussing the outcome definition.

Results

We enrolled 1339 patients, 563 in the derivation cohort and 776 in the validation cohort. Of those, 1083 patients were included in our analyses: 471 and 612 in the derivation and validation cohorts, respectively (Fig. 1). We found a 15.6% prevalence of PPCS (15.1% and 15.9% in the derivation and validation cohorts, respectively). Table 1 shows the characteristics of the study population.

FIG. 1.

Study flowchart.

Table 1.

Characteristics of the Study Population: ED Evaluation

	Derivation cohort N = 471		Validation cohort N = 612
	n	(%)	n	(%)
Sociodemographic variables
Age, median (Q1-Q3)	37	(23-57)	43	(25-60)
14-24	142	(30.1)	151	(24.7)
25-34	81	(17.2)	95	(15.5)
35-44	49	(10.4)	73	(11.9)
45-54	67	(14.2)	89	(14.5)
55-64	76	(16.1)	81	(13.2)
65-74	39	(8.3)	77	(12.6)
75-84	15	(3.2)	32	(5.2)
85+	2	(0.4)	14	(2.3)
Sex (M)	306	(65.0)	346	(56.5)
Past medical/injury history
History of TBI^*	166	(35.4)	249	(40.7)
Prior TBI <1 year	23	(5.0)	36	(5.9)
Prior multiple TBI	49	(10.7)	71	(11.6)
Prior moderate/severe TBI	19	(4.5)	38	(6.4)
History of mental health disorder^†	131	(28.0)	126	(20.6)
At least 3 different prescribed medications^¶	104	(22.4)	136	(22.2)
Evaluation in ED
Mechanism of injury^&
Sport	109	(23.5)	101	(16.5)
Fall from their own height	97	(20.9)	114	(18.6)
Motor vehicle accident	76	(16.4)	87	(14.2)
Bicycle accident	73	(15.7)	76	(12.4)
Fall more than own height	34	(7.3)	46	(7.5)
Others	75	(16.2)	188	(30.7)
Documented loss of consciousness^#	228	(48.7)	289	(47.3)
Unknown duration	78	(38.5)	111	(18.2)
< 5 min	124	(26.5)	150	(24.6)
≥ 5 min	26	(5.6)	28	(4.6)
Unknown information	115	(24.6)	100	(16.4)
Documented post-traumatic amnesia^¥	238	(50.5)	225	(36.8)
< 30 min	189	(40.1)	167	(27.6)
≥ 30 min and <3 h	37	(7.9)	47	(7.8)
≥ 3 h	12	(2.5)	11	(1.8)
Post-traumatic confusion§	261	(57.9)	320	(53.2)
Documented headache in ED^‡	313	(66.5)	433	(70.8)
≤ 3	166	(35.2)	154	(25.2)
4-7	104	(22.1)	202	(33.0)
8-10	43	(9.1)	77	(12.6)
Glasgow Coma Scale score at ED arrival
15	409	(86.8)	545	(89.1)
14	56	(11.9)	61	(10.0)
13	6	(1.3)	6	(1.0)
Concomitant injuries	278	(60.2)	374	(61.1)
Cervical sprain	130	(31.9)	158	(25.8)
Hemorrhage on CT	31	(6.6)	46	(7.5)

Missing data for derivation cohort: history of TBI (n = 2), prior TBI <1 year (n = 7), prior multiple TBI (n = 13), prior moderate/severe TBI (n = 45). Validation cohort prior moderate/severe TBI (n = 21).

†

Any pre-injury mental health disorder, including depression, anxiety, bipolar disorder, schizophrenia, attention-deficit/hyperactivity disorder (ADHD), personality disorders, obsessive-compulsive disorder, post-traumatic stress disorder (PTSD), and other psychiatric disorders (missing data for derivation cohort: n = 3).

Missing data for derivation cohort (n = 7).

Missing data for derivation cohort (n = 3), validation cohort (n = 1). Unknown information means that the patient was unsure if they lost consciousness and there was no witness.

Missing data for validation cohort (n = 7).

Missing data for derivation cohort (n = 20), validation cohort (n = 11).

‡

Verbal rating scale, 0-10.

ED, emergency department; Q, quarter; M, male; TBI, traumatic brain injury; CT, computed tomography.

The potential predictive factors of PPCS in ED are found in Table 2. The following factors were statistically associated with PPCS: age, sex (female), history of TBI or mental health disorder, taking prescribed medication, having sustained a fall, headache in ED and cervical sprain. There was no association between PPCS and age when treated as a continuous variable. However, we found a U-shaped association between PPCS and age when treated as a categorical variable (Fig. 2). Alcohol consumption, level of education, loss of consciousness, post-traumatic amnesia or confusion, hemorrhage on CT and the four biomarkers tested were not significantly associated with PPCS. Wearing a helmet (for sports or activities where applicable) seems to decrease the risk of PPCS.

FIG. 2.

Association between age and persistent post-concussion symptoms (PPCS).

Table 2.

Univariate Correlation between Potential Predictive Factors in ED and PPCS

Potential predictive factors	Odds ratio	95% CI^§
Sociodemographic variables
Age (continuous)	1.01	1.00-1.02
Age (category)
14-24 or ≥65 (reference category)	1.00
25-34	2.38	1.10-5.13
35-44	4.11	1.82-9.27
45-54	3.00	1.38-6.55
55-64	2.57	1.19-5.56
Sex (F)	2.02	1.21-3.36
Level of education	1.13	0.56-2.27
Past medical/injury history
History of TBI	1.32	0.78-2.20
Prior TBI <1 year	3.20	1.30-7.86
Prior multiple TBI	2.19	1.09-4.37
History of mental health disorder	2.14	1.27-3.62
Alcohol consumption	0.68	0.20-2.32
Drug consumption	0.98	0.28-3.42
Prescribed medication (at least one)	3.11	1.70-5.68
Evaluation in the ED
Mechanism of injury
Sport	0.56	0.28-1.11
Fall	1.77	1.01-3.10
Motor vehicle accident	0.97	0.51-1.86
Helmet wearing (if applicable)	0.43	0.19-0.97
Loss of consciousness	0.88	0.47-1.63
Post-traumatic amnesia	0.91	0.55-1.52
Post-traumatic confusion	0.90	0.52-1.57
Vomiting after trauma	0.58	0.22-1.52
Headache in ED	2.08	1.13-3.81
GCS on arrival (13-14 vs. 15)	0.83	0.09-8.10
Suspected intoxication in ED	0.40	0.12-1.34
Other diagnosis
Any concomitant injury	1.66	0.96-2.88
Cervical Sprain	2.53	1.51-4.24
Multiple body injuries	1.36	0.73-2.51
Hemorrhage on CT ^α	1.71	0.71-4.14
Biomarkers^†
S100ß ≥ 0.02 mcg/L (≥ 20 pg/mL)^‡ ≥ 0.10 mcg/L (≥ 100 pg/mL) ≥ 0.20 mcg/L (≥ 200 pg/mL)	0.950.740.46	0.55-1.650.28-1.960.06-3.61
NSE ≥ 0.1 mcg/L (≥ 100 pg/mL) ≥ 0.2 mcg/L (≥ 200 pg/mL) ≥ 1.0 mcg/L (≥ 1000 pg/mL)^‡	1.381.100.94	0.82-2.330.62-1.940.20-4.28
GFAP ≥ 0.045 mcg/L (≥ 45 pg/mL) ≥ 0.1 mcg/L (≥ 100 pg/mL)^‡	0.640.69	0.31-1.310.34-1.37
C-Tau ≥ 0.156 mcg/L (≥ 156 pg/mL) ≥ 1.5 mcg/L (≥ 1500 pg/mL)^‡	0.640.90	0.32-1.280.52-1.57

αAny traumatic hemorrhage on CT.

†

S100ß, S100ß protein; NSE, neuron specific enolase; GFAP, glial fibrillary acidic protein; C-Tau, cleaved-Tau.

‡

The most documented threshold in the literature.

95% confidence interval.

ED, emergency department; PPCS, persistent post-concussion symptoms; CI, confidence interval; F, female; TBI, traumatic brain injury; CT, computed tomography.

In the derivation cohort, multiple imputation was applied to handle missing ED data in 13.4% of patients with cervical sprain, 0.6% of those with a history of mental health disorders, 2.8% of patients with prior multiple TBI, and 1.5% for prior TBI <1 year.

Table 3 shows the univariate correlations between symptoms at the 7-day follow-up and PPCS. All RPQ symptoms were significantly associated with PPCS except restlessness. A RPQ score ≥21 at 7 days had a 10.52 OR (95% CI: 5.49-20.14) to predict PPCS 90 days post-trauma. For the derivation and validation cohorts, data from the 30-day follow-up were used to replace the 7-day missing data in 13% and 8% of cases, respectively. In the validation cohort, the receiver operating characteristic (ROC) curve for the RPQ score versus the impact on the patient's activities had an AUC = 0.90.

Table 3.

Univariate Correlation between Symptoms at 7-Days Follow-Up and PPCS

Variable	Odds ratio	95% CI^§
Rivermead Post-concussion Questionnaire (RPQ) symptoms^α
Headaches	8.03	3.66-17.64
Feelings of dizziness	4.21	2.28-7.75
Nausea and/or vomiting	3.62	1.77-7.42
Noise sensitivity, easily upset by loud noise	3.83	2.10-6.99
Sleep disturbance	3.75	2.06-6.81
Fatigue, tiring more easily	6.48	2.69-15.65
Being irritable, easily angered	4.05	2.22-7.37
Feeling depressed or tearful	4.27	2.32-7.86
Feeling frustrated or impatient	3.89	2.14-7.08
Forgetfulness, poor memory	4.67	2.53-8.62
Poor concentration	5.54	2.91-10.55
Taking longer to think	4.28	2.32-7.89
Blurred vision	4.46	2.35-8.49
Light sensitivity, easily upset by bright light	6.35	3.42-11.82
Double vision	8.57	3.10-23.68
Restlessness	1.97	0.83-4.69
Rivermead score ≥21	10.52	5.49-20.14
Number of symptoms spontaneously identified by the patient^†	2.42	1.83-3.18
Number of symptoms (≥ 2 points) on the RPQ^‡	1.40	1.28-1.53
Number of symptoms (≥ 3 points) on RPQ^‡	1.45	1.32-1.60
Cervical pain	4.33	2.15-8.73
Limb pain	2.46	1.36-4.47
Cognitive symptoms^*	5.57	2.62-12.61

At least 2 points.

†

Before asking questions from the Rivermead Questionnaire.

‡

Rivermead Post-Concussion Symptoms Questionnaire.

Cognitive symptoms are defined by the following symptoms: forgetfulness/poor memory, poor concentration, taking longer to think.

95% Confidence interval.

PPCS, persistent post-concussion symptoms.

The PoCS Rule is shown in Figure 3, with full model specification in Supplementary File S4. During assessment in the ED, the total score helps the clinician better identify which mTBI patients are at higher risk of PPCS and may benefit from a follow-up. Otherwise, a phone follow-up should be done for medium-risk patients (2-6 points at ED evaluation) to identify another sub-group of high-risk patients.

FIG. 3.

Post-Concussion Symptoms Rule (PoCS Rule).

None of the biomarkers studied helped improve the final model, irrespective of the threshold value. Table 4 and Supplementary File S5 describe the proportions of patients with PPCS for all predictors and risk categories. According to the rule, only 2.7% of low-risk ED patients will experience PPCS compared with 30% of high-risk patients. Supplementary File S6 presents the classification performance of the PoCS Rule for the evaluation in the ED and for the 7-day follow-up. With an ED score threshold ≤1 for low risk and ≥7 for high risk, and a 7-day follow-up score ≥2 for high risk, the rule has a sensitivity of 91.4% and 89.6%, a specificity of 53.8% and 44.7% and a NPV of 97.2% and 95.8% in the derivation and validation cohorts, respectively (Table 5). Data are also displayed using a ≥3-point threshold. There was no multi-collinearity between predictors, and discrimination was good, with an AUC of 0.75 for ED evaluation and 0.85 for the 7-day follow-up. The calibration of the model was excellent (Fig. 4). Based on our validation cohort, the unnecessary follow-up rate with and without the PoCS Rule would be 66.3% and 84.3%, respectively (absolute value: 84.3-66.3 = 18% reduction). We estimate that using our rule would have reduced follow-ups by 21% (relative value: (516-406)/516 = 21%) compared with systematic follow-up post-ED visit.

FIG. 4.

Calibration plots for derivation and validation cohorts.

Table 4.

Prediction of PPCS According to the PoCS Rule Risk Categories (Validation Cohort)

	Number of points	n with PPCS (%)
Total number of patients with PPCS		96 (15.9)
Step 1: Assessment in ED (n = 612)
ED low risk (n = 113)	0-1	3 (2.7)
ED medium risk (n = 466)	2-6	83 (17.8)
ED high risk (n = 33)	≥ 7	10 (30.3)
Step 2‡: Follow-up evaluation for ED medium risk patients (n = 456)^†
Follow-up low risk (n = 123)	0-1	7 (5.7)
Follow-up high risk (n = 333)	≥ 2	76 (22.8)
PoCS Rule final prediction
Advised to consult a healthcare provider PRN (n = 236)		10 (4.2)
Follow-up/interventions (n = 366)		86 (23.5)
Step 2^§: Follow-up evaluation for ED medium risk patients (n = 456)^†
Follow-up low risk (n = 184)	0-2	10 (5.4)
Follow-up high risk (n = 272)	≥ 3	73 (26.8)
PoCS Rule final prediction
Advised to consult a healthcare provider PRN (n = 297)		13 (4.4)
Follow-up/interventions (n = 305)		83 (27.2)

‡

Using threshold ≥2 points 7 days after trauma.

†

For 10 medium risk patients, data were missing for at least one predictor at follow-up evaluation.

Using threshold ≥3 points 7 days after trauma.

PPCS, persistent post-concussion symptoms; PoCS, Post-Concussion Symptoms Rule; ED, emerge0ncy department.

Table 5.

PoCS Rule Performance for Prediction of PPCS (90 Days)

	Derivation cohort (n = 471)		Validation cohort (n = 602)^†
Post-concussion symptoms, n (%)	71	(15.1)	96	(15.9)
Using threshold ≥2 points 7 days after trauma	%	95% CI^§	%	95% CI^§
Sensitivity	91.4	(84.9-98.0)	89.6	(83.5-95.7)
Specificity	53.8	(48.7-58.9)	44.7	(40.3-49.0)
Negative predictive value	97.2	(94.9-99.4)	95.8	(93.2-98.3)
Positive predictive value	26.6	(21.0-32.1)	23.5	(19.2-27.8)
Using threshold ≥3 points 7 days after trauma
Sensitivity	87.1	(79.3-95.0)	86.5	(79.6-93.3)
Specificity	66.7	(61.9-71.5)	56.1	(51.8-60.5)
Negative predictive value	96.6	(94.4-98.8)	95.6	(93.3-97.9)
Positive predictive value	32.4	(25.7-39.1)	27.2	(22.2-32.2)

95% Confidence interval.

†

For 10 medium risk patients, data were missing for at least one predictor at follow-up evaluation.

PoCS, Post-Concussion Symptoms Rule; PPCS, persistent post-concussion symptoms; CI, confidence interval.

Table 6 shows the PoCS Rule performance for all secondary outcomes. The rule has an excellent sensitivity (97.3%) and NPV (99.1%) for predicting moderate to severe symptoms.

TABLE 6.

PoCS Rule Performance for Prediction of Secondary Outcomes (90 Days), Validation Cohort

	Delphi definition outcome ^β (n = 602)		Impact on normal activities^† (n = 599)		Non-return to work/school^‡ (n = 302)		Moderate/severe symptoms^§ (n = 599)
PPCS, n (%)	113	(18.8)	58	(9.7)	58	(19.2)	74	(12.4)
Using threshold ≥2 points 7 days after trauma	%	95% CI §	%	95% CI §	%	95% CI §	%	95% CI §
Sensitivity^α	88.5	(82.6-94.4)	91.4	(84.2-98.6)	84.5	(75.2-93.8)	97.3	(93.6-100.0)
Specificity^α	45.6	(41.2-50.0)	42.7	(38.5-46.9)	34.4	(28.5-40.4)	44.4	(40.1-48.6)
Negative Predictive Value^α	94.5	(91.6-97.4)	97.9	(96.0-99.7)	90.3	(84.3-96.3)	99.1	(98.0-100.0)
Positive Predictive Value^α	27.3	(22.8-31.9)	14.6	(11.0-18.2)	23.4	(17.7-29.2)	19.8	(15.7-23.9)
Using threshold ≥3 points 7 days after trauma
Sensitivity^α	85.8	(79.4-92.3)	91.4	(84.2-98.6)	82.8	(73.0-92.5)	95.9	(91.5-100.0)
Specificity^α	57.5	(53.5-61.8)	54.0	(49.8-58.2)	42.6	(36.4-48.8)	55.8	(51.6-60.1)
Negative predictive value^α	94.6	(92.0-97.2)	98.3	(96.9-99.8)	91.2	(86.0-96.4)	99.0	(97.8-100.0)
Positive predictive value^α	31.8	(26.6-37.0)	17.5	(13.3-21.8)	25.5	(19.3-31.8)	23.4	(18.7-28.2)

>α

With 95% confidence interval.

As per Lagacé-Legendre and colleagues.⁸⁶ J Head Trauma Rehabil (2021).

†

PPCS with an impact on normal activities due to head trauma, according to patient.

‡

For those who were worker or student at the time of the mTBI.

3 symptoms of 3-4 points using the Rivermead Post-concussion Questionnaire.

POCS, Post-Concussion Symptoms Rule; PPCS, persistent post-concussion symptoms.

Discussion

We developed an ED-friendly CDR for the early stratification of PPCS after an mTBI. The lack of multivariable prognostic models for patients with mTBI has been highlighted in a 2015 systematic review, which concluded that none of the 26 included studies found a multivariable prognostic model able to predict individual mTBI patient outcomes adequately.⁷² Indeed, most frameworks were based on small cohorts with inadequate sample size or low event per predictor ratio,^63,95,96 were only internally validated³² or the model calibration was not reported or was poor.

Since then, additional models have been proposed,^{3,10,11,39,73} but to our knowledge, none of them have been externally validated. Two studies had only conducted internal bootstrap validation.^11,39 Moreover, some of them used retrospective data,³ had a high rate of loss to follow-up¹⁰ or recruited very small cohorts.⁷³ Otherwise, Lingsma and colleagues tried to validate some existing models using the Transforming Research and Clinical Knowledge in Traumatic Brain Injury Pilot (TRACK-TBI) dataset,⁷¹ but they concluded that the models performed unsatisfactorily. More recently, Mikolic and colleagues tried to externally validate three predictive models¹³ using the Collaborative European NeuroTrauma Effectiveness Research in Traumatic Brain Injury (CENTER-TBI) Study dataset. Still, the authors concluded that none of these models has good calibration and discrimination for early prediction of PPCS.¹³

Although the diagnostic criteria for mTBI and PPCS were different in these studies, some common factors were found to be strongly associated with PPCS. The most frequently reported predictors were sex,^{2,10,31

–41} prior TBI,^{3,10,31,32,35,42} history of mental health disorder,^{8,10,11,14
-16,34,40,43
-45} headache in ED,^{11,12,15,16,32} and patient-reported symptoms 1-2 weeks after trauma,^{3,8,32,55

–59} and selected variables in the PoCS Rule are concordant with these known risk factors. Moreover, the PoCS Rule includes two well-known concussion-associated conditions: sleep disturbance^8,57,58 and cervical sprain.^39,52,53 Even if the level of education was not associated with PPCS in our univariate analyses, it was tested based on suggestions in the literature, but not kept in the final model, as it did not demonstrate significance after backwards regression analyses. Headache^{11,12,15,16,32} in the ED was tested using a 10-point severity scale, but this had no added value compared with using headache as a dichotomized variable. Despite testing different thresholds, the four biomarkers sampled did not improve the model. This is consistent with other recent publications.^68,97,98

Our study has limitations. First, we were unable to recruit all consecutive patients. However, four sites we able to perform 24/7 patient recruitment. Even if some patients (19%) were lost to follow-up or had missing outcome data, we were able to compare the characteristics of patients included in our analysis⁹⁹ to those lost to follow-up (Supplementary Files S7–S10). Therefore, we believe that our results can be generalized to non-hospitalized ED patients presenting during the acute phase after a mTBI. With regard to the outcome measurement tool, the RPQ has been criticized, but it is widely used in clinical settings and was found to be as useful as an objective neuropsychological assessment to help target patients who would most benefit from further intervention following their trauma.^52,100 It is an easy tool to use in the context of limited resources.

As the sample size was initially estimated using a higher prevalence than that ultimately observed in our sample, the final model should include no more than seven variables. Nevertheless, the event per predictor ratio was adequate (12:1 and 16:1 for step 1 and step 2, respectively) for the validation cohort, and the performance was still good when the rule was applied to this prospective cohort. No impact measure was explicitly obtained in the derivation cohort. Still, a specific impact measure was available for the validation cohort, and this measure was strongly associated with the RPQ score at 90 days (AUC = 0.90).

Because Canada has public health care coverage, litigation was not considered. The type of insurance coverage would have been challenging to collect and assess. This would have defeated the purpose of creating an easy-to-use rule that would focus on clinical and biomarker data. Even though coverage may differ in some provinces/countries, the most common type of compensable mTBI would be motor vehicle accidents. This type of trauma mechanism concerned 16.1% and 14.2% of our derivation and validation cohorts, respectively. As shown in Table 2, we found no correlation between this injury mechanism and PPCS (OR: 0.97, 95% CI: 0.51-1.86) and therefore, even though this may be considered a limit to our study, we believe this had little impact on our results. Some authors did find an association between litigation and the number of PPCS¹⁹ and psychological distress.⁵⁴ However, even though this factor could play a role in the recovery of some patients, several authors agree that the presence of persistent symptoms is likely the result of a complex interaction between neuropathological, psychological, and social factors.⁵² Secondary monetary gain would be a rare phenomenon and would not be the only reason for which a person could amplify or invent symptoms.¹⁰¹ Generally, most patients who sought compensation were not seeking /receiving compensation by 3 months.¹⁰²

Finally, we also used information from the 7-day interview as a proxy for the Delphi definition of PPCS.⁸⁶ This may have slightly impacted our results as we cannot confirm that the symptoms appeared within hours of the trauma.

As recommended, we used multiple imputation and bootstrap models selection to handle missing data among ED predictors to add stability to the prognostic models.⁹¹ We also performed a simple imputation using 30-day follow-up data for patients who were not reached at the 7-day follow-up (13% and 8% in the derivation and validation cohorts, respectively). Performing imputation with data collected closer to the outcome in time could artificially increase the strength of the predictors. However, this strategy is representative of standard follow-up practice, with new telephone attempts when the patient has not been contacted. Moreover, since data were missing for all variables when the patients could not be reached, no predictor benefited from this method.

Age was analyzed in categories, reflecting the inverted “U curve” of its association with PPCS. In contrast, most studies that found no association between those factors consider age as a continuous variable.^15,33,34,45 Our analyses ensure a better adjustment for this variable, and our results are consistent with that of others who found that older age may even be slightly protective.^11,103

Because the literature on PPCS diagnostic criteria is heterogeneous, the PoCS Rule was validated with different outcome measure definitions, including non-return to work.³⁵ If patients at risk of PPCS are identified early, some rehabilitation interventions to control “negative perceptions of mTBI”¹⁰⁴ and illness perceptions^61,105 have been described as beneficial. Further, some data support interventions like cervical⁵³ and vestibular rehabilitation and multidisciplinary care.¹⁰⁶ The scientific community has been attempting to better understand the bio-psycho-social factors that influence the prognosis of patients who sustained an mTBI. The PoCS Rule takes these concepts into account.^107,108

In summary, our simple CDR may improve our health system by achieving three main goals: 1) the early identification of patients at higher risk of PPCS (≥ 90 days), who may benefit from tailored post-ED interventions; 2) the early identification of patients at low risk of PPCS, who may not need clinical follow-ups; and 3) the catalyzation of research and clinical efforts to enhance the quality of care and prevent complications. The PoCS Rule will be clinically useful in helping emergency physicians quickly stratify the risk of PPCS in mTBI patients and better plan post-discharge resources, which may result in a better use of human and financial resources, while improving the quality of care.

Footnotes

Acknowledgments

We would like to thank all the study participants. We also thank all the research coordinators, research assistants and nurses who worked on this project and the emergency physicians of each participating site* for their help in filling out the data collection forms. We also would like to thank our study partners and collaborators who participated in the Nominal Group Technique Consensus process.

*CHU de Québec-Université Laval (Hôpital de l'Enfant-Jésus and CHUL), CISSS de Chaudière-Appalaches (CHAU Hôtel-Dieu de Lévis), CIUSS du Nord-de-l'Île-de-Montréal (Hôpital du Sacré-Cœur-de-Montréal), The Ottawa Hospital, Foothills Medical Center and Sunnybrook Health Sciences Center.

Authors' Contributions

Le Sage designed the study. Chauny, Berthelot, Archambault, Lee, McRae, Lang, Le Sage, and Perry were site investigators. Neveu conducted the statistical analysis under the supervision of Le Sage. Frenette supervised the laboratory manipulations and testing of the samples. Le Sage drafted the manuscript. Chauny, Berthelot, Archambault, Moore, Boucher, Frenette, De Guise, Ouellet, Lee, McRae, Lang, Émond, Mercier, Tardif, Swaine, Cameron, and Perry critically reviewed and approved the manuscript.

Funding Information

This study was supported by two grants from the Canadian Institute of Health Research (#127055 and # 366162) and by a grant from the Fondation du CHU de Québec-Université Laval (#3062)

Author Disclosure Statement

No competing financial interests exist.

Supplementary Material

Supplementary File S1

Supplementary File S2

Supplementary File S3

Supplementary File S4

Supplementary File S5

Supplementary File S6

Supplementary File S7

Supplementary File S8

Supplementary File S9

Supplementary File S10

References

Public health Agency of Canada. Injury in Review: Spotlight on Traumatic Brain Injuries Across the Life Course. 2020. Available from: https://www.canada.ca/en/public-health/services/injury-prevention/canadian-hospitals-injury-reporting-prevention-program/injury-reports/2020-spotlight-traumatic-brain-injuries-life-course.html [Last accessed: 01/01/2022]

Bazarian

, Wong

, Harris

, et al. Epidemiology and predictors of post-concussive syndrome after minor head injury in an emergency population. Brain Inj, 1999; 13(3):173–189; doi: 10.1080/026990599121692.

Wojcik

SM.

Predicting mild traumatic brain injury patients at risk of persistent symptoms in the Emergency Department. Brain Inj, 2014; 28(4):422–430; doi: 10.3109/02699052.2014.884241

Fung

, Willer

, Moreland

, et al. A proposal for an evidenced-based emergency department discharge form for mild traumatic brain injury. Brain Inj, 2006; 20(9):889–894; doi: 10.1080/02699050600831934

Gozt

, Licari

, Halstrom

, et al. Towards the development of an integrative, evidence-based suite of indicators for the prediction of outcome following mild traumatic brain injury: results from a pilot study. Brain Sci, 2020; 10(1); doi: 10.3390/brainsci10010023.

Cassidy

, Cancelliere

, Carroll

, et al. Systematic review of self-reported prognosis in adults after mild traumatic brain injury: results of the International Collaboration on Mild Traumatic Brain Injury Prognosis. Arch Phys Med Rehabil, 2014; 95(3 Suppl):S132–S151; doi: 10.1016/j.apmr.2013.08.299.

Belanger

, Barwick

, Kip

, et al. Postconcussive symptom complaints and potentially malleable positive predictors. Clin Neuropsychol, 2013; 27(3):343–355; doi: 10.1080/13854046.2013.774438.

Langer

, Alavinia

, Lawrence

, et al. Prediction of risk of prolonged post-concussion symptoms: Derivation and validation of the TRICORDRR (Toronto Rehabilitation Institute Concussion Outcome Determination and Rehab Recommendations) score. PLoS Med. 18 United States2021. p. e1003652.

Barker-Collo

, Theadom

, Jones

, et al. Reliable individual change in post concussive symptoms in the year following mild traumatic brain injury: data from the longitudinal, population-based brain injury incidence and outcomes New Zealand in the community (Bionic) study. JSM Burns Trauma, 2016; 1(1):1006.

10.

Cnossen

, Winkler

, Yue

, et al. Development of a prediction model for post-concussive symptoms following mild traumatic brain injury: a TRACK-TBI Pilot Study. J Neurotrauma, 2017; 34(16):2396–2409; doi: 10.1089/neu.2016.4819.

11.

Falk

, Bechtold

, Peters

, et al. A prognostic model for predicting one-month outcomes among emergency department patients with mild traumatic brain injury and a presenting Glasgow Coma Scale of fifteen. J Neurotrauma, 2021; 38(19):2714–2722; doi: 10.1089/neu.2021.0137.

12.

Heidari

, Asadollahi

, Jamshidian

, et al. Prediction of neuropsychological outcome after mild traumatic brain injury using clinical parameters, serum S100B protein and findings on computed tomography. Brain Inj, 2015; 29(1):33–40; doi: 10.3109/02699052.2014.948068.

13.

Mikolic

, Polinder

, Steyerberg

, et al. Prediction of global functional outcome and post-concussive symptoms after mild traumatic brain injury: external validation of prognostic models in the Collaborative European NeuroTrauma Effectiveness Research in Traumatic Brain Injury (CENTER-TBI) Study. J Neurotrauma, 2021; 38(2):196–209; doi: 10.1089/neu.2020.7074.

14.

Ponsford

, Nguyen

, Downing

, et al. Factors associated with persistent post-concussion symptoms following mild traumatic brain injury in adults. J Rehabil Med, 2019; 51(1):32–39; doi: 10.2340/16501977-2492.

15.

Savola

, Hillbom

Early predictors of post-concussion symptoms in patients with mild head injury. Eur J Neurol 2003 Mar;10(2):175–181; doi: 10.1046/j.1468-1331.2003.00552.x.

16.

Varner

, Thompson

, de Wit

, et al. Predictors of persistent concussion symptoms in adults with acute mild traumatic brain injury presenting to the emergency department. CJEM 2021 May;23(3):365–373; doi: 10.1007/s43678-020-00076-6.

17.

Waljas

, Iverson

, Lange

, et al. A prospective biopsychosocial study of the persistent post-concussion symptoms following mild traumatic brain injury. J Neurotrauma, 2015; 32(8):534–547; doi: 10.1089/neu.2014.3339

18.

Sterr

, Herron

, Hayward

, et al. Are mild head injuries as mild as we think? Neurobehavioral concomitants of chronic post-concussion syndrome. BMC Neurol 2006;6:7; doi: 10.1186/1471-2377-6-7.

19.

Tator

, Davis

, Dufort

, et al. Postconcussion syndrome: demographics and predictors in 221 patients. J Neurosurg, 2016; 125(5):1206–1216; doi: 10.3171/2015.6.JNS15664

20.

Hiploylee

, Dufort

, Davis

, et al. Longitudinal study of postconcussion syndrome: not everyone recovers. J Neurotrauma, 2017; 34(8):1511–1523; doi: 10.1089/neu.2016.4677

21.

Ingebrigtsen

, Waterloo

, Marup-Jensen

, et al. Quantification of post-concussion symptoms 3 months after minor head injury in 100 consecutive patients. J Neurol, 1998; 245(9):609-12; doi: 10.1007/s004150050254.

22.

National Center for Injury Prevention and Control. Report to Congress on Mild Traumatic Brain Injury in the United States: Steps to Prevent a Serious Public Health Problem. Atlanta, GA: Centers for Disease for Disease Control and Prevention; 2003. Available from: https://www.cdc.gov/traumaticbraininjury/pdf/mtbireport-a.pdf [Last accessed: 7/6/2022].

23.

King

, Coates

Mixed messages from the 'Mild Traumatic Brain Injury' and 'Sport-related Concussion' literatures: Clinical implications. Brain Inj, 2021; 35(4):501–503; doi: 10.1080/02699052.2021.1890216.

24.

Polinder

, Cnossen

, Real

RGL

, et al. A multidimensional approach to post-concussion symptoms in mild traumatic brain injury. Front Neurol 2018;9:1113; doi: 10.3389/fneur.2018.01113.

25.

Ponsford

, Willmott

, Rothwell

, et al. Impact of early intervention on outcome following mild head injury in adults. J Neurol Neurosurg Psychiatry, 2002; 73(3):330–332; doi: 10.1136/jnnp.73.3.330.

26.

Cassidy

, Carroll

, Peloso

, et al. Incidence, risk factors and prevention of mild traumatic brain injury: results of the WHO Collaborating Centre Task Force on Mild Traumatic Brain Injury. J Rehabil Med 2004(43 Suppl):28–60; doi: 10.1080/16501960410023732.

27.

Carroll

, Cassidy

, Peloso

, et al. Systematic search and review procedures: results of the WHO Collaborating Centre Task Force on Mild Traumatic Brain Injury. J Rehabil Med 2004(43 Suppl):11–14; doi: 10.1080/16501960410023660.

28.

Peloso

, Carroll

, Cassidy

, et al. Critical evaluation of the existing guidelines on mild traumatic brain injury. J Rehabil Med, 2004; 36(43 Suppl):106-12; doi: 10.1080/16501960410023868.

29.

Ruffolo

, Friedland

, Dawson

, et al. Mild traumatic brain injury from motor vehicle accidents: factors associated with return to work. Arch Phys Med Rehabil, 1999; 80(4):392–398; doi: 10.1016/s0003-9993(99)90275-7.

30.

Korley

, Peacock

, Eckner

, et al. Clinical gestalt for early prediction of delayed functional and symptomatic recovery from mild traumatic brain injury is inadequate. Acad emerg med, 2019; 26(12):1384–1387; doi: 10.1111/acem.13844.

31.

Edna

, Cappelen

Late post-concussional symptoms in traumatic head injury. An analysis of frequency and risk factors. Acta Neurochir (Wien), 1987; 86(1-2):12–17; doi: 10.1007/BF01419498.

32.

Stulemeijer

, van der Werf

, Borm

, et al. Early prediction of favourable recovery 6 months after mild traumatic brain injury. J Neurol Neurosurg Psychiatry, 2008; 79(8):936–942; doi: 10.1136/jnnp.2007.131250.

33.

Bazarian

, Atabaki

Predicting postconcussion syndrome after minor traumatic brain injury. Acad Emerg Med, 2001; 8(8):788–795; doi: 10.1111/j.1553-2712.2001.tb00208.x.

34.

Meares

, Shores

, Taylor

, et al. Mild traumatic brain injury does not predict acute postconcussion syndrome. J Neurol Neurosurg Psychiatry, 2008; 79(3):300–306; doi: 10.1136/jnnp.2007.126565.

35.

Dumke

HA.

Posttraumatic Headache and Its Impact on Return to Work After Mild Traumatic Brain Injury. J Head Trauma Rehabil, 2017; 32(2):E55–E65; doi: 10.1097/HTR.0000000000000244.

36.

King

NS.

A systematic review of age and gender factors in prolonged post-concussion symptoms after mild head injury. Brain Inj, 2014; 28(13-14):1639–1645; doi: 10.3109/02699052.2014.954271.

37.

Gupte

, Brooks

, Vukas

, et al. Sex differences in traumatic brain injury: what we Know and what we should know. J Neurotrauma, 2019; 36(22):3063–3091; doi: 10.1089/neu.2018.6171.

38.

Lannsjo

, Backheden

, Johansson

, et al. Does head CT scan pathology predict outcome after mild traumatic brain injury? Eur J Neurol 2013;20(1):124-129; doi: 10.1111/j.1468-1331.2012.03813.x.

39.

Cnossen

, van der Naalt

, Spikman

, et al. Prediction of persistent post-concussion symptoms after mild traumatic brain injury. J Neurotrauma, 2018; 35(22):2691–2698; doi: 10.1089/neu.2017.5486.

40.

Booker

, Sinha

, Choudhari

, et al. Description of the predictors of persistent post-concussion symptoms and disability after mild traumatic brain injury: the SHEFBIT cohort. Br J Neurosurg, 2019; 33(4):367–375; doi: 10.1080/02688697.2019.1598542.

41.

Meares

, Shores

, Taylor

, et al. The prospective course of postconcussion syndrome: the role of mild traumatic brain injury. Neuropsychology, 2011; 25(4):454–465 doi: 10.1037/a0022580.

42.

Meehan

, 3rd, O'Brien

, Geminiani

, et al. Initial symptom burden predicts duration of symptoms after concussion. J Sci Med Sport, 2016; 19(9):722–725; doi: 10.1016/j.jsams.2015.12.002.

43.

Ponsford

, Willmott

, Rothwell

, et al. Factors influencing outcome following mild traumatic brain injury in adults. J Int Neuropsychol Soc, 2000; 6(5):568–579; doi: 10.1017/s1355617700655066.

44.

McLean

, Kirsch

, Tan-Schriner

, et al. Health status, not head injury, predicts concussion symptoms after minor injury. Am J Emerg Med, 2009; 27(2):182–190; doi: 10.1016/j.ajem.2008.01.054.

45.

Ponsford

, Cameron

, Fitzgerald

, et al. Predictors of postconcussive symptoms 3 months after mild traumatic brain injury. Neuropsychology, 2012; 26(3):304–313; doi: 10.1037/a0027888.

46.

Vikane

, Hellstrom

, Roe

, et al. Predictors for return to work in subjects with mild traumatic brain injury. Behav Neurol 2016;2016:8026414; doi: 10.1155/2016/8026414.

47.

Scott

, Uomoto

, Barry

. Impact of pre-existing migraine and other co-morbid or co-occurring conditions on presentation and clinical course following deployment-related concussion. Headache 2020;60(3):526–541; doi: 10.1111/head.13709.

48.

Stalnacke

, Bjornstig

, Karlsson

, et al. One-year follow-up of mild traumatic brain injury: post-concussion symptoms, disabilities and life satisfaction in relation to serum levels of S-100B and neurone-specific enolase in acute phase. J Rehabil Med, 2005; 37(5):300–305; doi: 10.1080/16501970510032910.

49.

Ouellet

, Boucher

, Beauchamp

, et al. Influence of concomitant injuries on post-concussion symptoms after a mild traumatic brain injury—a prospective multicentre cohort study. Brain Inj, 2021; 35(9):1028–1034; doi: 10.1080/02699052.2021.1945145.

50.

Iverson

GL.

Complicated vs uncomplicated mild traumatic brain injury: acute neuropsychological outcome. Brain Inj, 2006; 20(13-14):1335–1344; doi: 10.1080/02699050601082156.

51.

Voormolen

, Zeldovich

, Haagsma

, et al. Outcomes after complicated and uncomplicated mild traumatic brain injury at three-and six-months post-injury: results from the CENTER-TBI Study. J Clin Med, 2020; 9(5); doi: 10.3390/jcm9051525.

52.

Ontario Neurotrauma Foundation. Guideline for Concussion/Mild Traumatic Brain Injury & Prolonged Symptoms. 2018. Available from: https://braininjuryguidelines.org/concussion/fileadmin/pdf/Concussion_guideline_3rd_edition_final.pdf [Last accessed: 2022/01/01].

53.

Kennedy

, Quinn

, Tumilty

, et al. Clinical characteristics and outcomes of treatment of the cervical spine in patients with persistent post-concussion symptoms: a retrospective analysis. Musculoskelet Sci Pract 2017;29:91–98; doi: 10.1016/j.msksp.2017.03.002.

54.

Feinstein

, Ouchterlony

, Somerville

, et al. The effects of litigation on symptom expression: a prospective study following mild traumatic brain injury. Med Sci Law, 2001; 41(2):116–121; doi: 10.1177/002580240104100206.

55.

Zuckerman

, Yengo-Kahn

, Buckley

, et al. Predictors of postconcussion syndrome in collegiate student-athletes. Neurosurg Focus, 2016; 40(4):E13; doi: 10.3171/2016.1.FOCUS15593.

56.

Norrie

, Heitger

, Leathem

, et al. Mild traumatic brain injury and fatigue: a prospective longitudinal study. Brain Inj, 2010; 24(13-14):1528–1538; doi: 10.3109/02699052.2010.531687.

57.

Chan

, Feinstein

Persistent Sleep Disturbances Independently Predict Poorer Functional and Social Outcomes 1 Year After Mild Traumatic Brain Injury. J Head Trauma Rehabil, 2015; 30(6):E67–E75; doi: 10.1097/HTR.0000000000000119.

58.

Sullivan

, Berndt

, Edmed

, et al. Poor sleep predicts subacute postconcussion symptoms following mild traumatic brain injury. Appl Neuropsychol Adult, 2016; 23(6):426–435; doi: 10.1080/23279095.2016.1172229.

59.

King

NS.

Emotional, neuropsychological, and organic factors: their use in the prediction of persisting postconcussion symptoms after moderate and mild head injuries. J Neurol Neurosurg Psychiatry, 1996; 61(1):75–81; doi: 10.1136/jnnp.61.1.75.

60.

Sheedy

, Geffen

, Donnelly

, et al. Emergency department assessment of mild traumatic brain injury and prediction of post-concussion symptoms at one month post injury. J Clin Exp Neuropsychol, 2006; 28(5):755–772; doi: 10.1080/13803390591000864.

61.

Whittaker

, Kemp

, House

Illness perceptions and outcome in mild head injury: a longitudinal study. J Neurol Neurosurg Psychiatry, 2007; 78(6):644–646; doi: 10.1136/jnnp.2006.101105.

62.

McCauley

, Boake

, Levin

, et al. Postconcussional disorder following mild to moderate traumatic brain injury: anxiety, depression, and social support as risk factors and comorbidities. J clin experimental neuropsychology, 2001; 23(6):792–808; doi: 10.1076/jcen.23.6.792.1016.

63.

Heitger

, Jones

, Anderson

. A new approach to predicting postconcussion syndrome after mild traumatic brain injury based upon eye movement function. Annu Int Conf IEEE Eng Med Biol Soc 2008;2008:3570–3573; doi: 10.1109/IEMBS.2008.4649977

64.

Whitney

, Sparto

. Eye movements, dizziness, and mild traumatic brain injury (mTBI): a topical review of emerging evidence and screening measures. J Neurol Phys Ther 2019;43(Suppl 2):S31–S6; doi: 10.1097/NPT.0000000000000272.

65.

Beauchamp

, Boucher

, Neveu

, et al. Post-concussion symptoms in sports-related mild traumatic brain injury compared to non-sports-related mild traumatic brain injury. CJEM, 2021; 23(2):223–231; doi: 10.1007/s43678-020-00060-0.

66.

Roy

, Peters

, Everett

, et al. Loss of consciousness and altered mental state as predictors of functional recovery within 6 months following mild traumatic brain injury. J Neuropsychiatry Clin Neurosci, 2020; 32(2):132–138; doi: 10.1176/appi.neuropsych.18120379.

67.

King

, Crawford

, Wenden

, et al. Early prediction of persisting post-concussion symptoms following mild and moderate head injuries. Br J Clin Psychol, 1999; 38(1):15–25; doi: 10.1348/014466599162638.

68.

Kim

, Tsao

, Stanfill

. The current state of biomarkers of mild traumatic brain injury. JCI Insight 2018;3(1):e97105; doi: 10.1172/jci.insight.97105

69.

Papa

, Edwards

, Ramia

Exploring Serum Biomarkers for Mild Traumatic Brain Injury. In: Brain Neurotrauma: Molecular, Neuropsychological, and Rehabilitation Aspects. Frontiers in Neuroengineering (Kobeissy FH, ed). Press/Taylor & Francis: Boca Raton, FL; 2015.

70.

Yuh

, Cooper

, Mukherjee

, et al. Diffusion tensor imaging for outcome prediction in mild traumatic brain injury: a TRACK-TBI study. J Neurotrauma, 2014; 31(17):1457–1477; doi: 10.1089/neu.2013.3171.

71.

Lingsma

, Yue

, Maas

, et al. Outcome prediction after mild and complicated mild traumatic brain injury: external validation of existing models and identification of new predictors using the TRACK-TBI pilot study. J Neurotrauma, 2015; 32(2):83–94; doi: 10.1089/neu.2014.3384.

72.

Silverberg

, Gardner

, Brubacher

, et al. Systematic review of multivariable prognostic models for mild traumatic brain injury. J Neurotrauma, 2015; 32(8):517–526; doi: 10.1089/neu.2014.3600.

73.

Caplain

, Blancho

, Marque

, et al. Early detection of poor outcome after mild traumatic brain injury: predictive factors using a multidimensional approach a pilot study. Front Neurol 2017;8:666; doi: 10.3389/fneur.2017.00666.

74.

Holm

, Cassidy

, Carroll

, et al. Summary of the WHO Collaborating Centre for Neurotrauma Task Force on Mild Traumatic Brain Injury. J Rehabil Med, 2005; 37(3):137–141; doi: 10.1080/16501970510027321.

75.

King

, Crawford

, Wenden

, et al. The Rivermead Post Concussion Symptoms Questionnaire: a measure of symptoms commonly experienced after head injury and its reliability. J Neurol, 1995; 242(9):587–592; doi: 10.1007/BF00868811.

76.

Carroll

, Cassidy

, Cancelliere

, et al. Systematic review of the prognosis after mild traumatic brain injury in adults: cognitive, psychiatric, and mortality outcomes: results of the International Collaboration on Mild Traumatic Brain Injury Prognosis. Arch Phys Med Rehabil, 2014; 95(3 Suppl):S152–S173; doi: 10.1016/j.apmr.2013.08.300.

77.

McMahon

, Hricik

, Yue

, et al. Symptomatology and functional outcome in mild traumatic brain injury: results from the prospective TRACK-TBI study. J Neurotrauma, 2014; 31(1):26–33; doi: 10.1089/neu.2013.2984.

78.

Lannsjö

, Borg

, Björklund

, et al. Internal construct validity of the Rivermead Post-Concussion Symptoms Questionnaire. J Rehabil Med, 2011; 43(11):997–1002; doi: 10.2340/16501977-0875.

79.

Balalla

, Krägeloh

, Medvedev

, et al. Is the Rivermead Post-Concussion Symptoms Questionnaire a reliable and valid measure to assess long-term symptoms in traumatic brain injury and orthopedic injury patients? A novel investigation using Rasch analysis. Neurotrauma Rep, 2020; 1(1):63–72; doi: 10.1089/neur.2020.0017

80.

Eskesen

, Springborg

, Unden

, et al. [Guidelines for the initial management of adult patients with minimal to moderate head injury]. Ugeskr Laeger, 2014; 176(9):V09130559;

81.

Ingebrigtsen

, Romner

, Marup-Jensen

, et al. The clinical value of serum S-100 protein measurements in minor head injury: a Scandinavian multicentre study. Brain Inj, 2000; 14(12):1047–1055; doi: 10.1080/02699050050203540.

82.

Herrmann

, Curio

, Jost

, et al. Release of biochemical markers of damage to neuronal and glial brain tissue is associated with short and long term neuropsychological outcome after traumatic brain injury. J Neurol Neurosurg Psychiatry, 2001; 70(1):95–100; doi: 10.1136/jnnp.70.1.95.

83.

, Lindsell

, Rosenberry

, et al. Serum cleaved tau does not predict postconcussion syndrome after mild traumatic brain injury. Am J Emerg Med2008;26(7):763–768; doi: 10.1016/j.ajem.2007.10.029.

84.

Seidenfaden

, Kjerulff

, Juul

, et al. Diagnostic accuracy of prehospital serum S100B and GFAP in patients with mild traumatic brain injury: a prospective observational multicenter cohort study—“the PreTBI I study”. Scand J Trauma Resusc Emerg Med, 2021; 29(1):75; doi: 10.1186/s13049-021-00891-5.

85.

Diaz-Arrastia

, Wang

, Papa

, et al. Acute biomarkers of traumatic brain injury: relationship between plasma levels of ubiquitin C-terminal hydrolase-L1 and glial fibrillary acidic protein. J Neurotrauma, 2014; 31(1):19–25; doi: 10.1089/neu.2013.3040.

86.

Lagace-Legendre

, Boucher

, Robert

, et al. Persistent postconcussion symptoms: an expert consensus-based definition using the Delphi method. J Head Trauma Rehabil, 2021; 36(2):96–102; doi: 10.1097/HTR.0000000000000613.

87.

Voormolen

, Cnossen

, Polinder

, et al. Divergent Classification methods of post-concussion syndrome after mild traumatic brain injury: prevalence rates, risk factors, and functional outcome. J Neurotrauma, 2018; 35(11):1233–1241; doi: 10.1089/neu.2017.5257.

88.

McMillan

, King

, Tully

. How to use the nominal group and Delphi techniques. Int J Clin Pharm 2016;38(3):655–662; doi: 10.1007/s11096-016-0257-x.

89.

Retel Helmrich

IRA

, Lingsma

, Turgeon

, et al. Prognostic research in traumatic brain injury: markers, modeling, and methodological principles. J Neurotrauma, 2021; 38(18):2502–2513; doi: 10.1089/neu.2019.6708.

90.

Collins

, Reitsma

, Altman

, et al. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. BMJ 2015;350:g7594; doi: 10.1136/bmj.g7594.

91.

Vergouw

, Heymans

, Peat

, et al. The search for stable prognostic models in multiple imputed data sets. BMC Med Res Methodol 2010 Sep 17;10:81; doi: 10.1186/1471-2288-10-81.

92.

Sullivan

, Massaro

, D'Agostino

, Sr. Presentation of multivariate data for clinical use: the Framingham Study risk score functions. Stat Med 2004;23(10):1631–1660; doi: 10.1002/sim.1742.

93.

Maruta

, Lumba-Brown

, Ghajar

Concussion subtype identification with the Rivermead Post-concussion Symptoms Questionnaire. Front Neurol 2018;9:1034; doi: 10.3389/fneur.2018.01034.

94.

Lumba-Brown

, Teramoto

, Bloom

, et al. Concussion guidelines step 2: evidence for subtype classification. Neurosurgery 2020 Jan 1;86(1):2–13; doi: 10.1093/neuros/nyz332.

95.

Faux

, Sheedy

, Delaney

, et al. Emergency department prediction of post-concussive syndrome following mild traumatic brain injury—an international cross-validation study. Brain Inj, 2011; 25(1):14–22; doi: 10.3109/02699052.2010.531686.

96.

Sheedy

, Harvey

, Faux

, et al. Emergency department assessment of mild traumatic brain injury and the prediction of postconcussive symptoms: a 3-month prospective study. J Head Trauma Rehabil, 2009; 24(5):333–343; doi: 10.1097/HTR.0b013e3181aea51f.

97.

Mercier

, Tardif

, Cameron

, et al. Prognostic value of neuron-specific enolase (NSE) for prediction of post-concussion symptoms following a mild traumatic brain injury: a systematic review. Brain Inj, 2018; 32(1):29–40; doi: 10.1080/02699052.2017.1385097.

98.

Mercier

, Tardif

, Cameron

, et al. Prognostic value of S-100beta protein for prediction of post-concussion symptoms after a mild traumatic brain injury: systematic review and meta-analysis. J Neurotrauma, 2018; 35(4):609–622; doi: 10.1089/neu.2017.5013.

99.

Kristman

, Manno

, Cote

Loss to follow-up in cohort studies: how much is too much? Eur J Epidemiol 2004;19(8):751–760; doi: 10.1023/b:ejep.0000036568.02655.f8.

100.

de Guise

, Belanger

, Tinawi

, et al. Usefulness of the Rivermead Postconcussion Symptoms questionnaire and the trail-making test for outcome prediction in patients with mild traumatic brain injury. Appl Neuropsychol Adult, 2016; 23(3):213–222; doi: 10.1080/23279095.2015.1038747.

101.

Mayer

, Quinn

, Master

. The spectrum of mild traumatic brain injury: a review. Neurology 2017;89(6):623-32; doi: 10.1212/WNL.0000000000004214.

102.

Reynolds

, Paniak

, Toller-Lobe

, et al. A longitudinal study of compensation-seeking and return to work in a treated mild traumatic brain injury sample. J Head Trauma Rehabil, 2003; 18(2):139-47; doi: 10.1097/00001199-200303000-00005.

103.

Richey

, Rao

, Roy

, et al. Age differences in outcome after mild traumatic brain injury: results from the HeadSMART study. Int Rev Psychiatry, 2020; 32(1):22–30; doi: 10.1080/09540261.2019.1657076.

104.

Hou

, Moss-Morris

, Peveler

, et al. When a minor head injury results in enduring symptoms: a prospective investigation of risk factors for postconcussional syndrome after mild traumatic brain injury. J Neurol Neurosurg Psychiatry, 2012; 83(2):217–223; doi: 10.1136/jnnp-2011-300767.

105.

Silverberg

, Hallam

, Rose

, et al. Cognitive-behavioral prevention of postconcussion syndrome in at-risk patients: a pilot randomized controlled trial. J Head Trauma Rehabil, 2013; 28(4):313–322; doi: 10.1097/HTR.0b013e3182915cb5.

106.

Schneider

, Meeuwisse

, Nettel-Aguirre

, et al. Cervicovestibular rehabilitation in sport-related concussion: a randomised controlled trial. Br J Sports Med. 2014 Sep;48(17):1294–1298; doi: 10.1136/bjsports-2013-093267.

107.

Erichsen

JE.

Injuries of the nervous system: on railway and other injuries of the nervous system. 1866. Clin Orthop Relat Res 2007;458:47–51; doi: 10.1097/BLO.0b013e31803df099

108.

Erichsen

JE.

On railway and other injuries of the nervous system. Walton and Moberly: London, U.K.; p. 1866.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

1.09 MB

0.44 MB

0.98 MB

0.34 MB

0.08 MB

0.11 MB

0.04 MB

0.44 MB