Abstract
Background:
The Sport Concussion Assessment Tool–Third Edition (SCAT3) is currently considered the standard sideline assessment for concussions. In-game exercise, however, may affect SCAT3 performance and the diagnosis of concussions.
Purpose:
To examine the influence of exercise on SCAT3 performance in professional male athletes.
Study Design:
Controlled laboratory study.
Methods:
We examined the SCAT3 performance of 82 professional male athletes under 2 conditions: at rest and after exercise.
Results:
Athletes reported significantly fewer total symptoms (mean, 1.0 ± 1.5 vs 1.6 ± 2.3 total symptoms, respectively; P = .008; Cohen d = 0.34), committed significantly fewer errors on the modified Balance Error Scoring System (mean, 3.5 ± 3.5 vs 4.6 ± 4.1 errors, respectively; P = .017; d = 0.31), and required significantly less time to complete the tandem gait test (mean, 9.5 ± 1.4 vs 9.9 ± 1.7 seconds, respectively; P = .02; d = 0.30) during the at-rest condition compared with the postexercise condition.
Conclusion:
The interpretation of in-game (sideline) SCAT3 results should consider the effects of postexercise fatigue levels on an athlete’s performance, particularly if preseason baseline data have been collected when the athlete was well rested.
Clinical Relevance:
Exercise appears to affect symptom burden and physical abilities, such as balance and tandem gait, more so than the cognitive components of the SCAT3.
The Sport Concussion Assessment Tool–Third Edition (SCAT3) 5 was developed from the original SCAT and the SCAT2 15,16 and published following the 4th International Conference on Concussion in Sport held in Zurich in November 2012. 17 At the 2nd International Conference on Concussion in Sport in 2004, the Standardized Assessment of Concussion (SAC) was combined with a standardized symptom scale (Post-Concussion Symptom Scale [PCSS]), sport-specific orientation questions (ie, modified Maddocks questions), on-field markers of a concussion (eg, amnesia, loss of consciousness), and guidelines for systematic, stepwise return to play to create the first iteration of the SCAT. 15 Although the primary purpose of the SCAT was to standardize the sideline assessment of concussions, it also served as a tool for patient education. 15
The SCAT3 combines patient background information with tasks that target a spectrum of clinical signs and symptoms, cognitive dysfunction, and physical deficits that commonly result from a concussion, and it has been widely used to assess the acute effects of sport-related concussions on the sideline. 10 The recommended process for the sideline administration of the SCAT3 requires a 10-minute rest period for the athlete before commencing the assessment. 5 The premise for this recommendation is to mitigate the potential effects of postexercise fatigue on SCAT3 performance because exercise alone without the presence of a concussion may induce changes that make clinical interpretation difficult during sideline assessments. The purpose of this study was to examine the influence of exercise on SCAT3 performance in professional male athletes. We hypothesized that the athletes would endorse slightly more symptoms (eg, fatigue) and perform worse on the physical components of the SCAT3 (ie, modified Balance Error Scoring System [BESS] and tandem gait) after exercise. We also hypothesized that the cognition scores on the SCAT3 would not differ between conditions.
Methods
Participants
The participants of this study were composed of active male professional athletes, aged ≥18 years, from the Newcastle Knights rugby squad and the Newcastle Jets football (soccer) squad (N = 82). Those athletes with a recent history of concussions (within 3 months of the testing date), medical comorbidities (cardiac or respiratory disease), or significant injuries precluding participation in the exercise component of the study were excluded (n = 8). This study was approved by an institutional human ethics committee, and informed consent was provided by each player before participation.
Instruments
Sport Concussion Assessment Tool–Third Edition
The SCAT3 consists of 9 sections: (1) potential signs of a concussion covering the classic operational criteria of a traumatic brain injury (eg, loss of consciousness and amnesia); (2) the Glasgow Coma Scale; (3) sport-specific orientation and amnesia assessed via the Maddocks questions 13 ; (4) demographic variables, self-reported concussion history, potential outcome modifiers, and medications, collected in the background section; (5) the PCSS, which is a symptom checklist that evaluates 22 symptoms on a dimensional scale from 0 (none) to 6 (severe) and has been shown to be useful for the assessment of both symptom presence and severity 9 ; (6) the cognitive assessment, based on the SAC, 14 which consists of time orientation, immediate memory, concentration, and delayed recall components; (7) a neck examination; (8) postural stability, assessed with the modified BESS 21 and/or tandem gait; and (9) coordination, evaluated using the finger-to-nose test. For the purposes of this study, the following SCAT3 components were included in analyses: total number of symptoms endorsed, symptom severity, orientation, immediate memory, concentration, delayed recall, SAC total score, modified BESS, and tandem gait time (in seconds).
Age-Predicted Maximal Heart Rate Revisited
Participants were required to achieve a minimum of 75% of their age-predicted maximal heart rate on a stationary exercise bicycle. The age-predicted maximal heart rate revisited was calculated using a standardized cardiac monitor combined with manual palpation of participants’ pulse to monitor each participant’s heart rate while engaged in the exercise component of the study. Prior work has suggested that the maximal heart rate during a rugby league match is approximately 166 beats per minute. 6
Borg Rating of Perceived Exertion (RPE) Scale
The Borg RPE scale 2 was used to obtain qualitative information relating to participants’ perceived level of physical fatigue (ie, how strenuous the exercise was). This scale is a subjective (self-rated) measure of physical activity intensity level. The perceived exertion is based on the physical sensations that a participant experiences during physical activity, including increased heart rate, increased respiration rate, increased sweating, and muscle fatigue. Despite the subjective nature of the scale, a participant’s exertion rating is thought to be a good estimate of his actual heart rate during physical activity. The Borg RPE scale (Table 1) ranges from 6 (no exertion at all) to 20 (maximum exertion); as such, it is a “relative” scale. 3
Borg Rating of Perceived Exertion Scale
Procedure
Each participant completed the SCAT3 on 2 occasions, approximately 3 weeks apart, under a different condition on each occasion. A 3-week test-retest interval was used because of the limited availability of the professional athletes and to reduce possible practice effects. Prior work among adolescent and young adult athletes has suggested poor to moderate test-retest reliability for the SCAT3 components among noninjured controls when tested 7 days apart (intraclass correlation coefficient = 0.42-0.64). 4 A single male physician (J.H.L.) administered all tests during both testing sessions.
In condition 1, the SCAT3 was administered to the participants when they were rested. In condition 2, participants completed a 5-minute physical exertion protocol in which they achieved a minimum of 75% of their age-predicted maximal heart rate, subsequently rested for 5 minutes, and then completed the SCAT3. During National Rugby League game play, players can be removed from play, assessed, and returned if it is determined that they did not sustain a concussion (with no interchange charged to the team). However, this complete evaluation must take place within a 15-minute assessment window. Thus, the medical staff is encouraged to allow a 5-minute rest period before commencement of the SCAT3 to ensure that the remaining 10 minutes of the interchange window can be used for assessment. Therefore, we chose a 5-minute rest duration in our study to reflect this common clinical practice.
Each condition was randomly assigned to participants such that they were undertaken in a counterbalanced manner (ie, 50% of participants performed condition 1 in their first session and condition 2 during their second session, while 50% of the sample performed condition 2 in their first session and condition 1 in their second session) to control for order effects. In condition 2, participants used a stationary exercise bicycle to complete a cardiovascular workout. Participants were requested to exercise to 75% of their maximum heart rate and to maintain that level of intensity for a period of 5 minutes. The participants’ heart rate was monitored via a standardized cardiac monitor combined with manual palpation to record the time that it took for them to reach 75% of their maximum heart rate and to ensure that they maintained this level of intensity for 5 minutes. The cardiac monitor was used in conjunction with manual palpation of the heart rate to minimize any potential measurement error that may have resulted from the use of either method alone (ie, the cardiac monitor was used as a verification tool). The participants were required to achieve a heart rate of >75% maximum on both manual palpation and cardiac monitoring to satisfy the requirement of our study. Participants were then asked to rate the level of intensity of their workout using the Borg RPE scale. The 2 conditions were completed a mean 3.5 ± 2.6 weeks apart (median, 3.0 weeks; range, 1-12 weeks).
Statistical Analyses
Continuous variables are presented as means and SDs; categorical variables are presented as absolute values or percentages of the total. At-rest SCAT3 performance was compared with postexercise SCAT3 performance among all participants on each subtest using paired-samples t tests. To assess the effect of test order, SCAT3 component scores were compared between the group that completed the SCAT3 at rest (ie, condition 1) first with the group that completed the SCAT3 after the exercise protocol first (ie, condition 2) using independent-samples t tests. To classify the clinical meaningfulness of SCAT3 performance at rest and after exercise, we calculated the frequency of participants in each condition who were classified as broadly normal, below/above average, unusually high/low, or extremely low/high using previously published normative reference data. 10 We then examined the proportion of participants who were broadly normal compared with those who were not in both testing conditions using the Fisher exact probability test for each SCAT3 component. The proportion of participants reporting the presence (ie, greater than zero) of each of the 22 symptoms in the 2 testing conditions was also assessed using the Fisher exact probability test.
Additionally, to assess the effect of concussion history on SCAT3 performance in both conditions, we used a 2-way, mixed-effects analysis of variance to identify significant interactions or main effects of concussion history (yes or no) and test condition. To examine the association between self-rated exertion level and postexercise performance, postexercise SCAT3 scores were compared between participants who rated their exertion as “light” or lower (6-12 on the Borg RPE scale) with those who rated their exertion as “hard” or higher (13-20 on the Borg RPE scale) using independent-samples t tests. Initial screening to determine if a history of concussions, migraines, learning disorders, or mental health problems were potential modifiers indicated no effect of any of these factors. Thus, they were not included as covariates in further analyses. To reduce family-wise type I errors, statistical significance was Bonferroni adjusted to the number of comparisons made, as P < .006. Statistical analyses were performed with Statistical Package for the Social Sciences (version 23; IBM).
Results
A total of 82 professional male rugby league and football athletes (mean age, 21.3 ± 3.2 years; mean education, 12.1 ± 1.2 years) completed the SCAT3 both at rest and after exercise. A majority (60%) reported sustaining a prior concussion (range, 0-6 prior concussions), with the most recent concussion occurring no sooner than 3 months before testing (median, 23.0 months; interquartile range [IQR], 11.0-45.5 months; range, 3-180 months). Only 7 participants reported a history of migraines, learning disorders, or mental health issues (Table 2). During the exercise protocol, participants required a mean of 2.9 ± 1.8 minutes (IQR, 1.9-3.6 minutes; range, 0.7-10.0 minutes) to achieve a 75% maximum heart rate. The Borg RPE scale scores ranged from 7 to 17 (mean, 12.5 ± 2.0; IQR, 11.0-14.0). A weak, nonsignificant correlation between Borg RPE scale scores and the time required to achieve a 75% maximum heart rate was found (r = –0.17; P = .12).
Participant Demographic and Medical Information
At rest, participants reported fewer total symptoms compared with those after exercise (Table 3), but this difference was not statistically different after our Bonferroni adjustment. A greater proportion of participants reported the presence of “feeling slowed down” (20% vs 7%, respectively; P = .037) and “fatigue or low energy” (43% vs 21%, respectively; P = .004) after exercise compared with at rest, but there were no significant differences regarding any other symptoms. Additionally, at rest, the participants completed the modified BESS with fewer errors and required less time to complete the tandem gait test compared with after exercise (Table 3), but neither of these values was statistically significant after adjustment. When assessing the effect of test order, few significant differences were detected, however those who completed the SCAT3 at rest first had significantly better orientation scores and faster tandem gait times compared with those who completed the SCAT3 after exercise first (Table 4).
SCAT3 Components Stratified by Test Condition a
a Values are reported as mean ± SD unless otherwise indicated. BESS, Balance Error Scoring System; SAC, Standardized Assessment of Concussion; SCAT3, Sport Concussion Assessment Tool–Third Edition.
SCAT3 Components Stratified by Test Condition Completed First a
a Values are reported as mean ± SD unless otherwise indicated. BESS, Balance Error Scoring System; SAC, Standardized Assessment of Concussion; SCAT3, Sport Concussion Assessment Tool–Third Edition.
b This table compares rest to rest and exercise to exercise conditions, but the groups are independent and stratified by the order in which they were tested. The group that completed the SCAT3 at rest first had significantly higher orientation at rest and significantly faster tandem gait times after exercise than the group that completed the SCAT3 after exercise first.
Using previously published normative data, 10 the proportion of participants who were considered broadly normal for the modified BESS at rest (67%) was significantly greater than after exercise (49%) (Table 5). For all other SCAT3 components, the proportion of broadly normal participants was not significantly different between the at-rest and postexercise conditions. No significant interactions between testing condition and concussion history and no main effects of concussion history were detected on any of the SCAT3 components.
Performance Classifications of SCAT3 Components Stratified by Test Condition a
a Values are reported as n (%). The cutoff scores presented in this table are normative reference values from a sample of 304 professional male ice hockey players from Finland. 1 Classification ranges are based on the natural distribution of scores because the distributions were not normal. The goal was to select a below/above average cutoff that corresponded with the 25th and 75th percentile ranks, but this usually was not possible given the score distributions. Unusually low/high scores correspond with approximately the 10th and 90th percentile ranks, and extremely low/high scores correspond with approximately the 2nd and 98th percentile ranks. The classifications are worded differently based on the direction of scoring for the SCAT3 components. Symptom scores and number of errors on the modified BESS are referred to as high, and performance on cognitive testing and tandem gait are referred to as low. The months in reverse were stated correctly by 94.0% (n = 265). These cutoff scores are provided for comparison only. Normative reference values for National Rugby League players at rest or after vigorous physical exercise are not available. All of the percentages for the cutoffs were calculated based on the sample sizes indicated for each measure. BESS, Balance Error Scoring System; SAC, Standardized Assessment of Concussion; SCAT3, Sport Concussion Assessment Tool–Third Edition.
b The proportion of participants who were considered broadly normal for the modified BESS at rest was significantly higher than after exercise (Fisher exact test, P = .025).
When rating their self-perceived level of exertion during the 5-minute stationary bicycle protocol, over half (n = 48, 59%) of the participants described it as light or lower (ie, lower Borg RPE scale score). No significant between-group differences in SCAT 3 components were detected among those who rated the exertion as light or lower compared with hard or higher, although medium effect sizes were detected for both total symptoms and symptom severity (Table 6). Finally, test-retest reliabilities were medium for total symptoms, symptom severity, concentration, and total SAC score but low for all other SCAT3 components.
SCAT3 Performance After Exercise Stratified by Borg Rating of Perceived Exertion Scale Score a
a Values are reported as mean ± SD unless otherwise indicated. BESS, Balance Error Scoring System; SAC, Standardized Assessment of Concussion; SCAT3, Sport Concussion Assessment Tool–Third Edition.
Discussion
This study of professional male athletes demonstrated that a brief physical exercise protocol mildly affected performance on some (eg, symptoms, balance, and gait) but not all (eg, cognitive components) aspects of the SCAT3. Each of the 3 hypotheses was partially supported in this study: participants (1) made more errors on the modified BESS and (2) completed the tandem gait test more slowly after exercise compared with at rest, and (3) there was no significant difference in performance on the cognitive measures of the SCAT3 between the 2 conditions. The effect sizes for the differences in modified BESS performance and tandem gait speed were fairly small, and the differences were not significant when taking into account multiple statistical comparisons.
Interestingly, a significantly greater number of symptoms were reported in the postexercise condition than in the at-rest condition, suggesting that exercise may induce symptoms in some athletes. This is consistent with a previous study that observed a significant increase in the number of symptoms reported immediately after a bout of exercise. 19 The individual symptoms that were provoked by the exercise protocol in our study included “feeling slowed down” and “fatigue or low energy.” Similarly, Lee and colleagues 12 reported that “fatigue or low energy,” “feeling slowed down,” and “pressure in the head” were commonly reported among athletes engaged in a standardized exercise protocol. In a study involving uninjured male collegiate athletes, vigorous exercise was associated with increased reporting of balance problems and numbness and tingling. 8 When assessed within 10 minutes of completing an exercise protocol, uninjured collegiate athletes also endorsed significantly more symptoms than their baseline (rested) symptom total, but these effects largely dissipated within the subsequent 24 hours when symptoms were reassessed. 20 Additionally, a 30-minute bout of aerobic exercise has been shown to affect the mechanisms responsible for executive control, potentially leading to improved cognitive functioning. 11 Thus, if cognitive performance improves as a result of exercise performed during sports, potential SCAT3 result changes on the sideline may be caused by the effect of exercise (although this effect has not been shown on the simple cognitive tests on the SCAT3). Regardless, this further supports the notion of a brief rest period before SCAT3 administration. Also in line with these findings, a systematic review has reported that exercise appears to induce concussion symptoms acutely after an injury and concussion-like symptoms in uninjured participants but that the symptoms appear to persist for only a short duration of time. 1 Such findings contribute to the generally accepted notion that the symptoms listed on the SCAT3 are not specific to a concussion, 18 are common in patients with clinical conditions (eg, migraines, anxiety disorders, depression, and bodily injuries), and may be increased by fatigue.
The proportion of participants who were considered broadly normal on the modified BESS, as defined previously by Hänninen and colleagues 10 among professional ice hockey athletes, was significantly greater at rest than after exercise. In fact, after exercise, fewer than half of the study participants were considered broadly normal on the modified BESS. Additionally, a small effect size suggested that the mean number of modified BESS errors was somewhat greater after exercise than at rest, aligning with prior research that has documented more BESS errors immediately after a fatigue protocol. 24 Prior research has documented a similar postexercise decrement on the modified BESS, but with a longer period of rest (20 minutes) the BESS scores returned to similar levels as baseline. 22
The sample of professional athletes who participated in this study performed worse in the at-rest condition compared with the professional Finnish ice hockey players who were used to create normative data for the SCAT3 10 on a number of the tests. Therefore, the Finnish norms might not be appropriate for other athlete groups. Specifically, 10% of the professional athletes in our study at rest and 12% after exercise obtained substantially low SAC scores. In contrast, tandem gait performance in both conditions was broadly normal for greater than 90% of the professional athletes in this study. While approximately 7.8% of professional ice hockey athletes were classified as unusually low on the modified BESS, 24.4% and 35.3% of the professional athletes in our study were classified as unusually low in the at-rest and postexercise conditions, respectively. Because the interrater reliability of the modified BESS is low, 7 differences in how test administrators interpreted errors may have contributed to the discrepancy between studies. Finally, as the tandem gait test’s primary outcome variable, test time, is a more objective measure than observer-rated errors, it may possess a higher degree of translatability across different sports and test administrators.
Based on the results of our study, a rest period of more than 5 minutes may be necessary and appropriate for tests of physical function, such as the modified BESS and tandem gait, but less necessary for components of the SCAT3 that probe cognitive abilities such as memory and concentration. Similar to our findings, prior work has found that high-intensity exercise contributes to worse performance on the single-leg stance component of the modified BESS and the tandem gait test. 23 Furthermore, the test-retest reliability of the SCAT3 may differ based on the amount of time between tests. 4 Specifically, the reliability of the SAC and BESS is higher among uninjured adolescent and young adult athletes when the 2 tests are administered within 1 week of each other compared with when they are administered approximately 6 months apart. 4 In our sample, relatively modest test-retest correlations were found, perhaps because of the different conditions in which participants were tested despite tests occurring, on average, around 3 weeks of each other. Therefore, in addition to the effect of exercise on SCAT3 performance, clinicians should consider the duration of time since the baseline examination because practice effects may dissipate if test-retest intervals exceed 1 month.
The generalizability of these findings to other professional male athletes and the application of the results to a real-world, game-day SCAT3 sideline assessment may be limited. A majority (59%) of the professional athletes rated their exertion levels as light or lower, while only 4% perceived the exercise protocol as very hard or harder. Furthermore, reported fatigue scores and reported exertion levels were not associated. These observations may be because our study design incorporated a brief exercise protocol. Specifically, 5 minutes of activity on an exercise bicycle, at 75% age-predicted maximal heart rate revisited, is unlikely to reflect the exertion involved in match play for the majority of athletes. Although participants selected a range of different Borg RPE scale scores during the exercise protocol, no significant differences were found for each rating category on SCAT3 performance. Although this lack of difference between groups indicates that the perceived level of exercise had no effect on SCAT3 performance, some athletes likely felt as if they did not experience a noticeable level of physical exertion. Thus, the generalizability of our findings to on-field SCAT3 assessments may be limited. Future studies may better assess the effects of a more “game-like” exercise protocol aimed to increase the intensity and duration of exercise.
This study was also limited in that we were unable to obtain information regarding participants’ previous SCAT3 completion. That is, we did not have data on how many times each participant had completed the SCAT3 and the extent to which his current performance may have been improved by possible practice effects.
In conclusion, the interpretation of in-game (sideline) SCAT3 results should consider the potential effects of fatigue on an athlete’s performance, particularly if preseason baseline data have been collected when the athlete was well rested. Specifically, exercise appears to affect symptom reporting, balance, and gait more so than the cognitive components of the SCAT3, and medical providers involved in the sideline care of athletes should consider these potential exercise-induced changes during the interpretation of test performance.
Footnotes
One or more of the authors has declared the following potential conflicts of interest or source of funding: W.P.M. receives royalties from ABC-CLIO, Springer Publishing, and Wolters Kluwer and receives philanthropic research support from the National Hockey League Alumni Association through the Corey C. Griffin NHL Alumni Pro-Am Tournament; he is also supported by a grant from the Football Players Health Study at Harvard University, which is funded by the National Football League Players Association. G.L.I. has been reimbursed by the government, professional scientific bodies, and commercial organizations for discussing or presenting research relating to mild traumatic brain injuries (TBIs) and sport-related concussions at meetings, scientific conferences, and symposiums. He also has a clinical and consulting practice in forensic neuropsychology involving patients who have sustained mild TBIs (including athletes); has received research support from several test publishing companies including ImPACT Applications, CNS Vital Signs, and Psychological Assessment Resources; and receives unrestricted philanthropic support from the Mooney-Reed Charitable Foundation and ImPACT Applications. A.J.G. has a clinical practice in neuropsychology involving patients who have sustained sport-related concussions (including current and former athletes); has operated as a contracted concussion consultant to the Australian Rugby Union from July 2016; has received travel funding from the Australian Football League to present at the Concussion in Football Conference in 2013 and 2017; has received previous grants from the New South Wales Sporting Injuries Committee, the Brain Foundation (Australia), and the Hunter Medical Research Institute (HMRI), supported by Jennie Thomas; and is currently funded through the HMRI, supported by Anne Greaves, and the University of Newcastle’s Priority Research Centre for Stroke and Brain Injury.
Ethical approval for this study was obtained from the University of Newcastle’s Human Research Ethics Committee (reference No. H-2015-0352).