Longitudinal Improvement in Balance Error Scoring System Scores among NCAA Division-I Football Athletes

Introduction

The Sport Concussion Assessment Tool (SCAT) is a series of tests used to aid the clinician in concussion evaluation. When feasible, baseline assessment of these tests is recommended, which includes an assessment of balance. The Balance Error Scoring System (BESS) test is a clinical tool used in the standardized concussion evaluation and is considered a reliable and valid means of evaluating balance in athletes with suspected concussion. ¹ The BESS is also used in assessing for balance impairments in athletes before return to play post-concussion by comparing the number of errors made on the test at baseline to the number of errors made at the time of potential injury. Although the BESS is a widely used clinical tool, questions exist regarding its applicability to the broad spectrum of all athletes across age, sex, sport, injury history, and testing environment. ^2,3

Recent studies bring question to the stability and reliability of baseline BESS scores over time. Burk and colleagues reported improvement in BESS scores during a 90-day intercollegiate season in female athletes, whereas Mulligan and colleagues reported that after 4 weeks, the learned effect of repeat BESS testing in college aged adults continued. ^4,5 Aside from the athlete's performance on this testing, the reliability of the scoring of the exam has also been questioned. A study evaluating inter-rater reliability found that those experienced and trained in scoring the BESS do not exhibit clinically significant reliability in scoring the BESS. ⁶ Attributed to the multiple variables that may affect the BESS scores, we hypothesize that baseline BESS performance is a moving target making the BESS a progressively insensitive marker of concussion. The aim of this study was to evaluate for change in baseline BESS scores on an NCAA Division-I football program; the evaluation of inter- and intrarater reliability were secondary outcomes.

Methods

Institutional review board approval was obtained for this prospective cohort study. In year 1 of the study, 23 incoming freshman football athletes were videotaped performing baseline BESS testing before initiating organized football activities. Testing was performed on both firm and foam surfaces with double, single, and tandem stances on each surface as outlined in previous works. ³ All athletes in this study had been cleared for full participation in athletic activities by the yearly pre-participation examination. None of the athletes had suffered a previous concussion as documented on their entrance pre-participation evaluation. Information regarding an individual athlete's concussion history during the year-long study was maintained by the medical team overseeing the football athletes. After 1 calendar year of participation in organized football activities, the same 23 athletes were videotaped performing the BESS test.

In year 2 of the study, 25 incoming freshman football athletes were videotaped in the same manner as those athletes in year 1 of the study. Of note, 3 of these athletes had suffered a previous concussion as documented on their entrance pre-participation evaluation. Additionally, 21 of the athletes from year 1 (2 athletes had left the team after year 1) were videotaped after year 2 in the football program. No athletes had evidence of concussion on their pre-participation examination preceding either year.

The videos were de-identified and randomized. De-identification included removing the athletes' name and identifying information from the videos. The faces of the athletes were visible in the videos, given that viewing an athlete's eyes is necessary to properly score the BESS test. Four certified athletic trainers and three senior athletic training students (now certified athletic trainers), none of which were affiliated with the football team and/or the study subjects, were blinded to the purpose of the study. These individuals underwent standardized physician-guided training on scoring the BESS. Performed by the authors, this training included viewing practice videos, providing feedback, and performance based testing. Each reviewer scored every video independently and was blinded to the scores assigned by the other reviewers. The paired t-test was used to assess for change in score over time for the firm surface, foam surface, and the cumulative BESS score. Inter-rater reliability was calculated utilizing the intraclass correlation coefficient. To assess the intrarater reliability, 10% of all the previously scored videos (e.g., upon matriculation, after year 1 and after year 2) were selected utilizing a random number generator. The athletic trainers individually scored these tests in the same manner as with the previous videos.

Results

Cumulative errors on the BESS significantly decreased from a mean of 20.3 at baseline to 16.8 after 1 year of participation (95% confidence interval [CI] = −5.27 to −1.82); p < 0.01). Individually, the firm (95% CI = −2.35 to −0.38; p < 0.01) and foam (95% CI = −3.66 to −0.71; p < 0.01) scores significantly decreased (Table 1).

In the 21 athletes evaluated following the second year in the program, additional improvement in BESS scores was observed for the cumulative score as well as both the foam and firm surfaces individually. However, this did not reach statistical significance (Table 2).

Intraclass correlation coefficient for inter-rater reliability for the cumulative BESS score was 0.75 for videos obtained at the beginning of year 1, 0.65 for those obtained at the end of year 1 and beginning of year 2, and 0.75 for those obtained after year 2 (Table 3). Intrarater reliability was 0.81.

For those enrolled in year 1 of the study, 1 of the 23 athletes sustained two concussions during the year, whereas 2 additional athletes each sustained one concussion. During year 2 of the study, 1 athlete who sustained a concussion in the previous year was diagnosed with another concussion during year 2. One additional athlete suffered one concussion. For those enrolled in year 2, 3 athletes sustained one concussion each, and 1 athlete sustained two concussions.

Discussion

The baseline BESS score is commonly used on the sideline and in the athletic training room as part of the clinical assessment of concussion. We report a statistically and clinically significant improvement in BESS scores that occurs during the first year of participation in an NCAA Division-I football program and continued subsequent improvement, though not reaching statistical significance, during year 2 in the program. Given that an athlete's expected performance on the BESS improves over time, it becomes a progressively insensitive marker for concussion, amounting to a moving target when evaluating athletes for concussion. A sideline evaluator could evaluate a concussed athlete who produces an improved BESS score as compared to the documented baseline score of the athlete, which may have been collected weeks or months preceding the concussion evaluation. The frequency of reassessing an athlete's baseline BESS score needs be established; however, at what frequency this re-evaluation should occur remains a focus of future research.

The Sport Concussion Assessment Tool – 2nd Edition (SCAT2) utilized a modified BESS test (testing on the firm surface only); a past study on normative data in collegiate athletes reports modified BESS scores. ⁷ The Sport Concussion Assessment Tool – 3rd Edition (SCAT3) and The Sport Concussion Assessment Tool – 5th Edition (SCAT5) offer the option of adding the foam surface for further assessment of balance, which improves the sensitivity of balance testing. ^2,8 The authors of this study show a statistically significant decrease in the number of errors made in the cumulative score, individual firm surface, and the foam surface after 1 year of participation. Additionally, between the end of year 1 of participation and the end of year 2, further improvement (decreased number of errors made) was noted, although it fell short of statistical significance. The clinically meaningful change of the BESS test is unknown. However, it is safe to assert that if an athlete is improving on the test over time, the sensitivity of the test worsens. Inter-rater and intrarater reliability in this study is consistent with the reliability between experienced scorers as reported by Finnoff and colleagues. ⁶

In this study, the number of cumulative errors at baseline was higher than previously published normative data in adults. ⁹ The reason for the higher number of errors compared to published normative data is uncertain, although several factors may contribute. These factors may include the different cohorts studied and variability in intrarater and inter-rater reliability in the studies. In addition, the athletes at matriculation are transitioning from high school to college. High school male athletes make more errors on the BESS than college-age male athletes. ¹⁰

It is important to remember that the balance testing is only one aspect of the assessment that clinicians may utilize when evaluating athletes for concussion. It is not possible to definitively determine whether athletes purposefully perform poorly on baseline balance testing for secondary gain, (e.g., avoiding a diagnosis of concussion during the athletic season). Thus, patterns of performance must be interpreted by the clinician. ¹¹ The athletes in this study were not advised as to the purpose of the baseline BESS testing, and no other objective evidence of “sandbagging” was appreciated by the authors.

Limitations of this study include a relatively small sample size and moderate inter-rater reliability. There are also previously studied limitations on BESS scores, including individual variables such as age of the athlete, ⁵ learning effects of repeat testing, ^5,12,13 and level of fatigue of the athlete. ¹⁴ Eight subjects sustained concussions during the study. It is possible that either the concussion itself, increased exposure to the BESS test, or proprioceptive training associated with participation in the football program affected the results. However, our study is not significantly powered to detect such changes. Further investigation is warranted to evaluate the effect of additional exposure to the BESS, diagnosed concussions, and the proprioceptive training associated with football program participation. These considerations are part of the authors' ongoing data collection and analysis.

These authors also aim to evaluate additional factors that may impact longitudinal changes in BESS scores, including assessing the effects of multiple seasons of Division-I football participation, player position, player demographics, and concussion history. Further understanding of how BESS performance changes over time will clarify the need to modify the BESS test and/or alter the frequency with which repeat baseline testing is performed.

In conclusion, this study shows a statistically and clinically significant improvement after 1 year of participation and continued improvement (though not reaching statistical significance) in BESS scores between the first year and second year of participation in an NCAA Division-I football program. In addition, the cumulative number of errors in this cohort was higher than previously published norms. Therefore, NCAA Division-I football athletes should undergo baseline BESS testing at least yearly if the BESS is to be utilized as a diagnostic test for concussion.