Abstract
Many states and sports leagues are instituting concussion policies aimed at reducing risk of morbidity and mortality; many include mandates about the provision of concussion education to youth athletes. However, there is limited evidence if educational materials provided under these typically vague mandates are in fact effective in changing concussion risk-related behavior or any cognition predictive of risk-related behavior. The purpose of this pilot randomized controlled study was to conduct a theory-driven evaluation of three publically available concussion education materials: two videos and one informational handout. Participants were 256 late adolescent males from 12 teams in a single league of ice hockey competition in the United States. Randomization of educational condition occurred at the team level. Written surveys assessing postimpact symptom reporting behavior, concussion knowledge, and concussion reporting cognitions were completed by participants immediately before receiving their educational intervention, 1 day after, and 1 month after. Results indicated no change in any measure over any time interval, with the exception of perceived underreporting norms. In one of the video conditions, perceived underreporting norms increased significantly 1 day after viewing the video. Possible content and viewing environment-related reasons for this increase are discussed. Across all conditions, perceived underreporting norms increased 1 month after intervention receipt, raising the possibility that late in the competitive season underreporting may be perceived as normative. The need for the development of theory-driven concussion education materials, drawing on best practices from health behavior scholars, is discussed.
In recent years, increasing attention has been brought to the incidence, prevalence, and possible long-term consequences of concussions (Daneshvar, Nowinski, McKee, & Cantu, 2011; McCrory et al., 2013; McKee et al., 2013; Seichepine et al., 2013; Zemek, Farion, Sampson, & McGahern, 2013). Centers for Disease Control and Prevention (CDC) estimates suggest that between 1.6 and 3.8 million concussions from sport and recreational activities are sustained every year in the United States (Langlois, Rutland-Brown, & Wald, 2006). Among male junior ice hockey players (mean age of 18.2 years, range of 16-21 years), at least 21.5 concussions occur per 1,000 times an athlete participates in a game or practice (Echlin, Johnson, et al., 2010).
Among athletes of all ages, recent evidence suggests that there are magnified neurological consequences of sustaining a second impact prior to full recovery (Prins, Alexander, Giza, & Hovda, 2013). Immediate identification and removal from play of athletes who have sustained a concussive impact are critical steps for reducing this avoidable risk. Since many symptoms of concussions are not visible to observers, the honest and timely report of symptoms by injured players is an important component of this removal from play process. Across multiple sports, ages, and levels of competition many athletes underreport symptoms (Kerr et al., 2014) making symptom reporting an important behavioral target for concussion risk reduction. Frequently cited reasons for this nondisclosure include a lack of knowledge about concussions and not wanting to let teammates or coaches down (Kerr et al., 2014).
Athlete education about concussions is one strategy that has been used to encourage symptom reporting. Every U.S. state has now passed concussion legislation intended to protect the health of youth athletes (Harvey, 2013). While the exact content of this legislation varies by state, at least 80% require that youth athletes be educated about concussions in some capacity (Harvey, 2013; Tomei, Doe, Prestigiacomo, & Gandhi, 2012). Similarly, sports leagues such as the National Collegiate Athletic Association require concussion education for athletes (National Collegiate Athletic Association, 2013). Despite the seemingly intuitive importance of providing athletes with information about concussions, there has been limited evaluation to date about whether commonly used and publically available concussion education materials are in fact achieving the implicit or explicit goal of encouraging post–head impact symptom reporting and removal from play. Of the educational materials that have been evaluated, results do not indicate universal efficacy (Bagley et al., 2012; Benson et al., 2013; Cook, Cusimano, Tator, & Chipman, 2003; Cusimano, Chipman, Donnelly, & Hutchinson, 2014; Cusimano, Nastis, & Zuccaro, 2013; Echlin, Tator, et al., 2010; Kroshus, Daneshvar, Baugh, Nowinski, & Cantu, 2014; Miyashita, Timpson, Frye, & Gloeckner, 2013). Furthermore, many of these published evaluations include change in knowledge about concussions as a primary outcome. In contrast, existing evidence suggests the importance of psychosocial theory-driven evaluation, measuring behavioral outcomes and proximal psychosocial predictors of reporting behavior such as perceived reporting norms and intention to report concussion symptoms (Keats, Emery, & Finch, 2012; King-Chung Chan & Hagger, 2012; Kroshus, Baugh, Daneshvar, & Viswanath, 2014; Kroshus, Daneshvar, et al., 2014; McGlashan & Finch, 2010; Verhagen, van Stralen & van Mechelen, 2010).
Randomized evaluations of existing concussion education materials using theory-driven measures and behavioral outcomes are critical for understanding whether these materials are actually effective in altering athletes’ concussion reporting behaviors. This type of evaluation is also important for understanding mechanisms of change and for informing appropriate cognitive targets of future intervention design. The Theory of Planned Behavior (TPB; Ajzen, 1991) has recently been suggested as relevant for explaining the rational and volitional component of concussion reporting behavior (Chrisman, Quitiquit, & Rivara, 2013; Kroshus, Baugh, et al., 2014; Register-Mihalik et al., 2013). Within the conceptualization of TPB, attitudes about outcomes of performing the behavior, perceived behavioral norms, and perceived behavior control predict behavioral intention, with both intention and perceived behavioral control predicting performance of the behavior itself (Ajzen, 1991). Guided by TPB, the purpose of the present pilot study was to assess the efficacy of three existing concussion education tools: the educational video Concussions in Ice Hockey (National Athletic Trainers’ Association/National Academy of Neuropsychology, n.d.), the frequently downloaded CDC Heads Up Concussion in High School Sport: A Fact Sheet for Athletes handout (CDC, 2013), and the recently released full-length concussion documentary Head Games (James & Sheridan, 2012). The video Concussions in Ice Hockey was selected for inclusion based on the results of a previous cohort study examining education content, delivery, and effectiveness in male collegiate ice hockey teams (Kroshus, Daneshvar, et al., 2014). In this cohort, the only team with a posteducation change in any measured variable was the team that viewed this particular video; however, this video was delivered in conjunction with other educational materials, and confounding by other team compositional and contextual factors could not be eliminated as explanations for the observed change. Nonetheless, the finding suggested the utility of future randomized study of whether or not the video itself is in fact effective in changing reporting behavior or any psychosocial predictor of reporting behavior. The second video, Head Games, was selected for inclusion based on the written comments of the male collegiate ice hockey players in the previously described cohort study regarding the type of concussion education they believed would be most effective. They most frequently indicated that they would like to watch a video with information delivered by professional athletes about the long-term consequences of concussions; this description largely matches the content and delivery of information in the documentary Head Games. Although Head Games was not designed explicitly as an educational tool for athletes, it has been used in that capacity in some sports leagues (MLL Communications, 2013). Finally, the CDC handout Heads Up Concussion in High School Sport: A Fact Sheet for Athletes was selected because written handouts, and the CDC written handouts in particular, are among the most frequently mandated types of concussion education for athletes (Baugh, Kroshus, Bourlas, & Perry, in press). This handout lists basic concussion facts, common concussion signs and symptoms, and provides instructions about what the athlete should do if they suspect they have sustained a concussion.
The aim of this pilot study was to provide additional information about the efficacy of existing publicly available concussion education materials in one population of athletes. It represents a starting point and a possible model for theory-driven evaluation of concussion education efficacy.
Method
Sample
Coaches of one Tier III male Junior ice hockey league were contacted and invited to have their team members participate in the research study on a voluntary basis, with 12 of the 14 league teams agreeing to participate. On these teams, individuals were eligible to participate if they were 18 years of age or older. Of eligible players on participating league teams, 256 players (97%) consented to participate in study activities. On average, participants were 19.15 years old (SD = 0.85), had been playing organized ice hockey for 13.74 years (2.05), had 1.02 previously diagnosed concussions and (SD = 1.39) and 2.64 suspected (undiagnosed or unreported) concussions (SD = 2.83). All participants were either in their final year of high school or had recently graduated; Junior ice hockey is typically a stepping stone between high school hockey and college hockey. All teams were from the northeast region of the United States, and 97% of participants self-identified as White.
Procedure
At approximately the midpoint of the 2012-2013 ice hockey season, the same member of the research team met with each ice hockey team to explain the purpose of the study, answer questions, and obtain written informed consent. Immediately after, consenting participants completed a written survey that took approximately 15 minutes (see the appendix). After completing the survey, the team received their randomly assigned educational intervention. All intervention materials were introduced without didactic comment other than informing team members that they were receiving educational materials about concussions. All participants were given the Heads Up Concussion in High School Sport: A Fact Sheet for Athletes handout, with the control condition receiving only this as their educational intervention. Four teams were randomized to also watch the 12-minute video Concussions in Ice Hockey. This publically available video is joint production of the National Athletic Trainers Association, the National Academy of Neuropsychology, the National Hockey League (NHL), and the NHL Players’ Association. Four other teams were randomized to watch the 90-minute documentary Head Games. Because of the substantial variation in intervention length, teams were assigned to their educational condition using a random number generator after coach agreement and prior to baseline contact with team members so that coaches could more accurately build the research activities into their team schedule. No teams declined participation after randomization. Videos were watched in the area in which teams viewed game video, typically the locker room or a viewing area adjacent to the locker room. In the two video conditions, the researcher administering the protocol remained in the room for the duration of the video to ensure compliance with viewing protocol. In all conditions, consenting team members completed a follow-up written survey the next day and again approximately 1 month later. Just over one third of participants (39.2%) completed surveys at all three time points, a reduction primarily attributable to coach noncompliance with follow-up scheduling for several teams in the handout control and Concussions in Ice Hockey conditions; coach noncompliance occurred in 28.9% of all cases (accounting for 73.7% of all missing cases). In sum, approximately 35.6% of data were missing at Time 2 and 38.9% of data were missing at Time 3, with 13.0% of participants missing at both time points. Research activities were approved by the Harvard School of Public Health Institutional Review Board.
Measures
Survey measures were based on TPB (Ajzen, 1991), a theory that has recently been suggested for use in the design and evaluation of concussion education programs (Chrisman et al., 2013; Kroshus, Baugh, et al., 2014; Kroshus, Daneshvar, et al., 2014; Register-Mihalik et al., 2013). Concussion reporting cognitions measured included the TPB-based constructs of perceived negative reporting outcomes, perceived underreporting norms, reporting self-efficacy, and reporting intention. Detailed information on these measures, including information on reliability and validity, is available elsewhere (Kroshus, Baugh, et al., 2014). Concussion knowledge was assessed using a modified version of Rosenbaum and Arnett’s (2010) Concussion Knowledge Index (Kroshus, Baugh, et al., 2014). Kaut, DePompei, Kerr, and Congeni’s (2003) Head Injury Questionnaire was used to assess postimpact symptoms associated with concussion. At baseline, participants indicated whether they had experienced each of the eight symptoms in the season to date. They also indicated whether, during the season to date, they had experienced any of the listed symptoms and not told their coach or athletic trainer. At 1-month follow-up, they responded to the same questions; however, the reference period was modified to include the previous month rather thtimesan the season to date. To account for the different reference periods reporting behavior is described per 10,000 athletic exposures. Responses to all symptom and reporting questions were dichotomous (no = 0, yes = 1), with nonreport of symptoms conditional on having indicated that postimpact symptoms were experienced in the indicated time period.
Analysis
The relationship between education and each measure of interest over time was determined via repeated-measures analysis of variance (ANOVA), using listwise deletion of cases with missing values and Greenhouse–Geisser correction when sphericity could not be assumed, with post hoc pairwise comparisons assessed via Tukey HSD testing (SPSS v20). Bonferroni corrections were applied to within measure contrasts. Multiple imputation was performed to account for missing variables, most common at 1-month follow-up, under the assumption that data were missing at random (R v.2.15, Amelia II). In addition, follow-up analyses were conducted with multilevel modeling to explore the sensitivity of the analysis to the statistical method, and pattern-mixture models (Hedeker & Gibbons, 1997) were employed to explore whether attrition bias may have affected the results. Two-level models were fit to the data, with repeated measures nested in participants. A three-level structure was explored, but team-level clustering was nonsignificant for all models.
Results
To validate the randomization procedure, we conducted an ANOVA on the baseline levels of study variables across the three treatment arms. No baseline differences were statistically significant. When examining change over time, significant differences were found between education type and reporting norms over time on repeated-measures ANOVA with Greenhouse–Geisser correction, F(3.25, 107.24) = 3.45, p = .017; no other measures displayed significant between education type differences in effects over time (Table 1). While underreporting per 10,000 athletic exposures at 1-month follow-up was significantly higher compared with baseline for athletes watching Head Games, the difference in reporting by education type at 1-month follow-up was not significant (p = .368). Although there were significant changes in the subjective reporting norms across all athletes between baseline and the two follow-up time points regardless of education type, F(1.62, 107.2) = 31.2, p < .001, the subjective reporting norms for the Concussions in Ice Hockey group was significantly greater than for the control condition, but not Head Games, at 24-hour follow-up (d = 10.5, p = .046; d = 6.4, p = .184, respectively). To determine whether the lack of additional findings was simply due to loss of power resulting from complete case analysis, multiple imputation was performed to account for missing observations. To account for highly collinear variables, a ridge prior of 21 (~0.01n) was imposed. The results mirrored those from the repeated-measures ANOVA as subjective reporting norms were the only outcome that varied over time based on education type.
The Change in Measures of Interest as a Function of Time and Education.
Note. AE = 10,000 athletic exposures; CIH= Concussions in Ice Hockey video; Control = CDC handout only condition; ANOVA = analysis of variance. F and p ANOVA values are given for the data in each row. “Overall” F and p values are given for repeated-measures ANOVA, with education type as a between-subjects factor. “Total” F and p ANOVA values are given for the full sample, irrespective of education type. Variables significant at p < .05 are shown in bold.
To better understand potential differences between dropouts and completers (completer status was operationalized as individuals having completed the Time 3 survey), we conducted several sensitivity analyses. First, we conducted a series of t tests on study measures and covariates. We examined diagnosed concussions using a Mann–Whitney test to account for the nonnormal distribution. No between-group contrasts were significant.
In the sensitivity analysis with multilevel growth curves, quadratic trajectories were found to best fit each of the five outcomes. While quadratic models are not typically advised for studies with fewer than four waves of data, they are appropriate here because no time-variant predictors were included in the model, and because the focus was on comparing point estimates rather than interpreting trajectories. The results were largely consistent with the ANOVA, finding subjective reporting norms for Concussions in Ice Hockey group significantly higher than both Head Games and the control condition at 24-hour follow-up (t = 3.64, p < .001; t = 4.60, p < .001, respectively), but finding no between group differences for intention, knowledge, or attitudes.
Pattern mixture models were used to explore attrition bias. Participants without data at 1-month follow-up were coded as dropouts. By including dropout status as a covariate in the model, we were able to test whether participants who ultimately dropped out had changed differently on the variables of interest than did completers from baseline to 24 hours. The pattern mixture models indicated no such differences, suggesting that parameter estimates were not biased by missing data.
Discussion
As concussion education becomes increasingly mandated for athletes, it is important to understand whether existing materials are effective in changing symptom-reporting behavior. The present findings give cause for pessimism. The only significant changes across any condition were in perceived reporting norms, and these changes were not in an encouraging direction. One-day posteducation the players in the Concussions in Ice Hockey condition reported significantly higher norms scores, indicating increased perceptions that that their teammates and most other athletes would engage in unsafe reporting behaviors. It is possible that when professional hockey players described their experiences with concussions in the video they in fact normalized playing through injury. It is also possible the effect was attributable to an interaction of the video with the viewing context. The researcher who sat in the room with each team during all viewings reported a consistent response across teams to this particular video. Whenever footage was shown of big collisions, even though it was made clear that the individual on the receiving end sustained a concussion, many players cheered, and the mood in the room often temporarily turned rowdy. It is possible that this collective behavior, even if only endorsed by a few individuals, had the effect of making players think their teammates did not take concussions seriously. Although Head Games also showed big collisions, these hits were nested within a longer film and often preceded and followed by seemingly sobering content about the long-term consequences of concussions; perhaps consequently, the viewing of these collisions did not precipitate cheers and rowdiness. These observations speak to the importance of considering the role the viewing environment plays in education effectiveness. Furthermore, they highlight the need to consider whether in some populations, such as adolescent males, showing an educational video with a high ratio of big collisions to educational content might unintentionally undermine the purpose of the video. Nonetheless, due to their anecdotal nature these observations should be interpreted with caution and explored further with rigorous ethnographic data collection.
At 1 month posteducation there were no lasting changes for any educational condition in concussion reporting behavior or in any of the measured constructs (knowledge, reporting outcomes, self-efficacy, reporting intention) other than the perceived norms measure and no between-condition differences in any variable. Across all conditions, perceived norms increased significantly compared with baseline. Given the trajectories of change across all conditions, it is possible that the 1-month change in norms is attributable to an exogenous factor such as seasonality. One-month follow-up data collection occurred during the playoffs, a time of the competitive season in which there may be different perceptions of what type of injury reporting behavior is acceptable. Similarly, while there were no significant differences between education type and reporting behavior, in the sample as a whole there was a significant increase in underreporting behavior per 10,000 athletic exposures comparing baseline with 1-month follow-up (p = .001). Future research is encouraged to explore more comprehensively whether reporting norms and reporting behavior vary by timing in the competitive season. Should this temporal association be confirmed, providing teams with a “booster” educational session late in the competitive season is a possible risk-reducing strategy that should be evaluated.
Overall, these results should be seen as preliminary given the relatively modest sample size, pilot methodology, and attrition rate. Only 40% of participants completed surveys at all three time points. Although analyses indicated that bias due to attrition was unlikely, to the extent that differential attribution was not captured by these analytic techniques, it is most likely that those completing the surveys were more concerned about concussions than nonrespondents, so bias in the direction of safer reporting would have been expected. It is also possible that the lack of change observed for some of the cognitive variables is attributable to the insufficiency of the measures rather than a lack of change in the underlying constructs of interest. As psychosocial theory is increasingly used to explain reporting behavior and evaluate concussion education, it is important that gold standard and population-specific measures of relevant psychosocial constructs be developed. It is also important, where possible, to include clinical rather than self-report outcome measures and to measure contextual factors that may facilitate or constrain reporting such as coach attitudes and communication about concussions and access to personnel to whom concussions can be reported (e.g., certified athletic trainer on the sidelines). In addition to using improved measurement tools, future research is encouraged to extend the type of evaluation modeled in this study to other populations. It is possible that the apparent lack of effectiveness of these educational materials was because the 1-month follow-up period was not long enough for adequate variability in reporting behavior and that significant differences based on condition may have been seen with longer follow-up. It is also possible that the apparent lack of effectiveness of these materials was a function of the population studied. Athletes in this sample had been playing organized hockey for an average of approximately 13 years. It is likely that within that time frame they had been exposed to a substantial amount of information about concussions. Consequently, the information provided by the educational materials in this study may not have been new to them or may not have been sufficient to dislodge long-held beliefs, such as the perceived threat of concussions or what they perceive to be positive or negative consequences of reporting symptoms of a concussion. Future research is encouraged to systematically assess the age or level of sport at which certain types of concussion education are most effective. It is possible that the CDC handout targeted at “youth” athletes may not have been appropriate for the participants in this study as they are over the age of 18. Provvidenza et al. (2013) have written at length about the importance of tailoring concussion-related information provision to the specific knowledge needs of populations. For concussion education to modify reporting risk, it is important that population-specific knowledge needs be determined and that educational interventions be developed that address these particular knowledge needs.
Implications for Policy and Practice
While we reemphasize the pilot nature of the present results, they nonetheless suggest a troubling prospect: that at least some of the concussion education being provided to athletes may be ineffective by almost any metric. Even more troublingly, these findings raise the possibility that some educational materials, when delivered to certain populations in a group setting, may have an unintended iatrogenic effect. If concussion education is to be used as a state and sport league level risk-reducing strategy, it is important that the education provided does in fact reduce risk. Organizations and states mandating concussion education are encouraged to recommend education with proven population-specific efficacy in changing, at the very least, known predictors of concussion reporting behavior such as concussion reporting intention. Consequently, there is a clear need for the development of population-specific and theory-driven educational interventions. There are many models of effective theory-driven injury prevention interventions in other domains (e.g., Gielen & Sleet, 2003). Those designing new concussion education programs are strongly encouraged to build on these literatures. Providing educational content that addresses the TPB constructs of attitudes about reporting, perceived reporting norms and reporting self-efficacy is particularly encouraged. These constructs are likely to vary between different populations of athletes and formative work is needed to determine what consequences of reporting are particularly salient to a given population, who the most important normative referents are, and what conditions make performance of the behavior particularly challenging. In this particular population, short-term athletic consequences of reporting (such as losing a spot in the lineup, or letting teammates down) are strongly associated with reporting behavior (Kroshus, Baugh, et al., 2014). Information provision about the negative athletic consequences of continued play while symptomatic, such as slowed reaction time, and how this can in fact hurt the individual and team’s performance, may be useful for addressing this belief (Kroshus, Baugh, et al., 2014). The way in which this information is delivered should be considered carefully. Facts about reaction time and diminished athletic performance may be less compelling than experiential activities. For example, games that simulate decision making and reaction time while “symptomatic” may make these facts more salient, and may have the additional benefit of increase reporting self-efficacy (Bandura, 1982). Addressing the construct of perceived reporting norms directly may also be an important educational strategy. Prospective research with a similar population of male ice hockey players has indicated that perceived athlete norms are strongly associated with in-season concussion reporting behavior and has suggested that these norms may often be misperceived, with athletes perceiving that “most athletes” engage in less safe behaviors than they themselves do (Kroshus, Kubzansky, Goldman, & Austin, 2014). Consequently, it may be useful to provide content that corrects norms. For example, building on similar work with athletes in correcting misperceived alcohol consumption norms, Kroshus, Kubzansky, Goldman, and Austin (2014) have suggested the potential utility of an interactive educational strategy in which athletes provide information about their concussion reporting beliefs and intentions, with this information is communicated in real time to their teammates in a group setting.
While concussion education alone is not sufficient to eliminate the public health burden of concussions from sport, it is a relatively feasible component of a multifaceted and multilevel solution. Our findings suggest that, at least in a population of late adolescent competitive male ice hockey players, this solution should include educational materials other than those included in this study.
Footnotes
Appendix
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Christine M. Baugh and Daniel H. Daneshvar received funding from Head Games, The Movie.