Abstract
Background. Concussion education for coaches is being increasingly recommended by leading sports organizations, yet the effectiveness of available materials has not often been assessed. This study evaluated the impact of the National Collegiate Athletic Association’s (2017) publication: “Concussion safety: What coaches need to know” (the “NCAA Fact Sheet for Coaches”) on participants’ concussion knowledge and safety supportive intentions. Method. NCAA head coaches (n = 779) were randomized to view the NCAA Fact Sheet for Coaches either before (intervention condition) or after (control condition) responding to questions assessing concussion knowledge and intentions to engage in three behaviors (talk to team about concussion safety; allow symptomatic athlete to continue play; encourage medical staff to return athlete as quickly as possible). Results. Knowledge scores were significantly higher among coaches in the intervention condition as compared to the control condition, largely driven by differences in knowledge about the consequences of continued play postconcussion (injury, reaction time, possible death). Fewer coaches in the intervention condition expected that they would allow an athlete with a suspected concussion continue to participate in a game or practice or that they would encourage medical staff to return athletes to play as quickly as possible. Discussion. The present study provides evidence that educational content, like that contained in the NCAA Fact Sheet for Coaches, may have a positive short-term impact on concussion knowledge and behavioral intentions of coaches. In light of these findings, sports organizations should consider providing similar educational materials to coaches, evaluating whether the information is impactful in their population.
Concussions are an anticipated part of sports that involve routine contact or collision and can also occur in noncollision sports as a result of accidental contact with another athlete, equipment, or playing surface (Zuckerman et al., 2015). Concussion education for coaches is being increasingly recommended (Broglio et al., 2014; Harmon et al., 2013; McCrory et al., 2017; NCAA, 2017a). However, there is variability in the type of education that coaches receive (Cusimano et al., 2018; Kroshus, Baugh, & Daneshvar, 2016). The Centers for Disease Control and Prevention’s Heads Up initiative is the source of concussion information for coaches and other sport stakeholders with the widest reach in the United States. It is composed of a suite of materials that can be selectively disseminated within sports organizations, including an online training and handouts. Youth and high school coaches self-report improved knowledge and behavioral practices following exposure to a subset of these resources; however, they have not been evaluated using a more rigorous randomized controlled or pre–post design (Covassin, Elbin, & Sarmiento, 2012; Sarmiento, Mitchko, Klein, & Wong, 2010). Among youth sport coaches, viewing a theory-driven interactive online module was associated with improved knowledge and safety-supportive behavioral intentions (Glang, Koester, Beaver, Clay, & McLaughlin, 2010); however, this module has not been broadly disseminated or updated since its creation over a decade ago. A recent review of 19 different approaches to concussion education for adults found that only two (an informational card and an interactive quiz) led to statistically significant changes in knowledge (Cusimano et al., 2018), with the authors pointing to the importance of population specific tailoring of concussion education resources (Cusimano et al., 2018). Another review of concussion education concluded that improvements are needed in the design and evaluation of concussion education for coaches, given the paucity of resources with theory-driven design or rigorous evaluation (Mrazik et al., 2015).
The focus of the present study is on concussion education for college coaches. Improved coach education about concussion has been identified as a priority by college coaches themselves (Kroshus et al., 2016), and there is not as yet any evidence about the efficacy of different approaches to concussion education in this population. A study of coaches working in all three National Collegiate Athletic Association (NCAA) divisions of competition (n = 1,818) found that one of the most frequent ways that coaches have received information about concussion is through informational handouts (Kroshus et al., 2016). While handouts, typically provided in hardcopy or downloaded online, have been identified as having inherent limitations as an educational modality (Wilson et al., 2012), they have the benefit of being quick for coaches to review and feasibly disseminated across a variety of settings and institutional resource levels. With these considerations in mind, the NCAA (2017b) updated their educational materials for coaches in 2017 and published the informational handout “Concussion safety: What coaches need to know.” One aim of the update was to ensure content reflected emerging scientific information related to both the pathophysiology and natural history of concussion and the possible consequences of continued play with a concussion. The other primary goal was to ground the communication strategy in psychosocial theory about determinants of college coach behavior related to concussion safety, as informed by guidance about the need for theory-driven concussion education resources (Mrazik et al., 2015) that are tailored for the learning needs of the target population (Cusimano et al., 2018). Learning needs are directly related to the target behaviors that are appropriate for the learner. Target behaviors of college coaches in the United States may differ from those of coaches in other settings or at different levels of competition given NCAA rules related to concussion safety and the role of the coach, and the presence of medical staff at athletic activities. For example, in sport settings that do not have medical staff present, coaches may be relied on more to themselves notice and remove injured athletes from play.
The development of the NCAA Fact Sheet for Coaches was grounded in the Knowledge-to-Action (K2A) cycle for knowledge translation and implementation (Graham et al., 2006; Provvidenza et al., 2013). Consistent with this framework, context-specific target behaviors were identified and relevant knowledge for performing those behaviors synthesized. The NCAA Fact Sheet for Coaches attempts to target three material coach behaviors (1) talking to athletes about concussion safety; (2) removing athletes with a suspected concussion from play so they can be medically evaluated; and (3) deferring to medical personnel during the recovery and return to play process. Factual information necessary to perform these target behaviors was generated with input from members of the NCAA’s Concussion Task Force. For example, it was identified that in order for a coach to understand and support the removal of an athlete with a suspected concussion, the coach should be educated on (1) the primary signs and symptoms of concussion; (2) the basic mechanism of concussion injury; (3) the possible consequences of continued play while symptomatic; and (4) NCAA and institutional rules and guidelines related to removal from play postconcussion. A formative survey of college coach knowledge about concussion (Kroshus et al., 2016) was also reviewed to determine learning needs.
The K2A cycle specifies that barriers to knowledge acquisition and implementation should be explored and interventions tailored where possible to address those barriers (Graham et al., 2006). Prior surveys of college coaches have found lower likelihood of engaging in safety supportive behaviors among those coaches who (1) were less concerned about the potential threat of concussion (Kroshus, Baugh, Hawrilenko, & Daneshvar, 2015) and (2) believed there were more negative consequences of engaging in concussion safety behaviors (Kroshus, Kerr, DeFreese, & Parsons, 2016). Thus, a goal of the fact sheet was to make sure coaches were aware of the potential threat of concussions, in terms of the potential short- and long-term harms of the injury. This led to the selection of the extended parallel process model (EPPM; Basil & Witte, 2012; Witte, 1994) as an additional guide to content development. EPPM proposes that if the threat of concussions is perceived to be low then individuals are likely to ignore the educational messaging (Lewis, Watson, & White, 2013). Conditional to threat perceptions being high enough to warrant attention, action is a function of the coach’s efficacy beliefs. Such beliefs reflect the extent to which the coach believes that performing a defined behavior will reduce the threat (response efficacy), and their confidence in their ability to perform that behavior (self-efficacy). High threat and low efficacy often results in message rejection to control fear, whereas high threat and high efficacy often result in message acceptance and efforts to control the threat. Consistent with this framework, the NCAA Fact Sheet for Coaches was designed to not only provide (1) information about the potential short- and long-term consequences of concussion but also information about (2) the role that coaches can engage in to impact their athletes’ safety and why these behaviors matter (to better inform appraisals of response efficacy), and (3) tips for implementing these behaviors (to better inform appraisals of self-efficacy).
The content and visual presentation of the NCAA Fact Sheet for Coaches was finalized after multiple rounds of review by coaches, health education professionals, and members of the NCAA Concussion Task Force, consistent with K2A recommendations that knowledge users be involved iteratively throughout intervention development (Graham et al., 2006). The goal of the present study was to evaluate the potential impact of the NCAA Fact Sheet for Coaches on concussion knowledge and safety supportive behavioral intentions.
Method
Sample and Procedure
Head coaches of all 49 NCAA sports (n = 14,255) across all three NCAA divisions (I, II, and III) were sent an e-mail inviting them to view the NCAA’s updated concussion fact sheet and answer questions about concussions. Content was broken into four blocks. All consenting participants were asked to complete the first block, which included demographic questions and prior year behaviors. Participating coaches were then randomized to one of two conditions—the “intervention condition” or the “control condition”—each of which was distinguished by the ordering of the next two blocks. Randomization occurred using the embedded randomization tool in the Qualtrics survey platform. Coaches randomized to the intervention condition viewed the fact sheet and then answered knowledge and behavioral intention questions. Coaches assigned to the control condition answered knowledge and behavioral intention questions and then viewed the fact sheet. All coaches then completed a final block containing questions about fact sheet acceptability and perceived utility. Coaches were sent three reminder emails over a 2-week period. All study activities were approved by the NCAA Research Review Board.
Measures
Measures were specifically developed to evaluate impact of fact sheet content on knowledge and target behavioral intention outcomes. Survey questions were pilot tested in a sample of 50 coaches to examine psychometric properties of the measures and to obtain feedback on item wording. Knowledge questions where more than 90% of coaches answered correctly were removed from the final survey because they were determined to be ineffective discriminators of concussion knowledge. For example, the true/false statements “Concussions are a type of traumatic brain injury” and “Athletes who keep playing with symptoms of a concussion take longer to recover” were removed because 100% and 97% of participants, respectively, in pilot testing selected the correct response.
Knowledge
Participants answered 10 questions covering a subset of the more challenging facts presented in the NCAA concussion handout. Domains included possible concussion symptoms, consequences of continued play with concussion symptoms, moderators of injury recovery, and potential long-term consequences of concussion. Response options were true (1) or false (0), and correct responses were tabulated to create an index with a possible range of 0 to 10.
Behavioral Intentions
Participants indicated how likely they are to engage in each of the three listed behaviors during the upcoming year: (1) talk to their team about concussion safety; (2) allow an athlete with a suspected concussion to continue participating in a game or practice; and (3) encourage medical staff to return an athlete to play as soon as possible after a concussion. Response options were not at all likely, likely, and very likely. A three-item scale was provided to allow for a socially acceptable ambivalent response in “likely.” Responses subsequently dichotomized into a binary variable (very likely = 1, not at all likely or likely = 0).
Prior Year Behavior
Participants indicated whether they engaged in each of the three listed behaviors during the previous year. Responses were yes (1) or no (0).
Acceptability
Participants answered five questions about their experience with the handout. Responses were elicited on a 4-point Likert-type scale ranging from strongly disagree (1) to strongly agree (4).
Desired Information
For eight different concussion-related topics (e.g., What to do if you think an athlete has a concussion) participants indicated whether they would like additional information (yes = 1, no = 0).
Demographic Characteristics
Participants indicated their age, sex, the sport, and sex of the team they coach and their team’s division of competition. Sport was subsequently dichotomized by whether the sport involved contact and/or collision (yes, no; NCAA, 2014).
Time Viewing Fact Sheet
Timer functionality was embedded in the fact sheet page to approximate the length of time participants spent reading the NCAA Fact Sheet for Coaches. The timer started when they clicked to the page displaying the materials, and it was ended when they clicked to the next page of the survey.
Analysis
First, baseline demographic characteristics and prior season behavior were compared between participants in the intervention and control conditions to check on randomization. Next, two-sample t tests were used to compare by intervention condition mean knowledge index score, and the percentage of participants who indicated an intention to engage in each of the three target behaviors. Effect size thresholds of 0.2, 0.5, and 0.8 were used to denote small, medium, and large effect sizes, respectively (Cohen, 1988). Item-level knowledge responses were subsequently explored using post hoc item-level comparisons. Bivariate logistic regression analyses were used to assess the association between demographic factors, concussion knowledge, and odds of coaches engaging in the three target behaviors. Demographic factors and knowledge were subsequently included, along with intervention condition, in multivariable logistic regression analyses predicting odds that a participant would engage in the three target behaviors. Descriptive statistics were reported about the acceptability and perceived utility of the NCAA Fact Sheet for Coaches, as well as topics about which participants would like additional information.
Results
A total of 820 coaches responded to at least one survey question, of which 779 completed the survey through the randomization phase, a 5.75% response rate among coaches who were invited to participate. More than three quarters of participants coached a contact/collision sport (76.99%, n = 599). Additional demographic characteristics are reported in Table 1. There were no statistically significant differences in demographic characteristics between participants randomized to the intervention condition (n = 397) as compared with the control condition (n = 382). The mean length of time that participants in the intervention condition spent viewing the fact sheet was 4 minutes and 48 seconds. 89.99% of participants (n = 656) reported that they had talked with their team about concussion safety in the prior year. A small number of participants (1.86%, n = 13) reported knowingly allowing an athlete with a suspected concussion to continue play in a game or practice during the prior year. Thirteen percent (13.49%, n = 92) of participants reported that they had encouraged medical staff to return an athlete to play as soon as possible after a concussion during the prior year.
Demographic Characteristics and Prior Year Coaching Behaviors.
Participants in the intervention condition had knowledge scores that were on average 0.5/10 higher than coaches in the control condition (p < .001), representing a moderate effect size (Cohen’s d = 0.33; Table 2). Post hoc item level comparisons indicated that this difference was largely driven by significant differences in knowledge about the consequences of continued play after a concussion (injury, reaction time, possible death). There were differences between intervention and control condition in intentions about two out of the three target behaviors. Fewer coaches in the intervention condition (83%) as compared to the control condition (89%) expected that they would allow an athlete with a suspected concussion to continue participating in a game or practice (p = .031, Cohen’s d = 0.16). Similarly, fewer coaches in the intervention condition (69%) as compared with the control condition (77%) expected that they would encourage medical staff to return an athlete to play as quickly as possible after a concussion (p = .021, Cohen’s d = 0.17). There was no difference by intervention condition in coach expectations about talking to their team about concussion safety.
Differences in Primary Outcomes by Intervention Condition.
Note. T = true; F = false; RTP = return to play.
Knowledge sum reported as number of questions answered correctly (possible range of 0-10), with standard deviation reported in parentheses; for individual knowledge items, the proportion of coaches who selected the correct answer is reported; for expected behaviors next year the mean score is reported with standard deviation in parentheses.
Two-sample t test for differences in mean values between conditions.
Subsequently, logistic regression analyses were conducted to assess whether demographic and cognitive factors independently explained behavioral intentions. Coaches of contact/collision sports were more likely to talk to their team about concussion safety than were coaches of other sports (odds ratio [OR] = 2.25, 95% confidence interval [CI] [1.40, 3.62], p < .001; Table 3). Allowing an athlete with a suspected concussion to continue participating in a game or practice was significantly associated with intervention condition and team sex in a multivariable model (Table 3). Male coaches of female teams (OR = 0.50, 95% CI [0.29, 0.93], p = .029) and female coaches of female teams (OR = 0.46, 95% CI [0.24, 0.88], p = .019) were significantly less likely than male coaches of male teams to expect that they would allow their athletes to continue participating in a game or practice after a concussion. No factors beyond intervention condition were significantly associated with likelihood of intending to encourage medical staff to return athletes to play as soon as possible after a concussion. Logistic regression results are reported in detail in Table 3.
Results of Bivariate and Multivariate Logistic Regression Models Predicting Three Expected Behaviors Next Season.
Participant perceptions about the NCAA Fact Sheet for Coaches were largely positive (Table 4). The majority of participants indicated that they learned something new (74.04%) and would recommend it to other coaches (95.60%). Despite positive feedback about the fact sheets, participants still reported wanting more information about concussion across all topics listed (Table 5).
Coach Perceptions About the NCAA Concussion Fact Sheet.
Note. NCAA = National Collegiate Athletic Association.
Additional Information About Concussion Desired by Coaches.
Note. NCAA = National Collegiate Athletic Association.
Discussion
Participants who reviewed the NCAA Fact Sheet for Coaches prior to the knowledge assessment demonstrated more knowledge about concussion and more safety-supportive behavioral intentions than coaches who were randomized to the control condition. However, although the differences between conditions were statistically significant, effect sizes were small. This raises questions about the extent to which viewing the fact sheet will result in practically meaningful or sustainable behavior change.
Notably, a majority of coaches said they would be very likely to allow an athlete with a concussion to finish a game or practice or to encourage medical staff to return an athlete to play postconcussion as quickly as possible. Although responses were statistically significantly safer in the intervention condition, across both conditions they still largely represent unsafe intentions. However, when asked about their prior year behavior, very small minority of coaches indicated they allowed continued play (<1%) or encouraged return to play (<15%). Further research is warranted to understand the reasons for this discrepancy. It might reflect coaches not having been in a situation where they had agency over an athlete’s continued play postconcussion during the prior year—either because an athlete was not to their knowledge concussed, or because medical staff managed the injury with limited interactions with coaching staff.
It is possible that the small effect sizes observed for knowledge and intention comparisons between intervention and control condition coaches were attributable to the modality and duration of the intervention. Reading a static handout for on average fewer than 5 minutes likely has constrained potential for changing cognitions. Further research is needed to explore whether delivering the content from this handout through more active learning modalities, or modalities that involve a longer duration of exposure to information, differentially impact knowledge and behavioral intentions. Online approaches to education for coaches may hold potential for more expansive education as they would allow for individual customization based on topics of interest and sport coached. That said, participants reported a very high rate of acceptability and perceived utility with regard to the reviewed materials, and informational handouts have the advantage of being relatively inexpensive to produce and update and are feasible to disseminate. Furthermore, a prior review of 19 different approaches to concussion education for adult sport stakeholders did not find consistent evidence about online or more intensive approaches to education being more effective (Cusimano et al., 2018). Thus, the results of this study should provide the NCAA with some confidence that the educational content it is providing to coaches through this modality is acceptable and useful, however, further research exploring the acceptability and effectiveness of different educational modalities should also be undertaken.
Differences in demonstrated levels of knowledge by intervention condition appeared to be largely driven by differences in the responses to questions about consequences of continued play with a concussion. The NCAA Fact Sheet for Coaches attempted to address this topic more directly than concussion education materials previously directed at coaches, incorporating new information about reaction time and injury duration (e.g., Asken et al., 2016; Eckner, Kutcher, Broglio & Richardson, 2014). Consistent with EPPM, increased knowledge about the consequences of continued play while symptomatic can affect perceptions of the threat of the injury, which may increase message acceptance (Lewis et al., 2013; Watson, & White, 2013). They can also help make the target behaviors (e.g., supporting appropriate removal of athletes from play) relevant to the coach’s short-term occupational goals. Such personal relevance of education messaging has been identified as an important determinant of message acceptance (Cauberghe, De Pelsmacker, Janssens, & Dens, 2009; Merriam, 2001). Further study is needed to assess the extent to which the theorized mediating cognitions from EPPM (e.g., response efficacy, self-efficacy), are responsible for shifting behavioral intentions and behavior.
Even given the positive feedback from coaches about the acceptability and utility of the NCAA Fact Sheet for Coaches, coaches still wanted more information across a range of topics (Table 5). For example, 82% of coaches wanted more information about NCAA rules and guidelines. This might reflect lack of within-institution dissemination of NCAA guidance or lack of communication about what NCAA guidance means for their responsibilities as a coach. Further research is needed to understand the type of learning support that coaches would like about the topics listed in Table 5, and how this information could be best shared with coaches.
The K2A model is intended to be cyclical and iterative (Graham et al., 2006; Provvidenza et al., 2013); the present findings can help inform future iterations of the NCAA’s approach to concussion education for coaches. The NCAA is encouraged to bring the findings of the present evaluation back to coaches to help identify potential modifications to the NCAA Fact Sheet for Coaches. This may include both augmented content, and new approaches to content delivery to increase duration of coach engagement with the topic.
Limitations
The primary limitation of this study was that data were collected in a short window of time between fact sheet completion and their public dissemination, meaning that long-term follow-up was not possible logistically. Changes observed in behavioral intentions in the present study should not be extrapolated to mean there would be a commensurate change in coach behavior as there are many exogenous factors that could influence coach behavior (Sheeran & Webb, 2016). Observing a difference in these proximal cognitions immediately after viewing the fact sheet is a necessary first step in assessing fact sheet efficacy; however, it is possible that “boosters” or “reminders” of fact sheet content may be required throughout the season (Adrian & Lyon, 2016). Another limitation of the present study is that the evaluation assessed the efficacy of the fact sheets in a highly controlled setting, wherein the coaches were required to view the fact sheet online to answer survey questions. It is possible that coaches will be provided the fact sheet in a variety of modalities (e-mail, hard copy handout, sheet posted on wall) and these differences in delivery may impact effectiveness. Finally, only a small fraction of NCAA head coaches who received the initial email invitation chose to participate in the study. Those who chose to participate are likely more concerned about concussion, and potentially more knowledgeable about concussion, than those who chose not to participate, which may have biased the intervention effect toward the null and may mean that these findings are not generalizable to all coaches. Inviting all NCAA coaches to view the fact sheet and provide feedback about its utility was a decision that necessarily limited the potential for incentivized participation. However, this strategy was selected so as to allow all coaches to have a voice in the iterative adaptation of NCAA concussion education for coaches, consistent with the K2A cycle (Graham et al., 2006). Future research on this topic is encouraged to use designs that allow for a higher coach response rate, potentially collecting data in person at coaching conferences. Finally, it is possible that—despite the anonymous nature of the survey—coaches responded in what they believed was a socially desirable manner. Future research with coaches about health and safety topics should include a measure of socially desirable responding.
Implications for Practice
The present study provides support for the short-term efficacy and acceptability of the NCAA’s concussion fact sheet for coaches, while also indicating areas of continued coach need. Coaches should be engaged in interpreting the present findings and in suggesting changes to the fact sheet, including determining whether a different educational modality would be more appropriate. Organizations seeking to develop their own concussion education for their coaches consider building from the NCAA’s content, adapting it as necessary for (1) the target behaviors that they believe are appropriate for their coaches and (2) unique learning needs of coaches in their setting (e.g., organization-specific rules or guidelines about concussion identification and removal from play and/or the role of the coach). As was done in the present fact sheet, organizations should consider providing coaches with information about the range of consequences of continued play with a concussion in clear language that does not obscure or minimize the potential risks to their athletes.
Footnotes
Declaration of Conflicting Interests
The authors declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article. EK provides consulting services to the NCAA related to health education program development and evaluation. DB, JTP, and BH are employees of the NCAA.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was supported by the NCAA Sport Science Institute.
