Abstract
Keywords
Introduction
Individuals who sustain a traumatic brain injury (TBI) often have difficulties in returning to work post injury or obtaining and maintaining competitive employment. Vocational rehabilitation (VR) through the use of the federal and state vocational rehabilitation programs is one option for obtaining the needed supports that lead to successful competitive employment outcomes. Rehabilitation services following TBI focus on assisting individuals to adjust to their on-going impairments and re-enter their communities including the workplace. Some of the specific approaches that may be used include, but are not limited to, Employee Disability Management Programs, transitional programs, supported employment, job accommodations, and assistive technology. Graham and her colleagues conducted a systematic review with one of the goals to inform VRcounselors regarding the effectiveness of these rehabilitation employment interventions (Graham, West, Bourdon, & Inge, 2016). The findings from the review are consistent with prior TBI return to work studies that emphasize the importance of early intervention following TBI and in vivo training and support in the natural work setting such as those found in the supported employment model with civilianpopulations.
While research is available regarding evidence-based practices (EBP) for facilitating employment outcomes for individuals with TBI, one question is how to best deliver this information to practitioners? There is a growing recognition within the VR field of the value of incorporating EBP into counseling, practice, and education (Anderson, Matthews, Lui, Nierenhausen, & Lawler, 2014; Chan et al., 2010; Shaw, Leahy, Chan, & Catalano, 2006). This process requires adapting new knowledge gained from research to fit the contexts of various users or stakeholders, such as counselors, clients, employers, policy makers, or others (Lane & Flagg, 2010). However, there is little empirical research that guidance as to this adaptation process for VR practitioners.
Graham and her colleagues (2013) surveyed 355 VR staff in three states representing the South, Southwest, and Mid-Atlantic regions to identify knowledge translation strategies that counselors report as likely to promote their use of research. As part of this research, respondents were asked what they believe constitutes “evidence” in evidence-based practice. The three most commonly cited definition components, i.e., “Research-based,” “Documented Evidence,” and “Proven Effective,” were used by over 87% of respondents. Moreover, over half indicated comfort in being able to locate, read, and comprehend research findings. Far fewer respondents indicated that they actually used research findings and EBP in performance of their duties with clients, such as developing Individual Plans for Employment (IPEs). In addition, many reported that they had too little time to search and review the research on VR practices and they found academic research difficult to translate into effective practices. This information identifies the need to present evidence-based practices to practitioners using clear and concise resource documents.
Findings from this study also indicated that the primary modes for obtaining research information were nonacademic and informal, such as workshops, in-service training, online web courses and webcasts, and communications with other professionals, consumers, family members, and others (Graham et al., 2013). In addition, inadequate time was reported as a factor in the use of EBP, which indicates that resources must be easily accessible and provide “sound bites” of documented evidence that can be used on the job. Consideration regarding length of time needed to devote to the workshop, in-service training, or online distance education options also must be addressed based on the finding that VR counselors have limited time to access EBP information. Based on these findings, the current study was designed to test the effectiveness of a series of short, nontechnical recorded presentations, delivered by a trusted “coworker” peer to impact the use of an evidence-based practice, supported employment,for VR clients with TBI who wish to return to work. Using a pretest-posttest random group design, the study assessed the efficacy of two specific KT strategies by delivering the same information related to return to work following traumatic brain injury(TBI). The research questions to be addressed include the following: Are a series of brief, nonacademic online presentations on employment of individuals with TBI more effective in producing changes in the awareness and knowledge among VR professionals than a series of nontechnical briefs? What is the relationship between VR professionals’ characteristics and the effectiveness of the KT strategy to impact their evidence-based employment knowledge and use?
Method
Recruitment procedures
Participants were recruited from one state VR agency in the Mid-Atlantic States with the assistance of the agency’s Program Manager for TBI and the Director of the Division of Rehabilitative Services (DRS). The Director sent an email to the agency’s staff that included VR counselors throughout the state describing the study and encouraging the counselors to participate in training on employment for individuals with TBI. A link was included in the email to an online consent form with more details about the study including the fact that participants would be randomly assigned to one of two groups to receive the information. Once a counselor consented to participate, the individual was redirected to the online demographic section of the pretest. As part of this process, the individual submitted his or her name and contact information, which was stored in a contact database separate from the pre-test responses. Only the study personnel had access to the password protected database for contact information and the pre- and posttest databases. Identifying data were stored separately from test responses to ensure total confidentiality of the participants. In addition, the names of the participating counselors were not shared with the VR agency. Three follow-up reminders concerning the study were emailed and yielded 71 individuals who had consented to participate in the study. Consenting individuals were then randomly assigned to one of the two groups for the study.
As an incentive to participate, counselors were told that they would be able to take an online continuing education course free of charge after the research study was completed. The course offered was Ethical Issues and Decision Making for Rehabilitation Professionals, which is one of the required topics by The Commission on Rehabilitation Counselor Certification (CRCC) for continuing education. Certified rehabilitation counselors (CRC) are required to earn 10 hours of continuing education in ethics during a five-year period to maintain their certification. Forty of the 71 participants registered for this online course at the end of the study. Thirty-two earned CRCs for completing all of the course requirements for a completion rate of 80%. Of this number, 17 individuals (53%) were in the brief group and 15 (47%) were in the YouTube group.
Table 1 provides a summary of characteristics of consenting participants, including position title, gender, and educational level. Chi-square analyses found no significant differences in demographic characteristics between the two groups. As can be seen from this table, the majority of participants in both groups were females in Rehabilitation Counselor positions with Master’s Degrees.
Intervention and comparison conditions
As previously mentioned, this study was guided by the work of Graham and her colleagues (Graham et al., 2013). These findings indicated that counselors prefer to obtain information on evidence-based practices from trusted peers that is easily accessible as well as provided in “sound bites” of documented evidence. Counselors also indicated that in addition to more traditional face-to-face training or workshops, online courses and training webinars were considered to be an effective means of receiving evidence-based research information.
Based on this information, the knowledge translation (KT) intervention was developed in collaboration with the State VR agency’s Program Manager for TBI who served as the “trusted peer” to deliver the content. The lead researcher discussed the content for the intervention with the program manager and the topical area selected was supported employment for individuals with TBI. This topical area was selected because (1) supported employment is a service option funded by state VR agencies and would therefore likely be of interest to the target group; (2) the findings of a recent systematic review conducted by the study team concluded that supported employment was one of only three identified EBPs for assisting individuals with TBI to return to work (Graham et al., 2016), and therefore the topic would have practical applications to the target group; and (3) TBI is a large and growing segment of the state’s VR agency’scaseload.
The KT strategies selected to deliver the information were 1) a series of three short (15–20 minute) pre-recorded videos and 2) three nontechnical briefs. The videos were produced at the university’s multi-media lab and then archived as unlisted on YouTube. An unlisted video does not show up in a YouTube channel and does not show up in YouTube’s search results. A professional captioning service transcribed the videos that were then used to produce the briefs. These transcripts were edited for grammar but contained the same content as the pre-recorded videos. This was done to ensure that both groups received the same information varying only in the formats to be tested. In addition, the transcripts were uploaded into YouTube to provide captioning for accessibility. Table 2 provides a summary of the content used for the YouTube videos and the briefs.
Procedure
The research was reviewed and approved by the Institutional Review Boards for SEDL, the organization that received funding for the research, the lead author’s University, and the participating State VR agency. At the end of the recruitment 71 participants consented to participate, and they were sent a follow-up email with the link to the pre-test knowledge questions. These participants were randomly assigned to either the YouTube video group or the brief group. The intervention spanned a three-month period of time and began after all the consenting participants completed the pretest. The link to one YouTube video or one brief was sent to the participants via their agency email addresses at the beginning of each of the three months depending on group assignment. The participants could view the video or read the brief at their convenience during the one month period. At the end of the three months, the participants were asked to complete the posttest. Of the 35 participants randomly assigned to the YouTube group, 31 completed the pre- and the posttest. Of the 36 participants in the brief group, 33 completed the pre- and the posttest. In total, 64 participants completed both the pre- and posttests.
Measure
The measure contained a brief 10-item knowledge assessment, an 11-item list of services that could be funded for individuals with TBI, and a list of perceived barriers to employment for individuals with TBI. The knowledge assessment included items that covered supported employment evidence-based practices for clients with TBI with one best answer for each question. The “willingness to fund list of services” concerned the likelihood of VR professional to pay for supported employment and other types of VR services for individuals with TBI. The Willingness to Fund scale used a Likert-type response scale with responses ranging from 1 (very unlikely) to 5 (very likely). The list of perceived barriers for returning to work for individuals with TBI included six items with an option to add two additional items of the respondents’ choice. Participants were able to respond to these items using a Likert type response scale with responses ranging from 1 (strongly disagree) to 5 (strongly agree). Example items from the pretest are presented in Table 3.
Data analysis
The participant’s gender and employment characteristics (experience, education, and job title) were summarized separately for each treatment group. A repeated measures Analysis of Variance (ANOVA) was used to determine within and between group knowledge differences between the YouTube and brief groups. Other repeated measures ANOVA models were used to assess the perceived likelihood of funding, and barriers to employment. Secondary analyses were performed that adjusted the aforementioned analyses for the participant’s gender and employment characteristics. Each of these models uses a sandwich estimator to provide robust variance estimates. Chi-square goodness of fit tests were performed to determine differences in the amount of participants’ willingness to fund employment services and barriers to employment by group and time (baseline vs. post-intervention). Chi-square goodness of fit test was used to determine the difference in Status 26 Closures by group and time (baseline vs post intervention). All inference was performed at the 0.05 level using SAS V9.4.
Results
Seventy-one participants enrolled in the study and were randomized to the intervention (n = 35) and the comparison group (n = 36). One subject in the YouTube group did not provide any information about his/her knowledge and had nonvarying responses on both the pre- and post-intervention survey, and was excluded from all analyses. A separate subject’s follow-up values for the knowledge assessment were excluded since all but two items were missing. The intervention and comparison groups were similarly distributed with respect to demographic and position characteristics at baseline. Two subjects randomized to the brief group and three participants in the YouTube group were lost to follow-up, although their baseline information was used in the primary and secondary analyses.
Knowledge tests
Over both groups, there was a statistical significant finding between pre- and post intervention know-ledge scores [F(1, 61) = 20.05, p < 0.000, 95CI = 0.964, 2.465] with the mean baseline knowledge score of 12.81 (SD = 2.83) and the mean post intervention score of 14.52 (SD = 2.61) with a moderate standardized effect size (d) of 0.63. Whenexamining group differences between the YouTube and brief groups, no significant differences were found [F(1, 61) = 0.001, p < 0.981, 95CI = 0.74, 2.14]. Both groups had similar mean scores at baseline (YouTube M = 12.41, SD = 2.52; Brief M = 13.11, SD = 3.04) and post intervention (YouTube M = 14.11, SD = 2.98; Brief M = 14.83, SD = 2.30). There were no statistical differences by gender or employment characteristics.
Service funding and employment barriers
The likelihood of providing funding for the employment activities for people with TBI remained similar at baseline and post intervention. Both groups had similar mean scores at baseline (YouTube M = 31.66, SD = 6.24; Brief M = 31.69, SD = 4.12) and at post intervention (YouTube M = 31.50, SD = 4.34; Brief M = 31.49, SD = 3.80). No statistically significant differences were found between the YouTube and the brief group at baseline and post intervention for the likelihood of providing funding for employment activities for people with TBI. There were no statistical differences by gender or employment characteristics F(1, 64) = 0.1, p = 0.976, 95% CI = [–2.0, 1.7].
While differences between the two groups were minimal, some patterns emerged from a review of responses to individual items. Table 4 provides the percentages with which participants indicate that they would be either “Likely” or “Very Likely” to purchase various services for their clients with TBI at baseline and at post intervention. This table shows that the majority of responses were consistent between baseline and follow-up and between groups. However, YouTube participants reported a decline in their likelihood of funding of Employment Development Services (EDS) for clients with TBI while this rate increased substantially for those in the brief group, x2 = 5.43, p = 0.019. EDS is a peer support service for VR clients with TBI that is based on the Clubhouse Model. Although not statistically significant, there was an increased in likelihood of funding for evaluation by a Community Rehabilitation Program (CRP) was also evidenced by the YouTube group, x2 = 3.596, p = 0.056.
Similar non-significant findings were found for barriers to employment for individuals with TBI. Although not significant, participants in the YouTube group were less likely to identify barriers to employment for individuals with TBI than the brief group F(1, 64) = 0.1, p = 0.812 95% CI = [: –0.1, 2.6]. See Table 5 for baseline and post intervention mean scores.
With regard to employment barriers for clients with TBI, both groups showed increases in the perception that negative employer attitudes and limited funding of supported employment posed substantial barriers. However, perception of limited availability of supported employment funding as a barrier declined (63% to 55%) for the YouTube group [x2 = 11.11, p = 0.0009] and increased (33% to 56%) within the comparison group [x2 = 7.11, p = 0.0077]. Table 6 provides the percentages with which participants endorsed either “Agree” or “Strongly Agree” to barriers to employment by group and tt baseline and at post intervention.
Status 26
One year post intervention, a follow-up was conducted with the participants who completed both the pre- and posttests for the study. An email was sent to all participants asking them to report how many individuals with TBI had been closed in Status 26, successful employment, the year previous to the study as well as the number that they had closed in Status 26 the year after the study. The email sent a link to an online consent screen that redirected the participants to the two questions if they consented to provide the information. Of the 64 VR counselors who completed the study, 29 individuals provided the information of which 11 respondents were in the YouTube group and 18 were in the brief group.
Although not statistically significant, the brief group (n = 23 Status 26 closures) had more Status 26 closures at the post follow-up than the YouTube group (n = 17 Status 26 closures), [x2 = 0.90, p = 0.343). The only statistically significant difference in the number of Status 26 closures between baseline and post intervention was for the brief group [x2 = 4.24, p = 0.040]. The number of Status 26 closures more than doubled from baseline with 11 Status 26 closures to post intervention with 23 Status 26 closures. See Table 7 for frequency of Status 26 closures by group and time (baseline vs post intervention).
Discussion
Moving evidence-based research into practice is essential to improving the employment outcomes of individuals with disabilities. One purpose of this research was to determine if a series of brief, nonacademic YouTube presentations on employment of individuals with TBI is a more effective KT strategy for producing changes in the knowledge and use of evidence-based research among VR professionals than a series of nontechnical briefs. Information on the employment outcomes of individuals with TBI exiting the VR system can be obtained by reviewing the RSA-911 data. This database contains information for each individual whose case is closed by a VR Agency throughout the United States and its territories. The RSA-911 database is within the public domain and the data for FY 2011, 2012, and 2013 for individuals closed during this time period with TBI was provided in SPSS format to the researchers. Over the three-year period, 19,497 VR participants with TBI had their cases closed by the VR agencies in the U.S. with 6,631 in FY 2011, 6,259 in FY 2012; and 6,507 in FY 2013.
There are a number of reasons why an individual’s case is closed by VR. One of the closure codes is Status 26: successful rehabilitation, employment outcome achieved. Of the 6,631 individuals closed in 2011 by VR, 1,798 were closed as having successfully achieved an employment outcome or 27% of the individuals exiting services. In 2012 the number was 1,718 (n = 27%) and in 2013 the number was 1,789 (n = 28%). Across all three years, slightly less than one-third of the individuals exiting the VR system left successfully with an employment outcome. While there are many reasons for why individuals exited unsuccessfully, the question remains as to whether these outcomes can be impacted by providing evidence-based information using various KT strategies with VR counselors who support theseindividuals.
This research found that there was a statistical significance between the pre and post intervention knowledge scores over both groups. In other words, both of the KT strategies, pre-recorded YouTube presentations and nontechnical briefs, increased participants’ knowledge of employment issues related to TBI. When examining difference by group, no differences were found in knowledge gain indicating that one KT strategy was not more effective than the other in increasing knowledge of the participants. Although not statistically significant, the brief group had more Status 26 closures at the post follow-up than the YouTube group. The only statistical significant difference in the number of Status 26 closures between baseline and post intervention was for the brief group with the brief group having more Status 26 closures at post follow-up than prior to the study. This finding may support the hypothesis that providing evidence-based information using nontechnical briefs to VR counselors can impact the employment outcomes for individuals with TBI. These findings should be interpreted with caution due to the small response rate for the follow-up data collection, 45% (n = 29) for all the participants in the study.
The study was designed to compare the effects of two KT strategies on VR professional staff on attitudes and future behavior. In theory, the content of the two strategies is not as relevant to the study as is the means of delivering the content. In this study, we selected supported employment for individuals with TBI as the content area for assessing the two KT methods with sound rationale. Supported employment has been a service option for the VR program under the Rehabilitation Act Amendments for nearly three decades, and has been utilized for individuals with TBI nearly as long (c.f., Wehman et al., 1988; Wehman, Kreutzer, et al., 1989; Wehman, West, et al., 1989). This suggests that participants, as VR professionals, came into the study with prior knowledge and possibly experience in the content area. Ordinarily, this would be a positive situation; however, in a research study such as this, prior knowledge and experience may limit the degree to which knowledge, attitudes, and behaviors can be changed, and thus limit the potential effect of the information being delivered. Perhaps the findings would have been different had the content area been less known to the participants.
Another possible limitation of the study is related to how much content was provided to the participants. Each month for the three-month time period, the participants received a link to one 15–20 minute recorded presentation or a nontechnical brief that included the same information. The total time commitment was approximately 45 minutes. This was intentional based on findings from prior research (Graham et al., 2013), that indicated VR counselors preferred to have “sound bites” of documented evidence. In essence, this study offered the participants three “sound bites” of documented evidence. Future research regarding KT strategies may be able to show greater effect by testing interventions that occur at a greater frequency while still presenting the information in manageable chunks.
Last, preferences and attitudes concerning the mode in which knowledge was transmitted was not considered in that participants were randomly assigned to groups. Recent knowledge translation research using personas suggests that preference and attitudes towards mode of transmission may contribute to learning information (Cole et al., 2015). The Centers for Disease Control (CDC) and the Institute of Cancer have begun developing personas to obtain buy-in by people to engage in healthier behaviors (Cole et al., 2015). A persona is a composite description of a group of people using common characteristics that include preferences, beliefs, attitudes, and behaviors and is used to influence change (Boutros & Purdie, 2014; Hendriks & Peelen, 2013; LeRouge, Ma, Sneha, & Tolle, 2013; Miaskiewicz &Kozar, 2011; O’Connor, 2011; Serio, Hessing, Reed, Hess, & Reis, 2015; Turner, Reeder, & Ramey, 2013). The knowledge transmission is tailored to each persona. In this way, rehabilitation and disability researchers can tailor presentations of knowledge with the intent to emotionally “hook” people in each persona into engaging in particular behaviors (Hendriks & Peelen, 2013; LeRouge et al., 2013; Phillips, Boswell, Boomer, Kwon, & Currie, 2015; Serio et al., 2015; Turner et al., 2013). In this case, it would be developing personas and tailoring knowledge transmission of information to VR counselors. Further research using this technique may be helpful to increase of VR counselor’s knowledge.
Additional research is needed to determine if knowledge and use of evidence-based practices can impact the outcomes of individuals with TBI. Another study limitation was that the research only included counselors from one state. VR agencies have varying priorities with regard to professional development, counselor specialization, order of selection, and other areas. Further research on KT strategies should attempt to engage multiple state VR agencies and to consider state agencies’ existing priorities and knowledge base. Further research with a larger number of VR practitioners for a longer period of time is required before effectiveness of either KT strategy can be determined.
Conflict of interest
The authors have no conflict of interest to report.
Footnotes
Acknowledgments
This study was supported by the Center on Knowledge Translation for Employment Research through grant 90DP0009 (formerly ED #H133A100026) to SEDL, an affiliate of American Institutes for Research, from the National Institute on Disability, Independent Living, and Rehabilitation Research (NIDILRR) in the U.S. Department of Health and Human Services (HHS), Administration for Community Living. However, the contents of this article do not necessarily represent the policy of the U.S. Department of HHS, and you should not assume endorsement by the Federal Government.
The authors express their appreciation to the Virginia Department for Aging and Rehabilitative Services (DARS) for assistance with this study. Individual recognition goes to Ms. Patricia Goodall, Program Manager, Brain Injury Services Coordination Unit, and Ms. Kathy Hayfield, Director of the Division for Rehabilitative Services, for their support and contributions. Finally, we thank the DARS’ vocational counselors who participated in the study, without which this study would not have been possible.
