Abstract
Relationship abuse (RA) and sexual assault (SA) are concerning issues for high school youth that occur at alarmingly high rates. Therefore, school-based prevention programs are often developed to try to mitigate these issues. The attitudinal and behavioral changes stemming from school-based intervention programs are often the primary focus of outcome research, but it is also important to identify program feasibility (e.g., dosage, adherence) and acceptability (e.g., likeability). The current study assessed the feasibility and acceptability of a high school classroom-based RA and SA intervention, Bringing in the Bystander–High School Curriculum (BITB-HSC); the BITB-HSC also includes a workshop and reading materials for school personnel and parents. Data were collected after the BITB-HSC was administered in high schools from students who participated in the program (N = 970) and school personnel who were exposed to some components of the BITB-HSC and provided feedback (N = 161). Results indicate that overall students were adherent to the BITB-HSC (e.g., attended sessions) and that the majority of students understood the information presented, thought the program was good, and liked the facilitators. School personnel data indicated that they were generally supportive of the program and found the school personnel workshop and handout helpful. These findings extend current knowledge regarding the implementation of school-based violence prevention programs and highlight the importance of research on program feasibility and acceptability when developing a program for large-scale dissemination.
Relationship abuse (RA; physical, sexual, and psychological aggression that happens between current or former dating partners) and sexual assault (SA; any unwanted sexual behavior ranging from sexual contact to completed rape that can occur between individuals in any type of relationship among high school youth) are major public health concerns among adolescents (Centers for Disease Control and Prevention, 2014). Hence, it is imperative that researchers, practitioners, and educators identify efficacious RA and SA prevention strategies that can be disseminated on a large scale.
In addition to efficacy, a critical component of identifying impactful RA and SA prevention strategies is to understand the feasibility and acceptability of the program. Feasibility describes the degree to which a program will be successful in real-world conditions, and includes aspects such as dosage, adherence, and practicality (Bowen et al., 2009; Durlak & DuPre, 2008). Acceptability describes how participants react to the program and the extent to which they find it acceptable (Bowen et al., 2009; Rounsaville, Carroll, & Onken, 2001).
A number of implementation models highlight the need for assessing feasibility and acceptability as part of implementation process evaluation (Bowen et al., 2009). One such model, proposed by Durlak and DuPre (2008), includes aspects of implementation science such as fidelity to the original program, dosage, quality of program delivery, participant responsiveness, and how well the program reached the audience. Related work on the elaboration likelihood model (Heppner, Humphrey, Hillenbrand-Gunn, & DeBord, 1995) also examines the depth to which participants process program messages as a component of implementation and engagement. These frameworks highlight the need to conduct implementation process evaluations that examine how a new prevention program or intervention strategy is working within the particular context where it is implemented.
Process evaluations also shed light not only on the success or failure of evidence-based practices in real-world settings, but also on the processes that influence the likelihood that practitioners will use them (Durlak & DuPre, 2008; Wandersman et al., 2008). Qualitative data collected as part of process evaluations can provide information that helps unpack findings from outcome evaluation methods (e.g., Edwards, Hunt, Meyers, Grogg, & Jarrett, 2005; Steckler et al., 2003). Moreover, researchers highlight the ways in which prevention work can be strengthened when youth voices are placed more at the center of program development and refinement (Edwards, Jones, Mitchell, Hagler, & Roberts, 2016). In addition, including the perspectives of teachers and school administrators is important because multiple stakeholders must buy-in for a program to be successful (Spoth et al., 2013; Steckler et al., 2003; Wandersman et al., 2008).
There is a dearth of research that has examined feasibility and acceptability of RA and SA prevention initiatives. Feasibility is important because it describes the potential for a program to be implemented and, thus, have large-scale impact (Spoth et al., 2013). Previous research on the feasibility of antiviolence and sexual health promotion programming shows that factors such as stability of involved agencies and other stakeholders, having highly qualified and motivated staff, and having an appropriate setting and adequate time all contribute to feasibility (Mihalic, Irwin, Fagan, Ballard, & Elliott, 2004; Mustanski, Greene, Ryan, & Whitton, 2015). Acceptability is a particularly important concern in SA prevention. Some research on acceptability of SA prevention programs shows that many participants do not feel the programs are relevant to their own lives because they are not sexually aggressive (Rich, Utley, Janke, & Moldoveanu, 2010). Factors that may increase acceptability to participants and facilitators include multiple teaching methods (i.e., including videos, discussions, and games) and providing comprehensive training materials (Edwards et al., 2005).
The purpose of this brief report is to use qualitative data from students and teachers regarding their perspective of a bystander-focused RA and SA prevention initiative. Data were collected as part of an implementation process evaluation that was conducted alongside a cluster randomized control trial outcome evaluation of Bringing in the Bystander–High School Curriculum (BITB-HSC; Edwards, Banyard, et al., forthcoming). The BITB-HSC includes a seven-session, classroom-delivered program that teaches students how to safety and effectively intervene before, during, and after situations of RA and SA to both prevent and stop RA and SA from happening, as well as supporting victims in the aftermath of these experiences. The BITB-HSC is grounded in the health belief model (Rosenstock, 1974), transtheoretical model of change (DiClemente & Prochaska, 1998; Prochaska, DiClemente, & Norcross, 1992), theory of planned behavior (Ajzen, 1991), and diffusion of innovation theory (Rogers, 2003). In addition to student programming, the BITB-HSC includes a 60-min school personnel workshop that teaches school personnel skills to be a positive bystander in situations of teen RA and SA as well as reading material that reinforces the information and skills conveyed in the school personnel workshop. Components believed to enhance feasibility include the fact that BITB-HSC is delivered to students within their natural classroom environment, as well as facilitator training and a detailed program manual including scrips for exactly what to say during each session. To ensure acceptability, the programs are delivered in ways that are intended to be personally meaningful and relevant to the lives of students and school personnel. Moreover, the BITB-HSC seeks to be a highly engaging curriculum that includes small and large group discussions, experiential exercises, skill-building activities, and multimedia presentations.
Our aims were as follows: to explore the feasibility of BITB-HSC, specifically, adherence to program model and program dosage among high school student participants (Aim 1); to explore the acceptability of BITB-HSC among the high school students who participated in the program (Aim 2); and to explore acceptability of the BITB-HSC among teachers (Aim 3).
Method
Students (Aims 1 and 2)
Participants
Participants were students (N = 970) from 12 treatment schools in upper New England who completed the immediate program posttest and answered at least one of the open-ended questions. The mean age of participants was 15.69 years (range = 13-19 years, SD = 1.16 years), and were in grades 9th to 12th. Half of the students were female (53.3%). The majority of participants identified as White (86.9%) and heterosexual (87.2%).
Procedure
Passive parental consent procedures were used for students below 18 years of age. Participants completed an assent form and paper and pencil surveys at school; the surveys were administered by members of the research team. Students received a fruit snack.
Measures
Researcher-created questions asked students which modules they attended, whether they gave their parent/guardian the fact sheet as instructed, what they liked most and least about the program, whether they understood the program content, whether they thought the BITB-HSC was a good program, what they learned in the program and how the program affected them, and whether they preferred a mixed versus separate gender program.
Data analysis
Descriptive statistics are provided for closed-ended questions in which participants provided ratings of the BITB-HSC. Students’ responses to open-ended questions were coded using qualitative content analysis (Krippendorff, 2004). Interrater reliability between coders was 96.84%. Coding discrepancies were discussed until agreement was reached.
School Personnel (Aim 3)
Participants
Participants were 161 high school personnel from the same treatment schools from which the student sample was drawn who completed the 2-month follow-up survey, reported exposure to some of the BITB-HSC intervention components, and answered at least one of the open-ended questions. The mean age of participants was 44.14 years (range = 22-69 years, SD = 11.74 years). The majority of participants identified as female (72.2%), White (98.1%), and were academic subject teachers (64.0%).
Procedure
A high school administrator sent out an email from the principal investigator describing the study and inviting all school personnel to participate in an online survey. Survey reminders (via email and in school personnel meetings) and fliers were additional recruitment methods. At the end of the online survey, participants had the opportunity to enter their name into a raffle for the chance to win one of two US $200 gift cards. In the current article, we use data from the second survey (i.e., 2-month follow-up) that followed the BITB-HSC components.
Measures
After assessing exposure to the different BITB-HSC programming components, we asked school personnel who observed the BITB-HSC programming components what they liked most and least, how likely they would be to use BITB-HSC (student curriculum, specifically) in their courses, and their reasoning for intending or not intending to use it.
Data analysis
The procedures used to analyze the school personnel data were identical to those we used to analyze the student data. Interrater reliability between coders was 94.01%.
Results
Student Data (Aims 1 and 2)
See Table 1 for a summary of study findings. Regarding adherence to the program model (Aim 1), on average, students (N = 970) attended six out of the seven BITB-HSC sessions (M = 6.4, SD = 1.5, range = 0-7). The majority of students attended all seven sessions (66.0%, n = 640), 9.3% attended six sessions (n = 90), 4.6% attended five sessions (n = 45), and 4.4% attended between one and four sessions (n = 43). Some students did not attend any of the sessions (2.4%, n = 22), and the remaining 13.3% of the sample did not answer this question (n = 129). Few students (9.2%, n = 89) reported that they gave their parent the RA and SA fact sheet that they received during the first session with the instructions to give the fact sheet to a parent or other trusted adult. The majority (53.3%, n = 517) of students did not give their parents the handout, and an additional 34.3% of students (n = 333) reported they could not remember whether they gave their parents the handout or not. The remaining 31 students (3.1%) did not respond to this question.
Summary of Key Findings.
Note. RA = relationship abuse; SA = sexual assault; BITB-HSC = Bringing in the Bystander–High School Curriculum.
Of the subsample of students who gave their parents a handout (n = 89), 84 students wrote in reasons for giving their parents the fact sheet. The reasons students gave for giving their parents the fact sheet included parents should be knowledgeable and prepared (30.6%, n = 26), they were told to by facilitators to do so (25.0%, n = 21), parents should know about the program that their student is doing in school (14.3%, n =12), and they thought the handout had good information in it (10.7%, n = 9). The remaining 19.4% (n =16) of the students did not answer the question (e.g., stated that they gave their parents the handout but did not say why). Of the subsample of students who did not give their parents the handout (n = 517), 446 students provided a reason they did not do so. The main reasons for not doing so included they forgot to (26.5%, n = 118), that they never got the handout (21.7%, n = 97; this number is higher than our fidelity records indicate), they lost it or it was destroyed (11.0%, n = 49), they thought their parents already knew the information so they would not need it (7.2%, n = 32), or that their parents would not care about it (5.4%, n = 24). Other reasons cited included thinking the sheet was not relevant to them (4.0%, n = 18), thinking that giving their parents the handout was not important (7.2%, n = 32), they did not want to (n = 13), they wanted to keep the handout for themselves (3.1%, n =14), they did not have an opportunity to give it to their parents (3.1%, n = 14), they felt awkward about giving the handout to their parents (1.6%, n = 7), they were too lazy (0.7%, n = 2) or too busy to do so (0.4%, n = 3), or they talked to their parents about the topic instead (1.1%, n = 5). The remaining 18 students’ responses (4.0%) did not answer the question (e.g., just stated that they did not give their parents the handout but did not say why).
Regarding students’ perceptions of the acceptability of the BITB-HSC (Aim 2), 91.1% (n = 884) of the students understood what the facilitators said during the sessions. The remaining 8.9% of the students reported that they did not understand the facilitators (n = 43) or did not answer the question (n = 43). The majority of students thought that the BITB-HSC was a good program (88.9%, n = 863), the remaining 11.1% of students either thought that it was not a good program (6.3%, n = 60) or did not answer the question (4.8%, n = 47). The majority of students also liked the presenters (87.7%, n = 851, 49 missing responses) and thought that the program was interesting (75.1%, n = 728, 56 missing responses).
Of the students (n = 880) who answered a question about the mixed gender delivery format of the BITB-HSC, the majority reported that they liked that the program was taught in a mixed gender setting (87.7%, n = 772), with only 12.3% (n = 108) reporting that they would have preferred sessions to be taught separately for girls and boys. Of the 772 students who liked the program being coed, 704 provided reasons. The main reasons students gave in support of mixed gender classes included that gender does not matter because the information should be the same for boys and girls (31.7%, n = 223), students need to hear “both sides” (32.9%, n = 232), students were comfortable with the mixed gender setup (12.9%, n = 91), and everyone needs to learn to work together (12.5%, n = 88). Other responses included concerns about promoting sexism or myths (3.0%, n = 21), concerns about gender-nonconforming students (1.1%, n = 8), ease of programming for facilitators/researchers (1.3%, n = 9), and thoughts about students being mature enough to learn about these topics in a coed environment (2.3%, n = 16). The remaining 2.3% of the responses were unclear. Of those who would have preferred the sessions to be conducted separately by gender (n = 108), 99 students provided reasons. The main reasons included it would be easier to share and relate to one another (53.5%, n = 53), the sessions would be less awkward (29.4%, n = 29), boys and girls think differently from each other (14.1%, n = 14), and students would feel less uncomfortable (3.0%, n = 3).
When students were asked to write what they liked most about the people teaching the program, 833 (94.7%) responded. The most common responses were the facilitator’s personality traits (44.1%, n = 367), the way the facilitators administered the program (e.g., knowledgeable, good at presenting material; 27.5%, n = 229), and the facilitator’s mannerisms (e.g., professionalism, how they spoke, 20.4%, n = 170). Students also liked the content (10.7%, n =89), and the fact that the facilitators were young and relatable (5.6%, n = 47).
In terms of what students reported liking the least, the main responses included there was nothing they disliked (24.5%, n = 115), the material was boring and/or repetitive (21.7%, n =102), and the program was too long (15.1%, n = 71). Some students did not like the content (8.3%, n = 39), thought facilitators were not good at presenting the material (9.0%, n = 42), or did not like the facilitator (9.2%, n = 43). A few students also disliked the time of day the program was taught (3.0%, n = 14) and the fact that they felt they needed to speak (1.5%, n = 7; although one of the ground rules of the BITB-HSC is voluntary participation). The remaining students only had negative general comments about the program (e.g., “it sucked”; 7.6%, n = 36).
Students (n = 840) reported one or more things they learned from the program, including how to intervene in situations of RA and SA (32.7%, n = 275), practical information about the topics of RA and SA (32.3%, n = 271), specific facts and figures (26.5%, n = 223), and to be more aware of what is going on around them (13.5%, n = 113).
Students (n = 779) suggested one or more improvements on the program including more activities in the sessions (22.8%, n = 178), including more examples (11.7%, n = 91), making the program less repetitive (10.1%, n = 79), changing the length of the program (10.5%, n = 82), changing the content in general (7.1%, n = 55), and changing the information in the sessions to be more/less advanced (4.1%, n = 32). Some participants (19.8%, n = 154) thought the program was fine as is, and 10.4% (n = 81) did not have suggestions on how to improve the program. The remaining 3.5% of students (n = 32) wanted a different facilitator (2.8%, n = 22), wanted better snacks (2.1%, n = 16), and wanted the program split by gender (1.2%, n = 9).
Finally, students were asked how the program would change the way they think and act. Of the 815 students who responded, 45.3% (n = 369) thought it would make them a more active and knowledgeable bystander, 24.3% (n = 198) thought it would make them more aware of situations involving RA and SA, 17.7% (n = 144) said they did not anticipate that the program would make them act differently, and 12.8% (n = 104) said it would change their behavior in general but did not mention specific examples.
School Personnel Data (Aim 3)
Regarding acceptability of BITB-HSC among school personnel, among the school personnel who attended the bystander training workshop and provided qualitative feedback (n = 124), responses were as follows: liked the content (37.1%, n = 46), the professional manner in which the content was delivered (25.8%, n = 32), the informative nature of the workshop (18.5%, n = 23), the relevance of the topic (7.3%, n = 9), and the empowering nature of the workshop (3.2%, n = 4). The remaining school personnel did not remember the workshop enough to comment on what they liked (4.0%, n = 5), did not know what they liked (1.6%, n = 2), or did not like anything about it (2.4%, n = 3).
One hundred and three school personnel commented on what they did not like about the workshop. Dislikes included the length and time of the workshop (16.5%, n = 17), the presentation style (20.4%, n = 21), the lack of instructional material provided (1.9%, n = 2), the repetitive nature of the workshop (1.9%, n = 2), and the perceived lack of relevance (4.9%, n = 5). In addition, 32.7% (n = 34) of the school personnel did not dislike anything, 6.7% (n = 7) did not remember the workshop well enough to comment, 3.9% (n = 4) did not know what they did not like, 4.9% (n = 5) had a general, negative reaction to the workshop but did not provide specific examples, and the rest of the school personnel had miscellaneous responses (5.8%, n = 6).
Of the school personnel who reported reading the handout and responded to the prompt “What did you like most about the handout?” (n = 54) responses indicated that the handout was informative (46.3%, n = 25), presented information in a clear and concise manner (27.8%, n = 15), and included up-to-date information (25.9%, n = 14). Forty-two school personnel responded about what they did not like about the handout, and many did not list any dislikes of the handout (52.4%, n = 22). Some school personnel did not remember specifics aspects of the handout (19.0%, n = 8), and 11.9% (n = 5) did not like the lack of visual data representation on the handout. Other dislikes of the handout included a lack of school-specific information (9.5%, n = 4) and some individuals wanted more resources and statistics listed on the handout (7.1%, n = 3).
Twenty-three school personnel observed at least one session of the BITB-HSC delivered by facilitators and provided qualitative feedback about what they liked about the program. School personnel liked the relevance of the program (43.5%, n = 10), the facilitators’ rapport with the students (34.8%, n = 8), and the interactive components of the session(s) (30.4%, n = 7). Dislikes of the program (n = 21 participants responded) included the variability in facilitators’ ability to deliver the program effectively (28.6%, n = 6), the lack of interactive components to the sessions (23.8%, n = 5), and the content (23.8%, n = 5). Specific dislikes of the content included lesson simplicity, lack of concrete examples on how to stop RA and SA, and too much lecture-based material. School personnel also disliked the length of the program (9.5%, n = 2). Three participants (14.3%) did not have any dislikes of the program.
We also asked school personnel who viewed BITB-HSC to write why they would or would not use/be likely to use the BITB-HSC in their classrooms. Of the 18 school personnel members who gave a response (58.1%), the majority (72.2%, n = 13) said that they would be likely to use the BITB-HSC in their classrooms. The main reasons cited for wanting to use the BITB-HSC included the relevance of the topic to their students (46.2%, n = 6), as well as the good information presented in the program (53.8%, n = 7). Reasons given for being unlikely to use the BITB-HSC (n = 5) included the inability to find the time to fit it into the regular curriculum (60.0%, n = 3) and a lack of relevance to the class (e.g., math) taught by the instructor (40.5%, n = 2).
Discussion
Overall, findings suggest that students were generally adherent to the BITB-HSC and that student and school personnel had positive perceptions of the BITB-HSC. Regarding students’ adherence to the BITB-HSC, the vast majority of students attended all sessions. Although we did not ask students reasons for not attending the BITB-HSC, it is presumable that students missed the sessions for reasons that students typically miss classes (e.g., absent, truant). Moreover, as documented by the researchers, a few students refused to attend the BITB-HSC classes, and some parents opted their students out of the sessions. Examining alternative ways to reach students who are not in school, refuse to participate, or do not have parental permission to participate in such programming are important areas for future work because these students may be at some of the greatest risk of experiencing RA and SA.
Given that the vast majority of students did not give their parents the fact sheet or remember whether they gave their parents the fact sheet, it seems important that alternative methods of engaging parents in prevention be explored. Work by Finkelhor, Vanderminden, Turner, Shattuck, and Hamby (2014) has shown enhanced prevention effects among youth where parents were also included in prevention efforts. Although simply reading a fact sheet likely would not lead to attitudinal and/or behavioral changes among adults, the fact that very few students thought it was important to engage in discussions with their parents about RA and SA prevention is concerning given the critical role that parents likely play in helping to prevent these types of issues. Also, adult involvement in multiple domains of students’ lives (e.g., school, home, neighborhood) is consistent with best practices of prevention; prevention programs should be comprehensive, in that they should address several domains that influence development and should bolster positive relationships with adults (Nation et al., 2003)
Moving beyond adherence, we found that overall, both student and school personnel’s perceptions of the BITB-HSC were positive. When asked about how the program would affect them, most participants were able to identify knowledge acquired and/or specific skills that they intended to use as a result of the program. This finding is consistent with meta-analyses such as Anderson and Whiston’s (2005), who found that level of facilitator training was a key variable for significant prevention program effects. More research is needed to better understand how perceptions of facilitators is linked to knowledge gain and behavior change. Facilitator training is an important and resource-intensive foundation for prevention work (Nation et al., 2003). An area for critical work is to better understand key characteristics of facilitators that affect program outcomes and implementation.
Interestingly, a number of participants felt that the BITB-HSC improved their ability to take positive bystander action in situations of RA and SA; however, as documented in our outcome evaluation data (Edwards, Banyard, et al., forthcoming), there was little impact of the BITB-HSC on actual bystander behavior. This lack of congruence may suggest that there were issues with the measurement of bystander behavior. Indeed, how to most accurately capture bystander behavior is a topic of critical inquiry (Palmer, McMahon, Rounsaville, & Ball, 2010). It is also possible that students’ perceptions of their own behavioral change is not entirely accurate, given some research suggesting that bystander intentions do not prospectively predict bystander behavior (Murphy Austin, Dardis, Wilson, Gidycz, & Berkowitz, 2016).
School personnel were also generally positive about the BITB-HSC, including the student curriculum, which outcome evaluation data suggest was effective in leading to positive changes among school personnel (Edwards, Sessarego, Mitchell, Chang, & Banyard, forthcoming). Given that teachers and school personnel are the individuals who often chose which prevention curricula are implemented in schools, understanding program acceptability is important. In the current study, we found that most school personnel liked the BITB-HSC content, although some school personnel had concerns. Although maintaining fidelity to a prevention’s core components and underlying theory is important, it is also important to develop prevention programming alongside school personnel and other community partners to ensure high acceptability of the content (Levine, 2018).
The current study offers other important implications for practice. For example, whereas most students reported liking the mixed gender format, some students wished that the groups were gender specific. This suggests that to the extent possible, offering option for students to attend programs that are mixed versus single gender could enhance program impact. Also, students delineated characteristics of facilitators that they liked the most (e.g., knowledgeable, professional, relatable), which can be used to enhance training of facilitators across a broad array of programming. Moreover, what both students and school personnel liked most and least about programming components (e.g., too repetitive, too long) can be used to refine and develop other types of prevention programming. Specifically, school-based prevention efforts need to work in collaboration with school districts to identify ways to better incorporate topics such as RA and SA prevention into the curriculum so that teachers experience the value of devoting time to it. In an era of increasing pressure for academic and test score results, school staff may have a hard time justifying using classroom time for nonacademic subjects. Furthermore, the low number of parents who read and used the worksheet suggests that other forms of information dissemination (e.g., texting, social media) may be useful.
Despite the contributions of the study to the field of implementation sciences, some limitations should be noted. Although our sample of students is quite large, the students were drawn from New England communities and do not represent the degree of racial/ethnic diversity needed to draw broad generalizations from our data. Along these lines, we would likely need to make adaptations for a more diverse group of youth for it to be deemed acceptable, given that youth’s experiences of sexual and dating relationships and violence differ among various ethnic and racial groups (Brown, Banyard, & Moynihan, 2014; Espinosa-Hernández & Lefkowitz, 2009). In addition, we had limited indicators of feasibility and acceptability. Nonetheless, the current study shows that in a sample of high schools, both school personnel and students were receptive to RA and SA prevention programming. School personnel were also interested in prevention materials directed at them specifically. Findings express the importance of engaging schools and school personnel as prevention partners.
Footnotes
Acknowledgements
We owe a great deal of gratitude to our school and community partners and the 50+ research assistants and program facilitators. Without these agencies and individuals, this project would not have been possible.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Funding for this study was provided by the Centers for Disease Control and Prevention (CDC), Grant #R01-CEO02524. The findings and implications presented in this article do not represent the official views of the CDC.