Abstract
This 3-year study evaluated the effectiveness of the Second Step–Student Success Through Prevention (SS-SSTP) social-emotional learning program on increasing prosocial behaviors that could serve as protective factors against peer conflict and bullying among students with disabilities. Participants included 123 students with disabilities across 12 schools in Midwest United States. Students labelled with a disability were selected for inclusion. Students completed self-report measures of school belonging, empathy, caring, and willingness to intervene in bullying situations. Report card grades and standardized test scores were collected from school records. Students with disabilities in the intervention schools reported a statistical and clinical significant increase in willingness to intervene in bullying incidents in comparison with students with disabilities in control schools and an increase of half a grade on their report cards in comparison with the control sample. The current study demonstrates the promise of social-emotional learning programming for students with disabilities.
Students with disabilities often have both academic and social challenges at school. For example, in a study of bully victimization among students with disabilities, using the Special Education Elementary Longitudinal Study and the National Longitudinal Transition Study-2, longitudinal data sets revealed a prevalence rate of 24.5% in elementary school, 34.1% in middle school, and 26.6% in high school (Blake, Lund, Zhou, Kwok, & Benz, 2012). Unfortunately, increased victimization among students with disabilities is often associated with prosocial skill deficits and peer rejection (Fox, 1989; Haager & Vaughn, 1995; Rose, Monda-Amaya, & Espelage, 2011). Therefore, it is imperative that prevention programs begin to address this victimization and promote prosocial attitudes and behaviors including caring, empathy, and willingness to intervene in bullying situations to reduce the number of students, including students with disabilities, who experience victimization.
Within the last 10 years, social-emotional learning (SEL) programs have shown success in reducing bullying experiences among students without disabilities (Durlak, Weissberg, Dymnicki, Taylor, & Schellinger, 2011) and, more recently, for students with disabilities (Espelage, Rose, & Polanin, 2015). Durlak and colleagues’ meta-analysis also demonstrated significant increases (i.e., 11 percentile higher) in standardized academic testing for students in schools with SEL programming compared with students in schools without SEL. It is not clear, however, whether these programs also foster gains in prosocial attitudes and behaviors, or result in improved academic outcomes for students with the most intense needs, including students with disabilities. Thus, the current study reports on results of a randomized clinical trial evaluation of a middle school program (Second Step–Student Success Through Prevention [SS-SSTP]; Committee for Children, 2008) in increasing school belonging, prosocial attitudes and behaviors (i.e., empathy, caring, willingness to intervene to help a victim of bullying), and academic achievement (i.e., report card grades, standardized test scores) among a small sample of students with disabilities.
School Belonging and Willingness to Intervene in Bullying Situations
School belonging is a critical issue for all students, including students with disabilities. Specifically, students with disabilities tend to have fewer close friendships, are often rejected by their peers without disabilities, and are regarded as unpopular (Baker & Donelly, 2001; Carter & Spencer, 2006; Kuhne & Wiener, 2000; Llewellyn, 2000; Martlew & Hodson, 1991; Morrison, Furlong, & Smith, 1994; Nabuzoka & Smith, 1993), which may lead to a lower sense of school belonging (Estell et al., 2009; Pavri & Luftig, 2000; Sánchez & Cerezo, 2010). School belongingness and positive peer relationships are directly linked (Knesting, Hokanson, & Waldron, 2008), and this link may influence students’ willingness to intervene in bullying situations. McMahon, Parnes, Keys, and Viola (2008) argued that students with disabilities who have fewer school-related stressors and greater access to school-related social resources report higher levels of school belonging, which predicts increased self-efficacy and school satisfaction. In addition, school belonging is a key factor in promoting positive academic outcomes over time, especially for students with disabilities (McMahon, Keys, Berardi, & Crouch, 2011).
Empathy and Caring
The ability to connect emotionally to others often requires strong social and communication skills, which can be challenging for some students with disabilities. Although disability identification is based on specific diagnostic criteria (Overton, 2009), some commonalities exist between various disability groups. For example, social and communication skills deficits are foundational criteria to autism spectrum disorders (American Psychiatric Association, 2013), which often manifest as a lack of empathy or caring (see Bons et al., 2013, for review). Similarly, students with learning disabilities, which represent the largest subpopulation of students with disabilities (Aud et al., 2012), often have lower social skills than their peers without disabilities (Kavale & Forness, 1996; Kavale & Mostert, 2004; Nowicki, 2003). In addition, students with emotional or behavioral disorders often have behaviors that manifest as an inability to develop and maintain peer relationships and a pervasive mood of unhappiness or depression (Individuals with Disabilities Education Act [IDEA], 2004), and are often identified as having low levels of empathy (Deschamps, Schutter, Kenemans, & Matthys, 2015). Given the relations among empathy, caring, and disability status, SEL programs represent a logical intervention to address these deficits among students with disabilities.
Academic Outcomes
The association between peer aggression and academic achievement is notable because current legislative efforts have almost exclusively focused on academic outcomes (Rose, Forber-Pratt, Espelage, & Aragon, 2013); yet, increasing evidence suggests that interventions should be multi-faceted to address the reciprocity between academic and behavioral outcomes. This is especially disconcerting for students with disabilities, as they have been identified with disabilities that are grounded in cognitive, behavioral, or functional deficits (Yell, Shriner, & Katsiyannis, 2006). More specifically, as a function of disability identification, students with disabilities, to varying degrees, have a discrepant educational experience from individuals without disabilities because they receive special education supports based on their individual needs (Rose et al., 2013; Yell et al., 2006). It is conceivable that the characteristics of certain disabilities may be associated with increased peer aggression and rejection (Rose & Espelage, 2012), as well as deficits in academic achievement. For example, in school year 2011 to 2012, 29.3% of sixth-, 25.6% of seventh-, and 25.4% of eighth-grade students with disabilities were proficient on state standardized exams for mathematics, and 29.6% of sixth-, 32.1% of seventh-, and 30.0% of eighth-grade students with disabilities were proficient on state standardized exams for reading (U.S. Department of Education, 2014). Therefore, interventions that address social and emotional learning may be especially beneficial for students with disabilities, as SEL programming can address deficits related to specific disabilities, as well as provide functional skill development.
Purpose
Given the disproportionate involvement of individuals with disabilities within the bullying dynamic, this study sought to evaluate the effectiveness of the SS-SSTP (Committee for Children, 2008) on increasing prosocial behaviors that could serve as protective factors against peer conflict and bullying among students with disabilities. The SS-SSTP program (Committee for Children, 2008) includes direct instruction in protective factors linked to academic and social success, including empathy training, emotion regulation, communication skills, and problem-solving strategies. Students learn about the importance of intervening to help others that are being victimized and practice the skills of intervening. Unfortunately, a lack of systematic evaluations of SEL programming exists in the disability literature, so this study directly addresses this gap by serving as a foundation for increased support for social and emotional learning for students with disabilities. Although SS-SSTP is considered a universal program, the basic tenets are germane to students with disabilities, as universal supports should be embedded across the entire student population, regardless of location of services. Furthermore, students with disabilities who receive a majority of their services (i.e., 80% or more of their instructional time) within the general education classroom have almost doubled over the past decade, where approximately 61% of youth identified with a disability aged 3 to 21 years are served primarily in inclusive classrooms (National Center for Education Statistics, 2015). Based on the extant literature, the following hypotheses were examined:
Method
Participants
Data for this study were taken from a larger randomized clinical trial (RCT) that included a total of 36 schools across Illinois and Kansas. The RCT included an evaluation of a middle school SEL program, Second Step© (Committee for Children, 2008), and included more than 3,300 sixth graders who participated across each of the 3 years of the trial (see Espelage, Low, Polanin, & Brown, 2013 for more information). Schools in Illinois and Kansas were matched in pairs on key demographics and then randomly assigned to the intervention condition or a control or business as usual condition.
Participants in the current study were 123 students with disabilities across 12 schools in two Midwest school districts (Table 1). Inclusion criteria for the current study was based on the student’s legal disability diagnosis as defined by the state’s eligibility criteria in reference to IDEA (2004). Therefore, the students included in the current study were receiving special education services as a function of their disability diagnoses and individualized education programs (IEP), which did not include students who received services under a 504 plan. In total, 47 students were included in the intervention condition while 76 students were included in the control condition. Across both conditions, 43% of the sample were female; 65% were 11 years of age and 35% were 12 years of age; 31% of the sample identified as White, 53% identified as African American, 6% Hispanic, and 10% as biracial. No significant differences were found between students in the intervention versus control conditions on demographic variables (Table 1).
Descriptive Statistics.
Note. Numbers represent column percentages and may not add to 100 due to rounding.
Intervention Condition: Second Step Curriculum
Second Step© (Committee for Children, 2008) is designed for sixth- through eighth graders and covers similar social-emotional targets (e.g., empathy, communication skills, problem-solving, bully prevention, friendship skills, harassment, emotion regulation, alcohol and drug prevention) across the grades, but the context increases in complexity from one grade level to the next. Fifteen lessons are delivered in sixth grade, and 13 lessons are delivered at the seventh- and eighth-grade levels. Each lesson is designed to take one 50-min session or two 25-min sessions. Prior to implementation, teachers in the intervention condition completed a 4-hr training that covered several areas. First, the research on bully prevention and social-emotional learning was reviewed to help the teachers understand the rationale for the project. Second, teachers were given the intervention kits, and the trainer took them through several lessons to demonstrate that they could be successful in implementing the program. Finally, teachers were given specific implementation strategies to maximize fidelity. For example, we encouraged them to prepare the lesson ahead of time, prepare the lesson as a team, and we provided tips for facilitating the lesson.
Control Condition: Stories of Us Curriculum
Control schools were provided with one copy of the P3R: Stories of Us—Bullying program (Faull, Jimerson, Swearer, & Espelage, 2008). P3R includes two scripted films that were created by a filmmaker working with two groups of middle school students for 5 weeks to cover what bullying looks like in middle school. One film took place in Nebraska, and one in Illinois. In each of the films, there is a “male” storyline and a “female” storyline. Viewers watch as a bullying incident starts and escalates between peers, how adults respond to the bullying (from the writer’s perspective), and two endings are provided for each story. The P3R files come with curricula materials with activities for youth to complete while watching the stories. These discussions are facilitated by a teacher or counselor. None of the control schools in the subsample analyzed here adopted the P3R curriculum.
Procedures
Parental consent and student assent
The university review board and the school district agreed to the use of a waiver of active (passive) parental consent. Parents received information about the study through several outlets, including electronic newsletters, presentations to parent-teacher associations, e-mail blasts from school administration, and family information nights. Information letters were sent to all parents of sixth graders in the participating districts. Parents could opt their child out of the study by calling the school or the researcher, or returning the signed parent form. Non-consented youth did not complete the survey and were removed from the room during the administration. However, non-consented youth did take part in the SEL program. Students provided assent to participate by signing the front page of the survey. An 86% participation rate was achieved in schools in the analyses reported here.
Survey administration
Teams of researchers administered the survey, and the team included advanced psychology graduate students, a faculty member, and several trained undergraduate research assistants. At the beginning of each administration, students were told about the purpose of the project and their rights related to participation. Accommodations were made that mirrored typical testing accommodations available on a student’s IEP, trained research assistants conducted the assessment in known self-contained classrooms, and concepts were explained when necessary. Of note, students who were deaf/hard of hearing were not included in the speech/language group. Students with disabilities took on average 1 hr to 1.5 hr to complete the survey. Surveys were administered across four waves: Fall 2010 (T1), Spring 2011 (T2), Spring 2012 (T3), and Spring 2013 (T4). T1 represented the baseline survey prior to implementation of the program.
Measures
The student survey was self-report and disability data were obtained from the school districts, where the diagnoses were based on the legally identified disability category in accordance with the Individuals With Disabilities Education Act (2004) and state regulations, and, therefore, was not assessed on the student surveys. Educational scores (i.e., state test scores, report card grades) were collected from school records. Variables of interest in this study including demographics (e.g., age, race, gender) as well as measures of school belonging, empathy, caring behaviors, and willingness to intervene in bullying situations, are described below.
School belonging
The Psychological Sense of School Membership (PSSM; Goodenow, 1993) was used to assess students’ sense of belonging or psychological membership in his or her school (i.e., the extent to which middle school students feel personally accepted, respected, included, and supported by others in the school). PSSM consists of 18 items (e.g., “Other students in this school take my opinions seriously,” “The teachers here respect me”). Participants responded to the items using a 5-point Likert-type scale (1 = not at all true to 5 = completely true), and higher scores reflect a stronger sense of school belonging. A Cronbach’s alpha coefficient of .66 was found for this study at Wave 1.
Empathy
A seven-item Empathic Concern (EC) scale assessed empathy and concern for others, and an example item is “When I see someone being taken advantage of, I feel kind of protective towards them” (Davis, 1983). Response options ranged from “does not describe me well” through “describes me very well.” Internal consistency coefficients have ranged from .71 through .77 for the scales and test-retest reliabilities of .62 through .71 have been reported (Davis, 1983). A Cronbach’s alpha coefficient of .72 for the EC scale at Wave 1 was found in this study.
Caring behaviors
The four-item Caring of Others (COO) scale (Crick, 1996) measures caring behaviors directed toward other students. Students are asked how often they engage in certain behaviors at school (e.g., Help out other kids when they need it). Response options are “never,” “almost never,” “sometimes,” “almost all the time,” and “all the time.” A confirmatory factor analysis supported the scale’s construct validity (Crick, 1996), and the scale’s Cronbach’s alpha was .89 in a similar middle school sample (Espelage, Mebane, & Adams, 2004). Cronbach’s alpha coefficient was .80 for this study at Wave 1.
Willingness to intervene in bullying episodes
The five-item University of Illinois Willingness to Intervene in Bullying Episodes scale was used in this study (Espelage, Green, & Polanin, 2012). Students are asked the extent to which they agree with statements about intervening directly or indirectly when they encounter bullying (e.g., “If a kid is being teased, I will stick up for him/her,” “I will tell an adult if a kid is being teased a lot”). Response options were “strongly disagree,” “disagree,” “agree,” and “strongly agree.” Cronbach’s alpha coefficient of .78 was found for Wave 1.
Grades and achievement data
Districts provided data on grades and state test scores. For grades, districts provided grades at Wave 1 and Wave 4 for science, math, and social studies, and these grades were averaged at each wave to calculate an average grade point average (GPA). Higher scores indicated higher GPA (0 = F, 4 = A as anchors). Furthermore, reading and math scores from the Illinois Standards Achievement Test (ISAT) were provided for each student at Waves 1 and 4. Scale scores were used and ranged from 120 (Academic Warning) through 270 (Exceeds Expectations).
Analysis
Missing data analysis
We used a multiple imputation procedure to avoid biases from missing data. Any student with a survey completed at Wave 1 was eligible for analysis. The imputation procedures were completed using SPSS version 21 (IBM, 2013), using the fully conditional specification Markov Chain Monte Carlo (MCMC) maximum likelihood procedure. Enders (2010) recommended the replication and use of 10 complete data sets. In addition, we followed an intent-to-treat design where students were analyzed by their condition assignment instead of treatment actually received (Little & Rubin, 1987). This procedure provides “practical utility” of the intervention (Little & Yau, 1996, p. 52) while allowing for the use of all individuals included in the intervention, so long as they are measured at Wave 1.
Statistical analysis
We estimated an analysis of covariance (ANCOVA) model for each outcome. Due to sample size restrictions, we were unable to fit the original, multilevel model, and instead fit a model that sought to assess intervention effects at each data collection point, controlling for pre-intervention differences. The model is represented as follows:
where Yi represented the outcome scale score for person i; b0 represented the intercept of person i; b1 to b8 represented control variables, where females and African Americans were the reference groups; b9 represented the relation between the pre-intervention scale score (at Wave 1) and the outcome; and b10 represented the variable of interest where the control condition was the reference category.
The goal of this model was to capture the relation between the condition variable and the outcome, controlling for pre-intervention differences in demographics and pre-intervention scale scores, to assess the impact estimate of the intervention. As such, we chose to present the adjusted means and standard deviations for each outcome at each wave, and omitted the regression results, which were of little substantive interest, but are available upon request. The standardized mean-difference effect sizes represent the treatment effect in magnitude of standard deviation units and corrected for small sample size bias (Hedges, 1981). To adjust for the clustering inherent within schools, we also adjusted the effect sizes’ standard errors using a correction suggested by Hedges (2007). We used a conservative intraclass correlation of .05, based on Hedges and Hedberg’s (2007) estimates as well as the empirical intraclass correlation derived from this sample. We hypothesized a positive effect size for school belonging, empathy, caring, willingness to intervene, GPA, and reading and math test scores. The models were calculated using SPSS version 21 (IBM, 2013), and the effect size estimates were calculated using R (R Core Team, 2012).
Results
ANCOVA models resulted in estimated marginal means for each time point (Table 2). We used these estimates to calculate the standardized mean-difference for each outcome of each wave of data collection (e.g., 2, 3, 4). All effect size estimates are available in Table 3, and below we discuss effect size estimates that are either statistically or clinically significant.
Means and Standard Deviations of Outcomes for Intervention and Control Conditions.
Note. Intervention: n = 47, Control: n = 76; W1 to W4 = Wave 1 to Wave 4. Numbers in parentheses are the standard deviations.
Effect Size Estimates of Each Outcome For Each Wave.
Note. Ns (Intervention: n = 47, Control: n = 76); numbers in parentheses represent 95% confidence interval; effect sizes adjusted for demographic variables (e.g., age, gender, race, mother’s education); standard errors adjusted for clustering (intraclass correlation = .05).
Wave 2 Analysis
The results of the analysis for Wave 2 effects indicated varied results. In terms of the outcomes we hypothesized would increase for the treatment group, willingness to intervene showed the largest increase (g = .32, 95% confidence interval [CI] [–.14, .77]), but was not statistically significant. This Hedges’ g of .32 represents a medium effect and indicates a .32 standard deviation increase in the intervention group’s willingness to intervene and is equivalent to 1.78 odds ratio (OR). This OR is higher than 19 of the 23 significant ORs for bully perpetration in the most comprehensive meta-analysis of anti-bullying interventions (Ttofi & Farrington, 2011). Thus, relying solely on significance does not measure the true effect of this program. The empathy outcome actually decreased for intervention students, but the decrease was not statistically significant. GPA and test scores were not available for this wave.
Wave 3 Analysis
The next analysis used data collected at the third data collection wave, after the students received an additional 13 intervention lessons. The results again revealed varied intervention effects. Of the outcomes we hypothesized to increase for the intervention students, willingness to intervene showed a statistical and clinical significant increase (g = .67, 95% CI [.21, 1.14]), representing a medium-large effect. Students’ empathy and caring also increased, but the gains were not statistically significant. School belonging decreased for intervention students. The students’ GPA and test scores were not available for this wave.
Wave 4 Analyses
The final set of analyses were conducted on the fourth wave of data collection after another 13 lessons were provided to intervention students. Students’ willingness to intervene was again positive, but for this wave, was not statistically significant (g = .13, 95% CI [–.32, .58]). Students’ self-reported school belongingness also increased relative to control students, but was also non-statistically significant. In addition, we were able to collect information with regard to the students’ GPAs as well as reading and math standardized test scores. The intervention students increased their GPA by a clinically significant margin, but it was not statistically significant (g = .41, 95% CI [–.05, .87]), but this effect was considered a medium effect. The results for the reading (g = .08, 95% CI [–.38, .53]) and math (g = –.09, 95% CI [–.54, .36]) test scores analyses, however, yielded no treatment effect. Therefore, the analysis of achievement records are promising yet inconclusive.
Discussion
SEL programs could serve as a vehicle for increasing prosocial skill development and academic outcomes, especially for students with disabilities. The current study examined the prosocial behaviors and academic achievement over time for students with disabilities who attended schools with an SEL program (Committee for Children, 2008) or students with disabilities in a control condition. After 2 years of SEL instruction, students with disabilities in the SS-SSTP schools reported greater willingness to intervene in bullying situations than students with disabilities in the control schools. This finding was both clinically and statistically significant, which is noteworthy given the small sample size of students with disabilities. This is very encouraging given that students with disabilities are disproportionately targets of bullying and rarely hold the social capital to stand up to youth who bully (Rose et al., 2011) because they are often rejected by their peers without disabilities and maintain fewer close peer relationships (Baker & Donelly, 2001; Carter & Spencer, 2006; Kuhne & Wiener, 2000; Llewellyn, 2000; Martlew & Hodson, 1991; Morrison et al., 1994; Nabuzoka & Smith, 1993). These gains in willingness to intervene in bullying situations were sustained and clinically increased after the third year of exposure to the SEL curriculum. These findings taken together with a previously published finding of a reduction in bullying perpetration among these same students with disabilities in the SS-SSTP schools (Espelage et al., 2015) provide strong support for using SEL curriculum to prevent bullying among students with disabilities.
Perhaps the most promising finding from this study is the increase in report card grades for students with disabilities in the SS-SSTP condition after 3 years of receiving SEL instruction. More specifically, the grades of these students increased by half a grade, from a solid C to an average of a B+. Like many SEL curricula, SS-SSTP targets self-regulated learning both directly and indirectly. As students are better able to control their feelings, thoughts, and actions, especially under emotional demands (i.e., cognitive complexity), academic learning is optimized through activities on problem solving, goal setting, and/or emotion management support (Durlak et al., 2011). Although these gains in academic achievement were not mirrored in the students’ standardized reading and math test scores, students with disabilities in both treatment conditions performed better at the final wave of data collection, which is important because a majority of middle school youth with disabilities are not proficient on their state’s standardized exam in reading or mathematics (U.S. Department of Education, 2014). It is noteworthy that the students in the SS-SSTP schools demonstrated better classroom performance than their peers in the control condition because behaviors, including behavioral skill deficits, and academic achievement are linked (DeRosier & Mercer, 2009).
Results associated with empathic concern and caring behaviors were less encouraging. Although some gains in empathic concern were noted after 2 years of the program for students in the SS-SSTP, at the end of the study, there were no notable differences between students in the two conditions. This could be due to the fact that the SS-SSTP curriculum focuses heavily on empathy in sixth and seventh grades, which is equivalent to the first 2 years of this study. Of note, all students increased in their empathic concern over the course of middle school, suggesting that this simply could be a result of maturity and experiences interacting with others. However, future research should assess both cognitive and affective empathy given the recent research that shows that as youth enter adolescence, increases are noted in cognitive empathy, but not affective or emotional empathy (Schwenck et al., 2014).
Although this study addresses a significant gap in the literature by focusing on students with disabilities in a randomized clinical trial, it does have several limitations. First, given the difficulty in obtaining school district disability data, our study sample of students with disabilities was relatively small in comparison with the 3,600 students in the larger clinical trial. In addition, specific disability groups (e.g., learning disability, autism spectrum disorders) could not be examined, which may produce different results based on disability characteristics. Second, because of grant budget constraints, only self-report data were collected, which is a notable limitation for individuals with disabilities because they may have cognitive deficits that limit item-level understanding. Given these limitations, future research should examine SEL programming, yet expand the sample size by specifically addressing disability categories and providing measurement modifications or collecting data from multiple informants.
In addition to the aforementioned future directions, coupled with the individualized needs of youth with disabilities, future research should also examine SEL programs in tandem with targeted social and communication skills programming. Specifically, low social and communication skills serve as predictors of increased bullying involvement of youth with disabilities (Rose et al., 2011), and while SEL programs address these skills at the universal level, some students may necessitate more individualized or direct instruction. Therefore, future research should examine a tiered approach to SEL by exploring the differential effects of a universal program versus a universal program coupled with targeted interventions for individuals with more intense needs.
Overall, students with disabilities tend to have lower social and communication skills than their peers without disabilities, which serve as two of the most notable predictors of victimizations for this population of students (Rose et al., 2011). Therefore, schools should begin to incorporate interventions that are designed to increase skill development. The current study demonstrates the promise of SEL programming for students with disabilities, especially for willingness to intervene in bullying situations and academic outcomes. While future research is needed to support these findings, this study serves as the foundation for increased longitudinal analyses for students with disabilities and social-emotional outcomes. Most importantly, this study demonstrates the importance for implementing targeted interventions for individuals with disabilities.
Footnotes
Authors’ Note
Opinions expressed herein do not necessarily reflect those of the Centers for Disease Control and Prevention, or related offices within.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Research for the current study was supported by the Centers for Disease Control and Prevention (#1U01/CE001677) to Dorothy Espelage (PI) at the University of Illinois at Urbana–Champaign, and by an Institute of Education Sciences Postdoctoral Fellowship grant to Vanderbilt University (R305B100016) supporting a portion of Joshua R. Polanin’s contribution.
