Abstract
Black learners, like all other learners, bring their cultural values into the classroom, how they express themselves, and how they problem-solve. In addition, their life experiences influence their social and emotional behaviors. This study examined the effects of a multicomponent intervention approach—including a culturally adapted social and emotional learning curriculum, check-in/check-out, and self-monitoring—on the externalizing behaviors of Black male learners in an urban elementary school. A multiple-baseline-across-classes, single-case, experimental design evaluated the effects of the intervention package. Results from the study show a consistent decrease in externalizing problem behaviors, increases in social and emotional competencies, and high levels of social validity for the intervention package based on the input of relevant stakeholders. An embedded alternating-treatment, single-case design explored the relative impact of self-monitoring. The authors also include a discussion of the limitations of this work and the implications for future research.
Keywords
Practitioners and researchers increasingly agree on the need to culturally adapt evidence-based social, emotional, and behavioral interventions to meet the needs of the wide range of diverse learners in school-based settings (Fallon et al., 2015). Cultural and contextual relevance consists of students’ and educators’ unique variables, characteristics, and learning histories (Fallon et al., 2015). Classrooms are not culturally neutral territories; therefore, it is important to use knowledge of the learners’ backgrounds to implement socially and culturally appropriate intervention procedures (Obiakor, 2008; Sugai et al., 2000). Particular concern exists regarding developing effective social, emotional, and behavioral intervention approaches for Black male students because this student population continues to have disproportionate overrepresentation in emotional and behavioral disorders (EBDs; Graves et al., 2017). As an illustration, Black students made up 13.79% of the total school-age population (5–21 years old) in the 2019–2020 school year, but represented 17.66% of all students served under the Individuals with Disabilities Education Act (IDEA) and 22.98% of the emotional disturbance category (Office of Special Education Programs [OSEP], 2021). With a risk ratio of 1.8, Black students ages 5 to 21 years were more likely to be identified with an EBD than students in any other racial/ethnic group (OSEP, 2021).
Students with EBDs experience higher rates of exclusionary discipline practices and receive special education services in more restrictive placements than any other disability group (Novak et al., 2020). Data from the Special Education Elementary Longitudinal Study revealed students with EBDs and attention-deficit/hyperactivity disorder (ADHD) as more likely to experience suspension or expulsion than students with learning disabilities alone (Achilles et al., 2007). More recent data have supported this conclusion by showing more than double the out-of-school suspensions for students with EBDs compared to the closest disability category (i.e., other health impairment, including ADHD) and four and a half times more suspensions than any other disability category (OSEP, 2022). Similar patterns exist across the data on in-school suspensions. Findings from the longitudinal data analysis by Achilles et al. (2007) also indicated a greater risk of exclusion for Black males with EBDs than for other student groups. National data have supported the findings of Achilles et al. (2007), showing that Black students with disabilities are more likely to experience disciplinary removals (OSEP, 2021). For every 100 Black students with disabilities, there were 64 discipline removals, while the average across all racial/ethnic groups was 29 discipline removals per 100 students with disabilities (OSEP, 2021).
Black students experience exclusionary discipline procedures, excessive disciplinary actions, and special education referrals, which may include restrictive placements (Betters-Bubon et al., 2016). The approach to discipline and support in schools, particularly for students of color, impacts the amount of time these students spend in the classroom (McCarter, 2017). When practices exclude students from learning environments, the effects are poor postschool outcomes (Rocque, 2010). Black students are more likely to drop out of school, have judicial records, and experience substance abuse, and they face a higher chance of going to prison than having a job (Rudd, 2014). These negative outcomes have been a persistent concern for Black male students in the United States for the past four decades (Losen & Gillespie, 2012; Zhang et al., 2014). Multitiered systems of support (Leverson et al., 2021) have the capacity to lessen the disproportionality and exclusionary discipline practices associated with the school-to-prison pipeline. Two converging movements focus on supporting social, emotional, and behavioral needs, and a comprehensive tiered model of prevention and supports can jointly apply them. The first is a focus on social and emotional learning (SEL), and the second is a commitment to interventions grounded in the tenants of behavioral theory.
Social and Emotional Learning (SEL)
The Collaborative for Academic, Social, and Emotional Learning (CA-SEL) defines SEL as the process through which all young people and adults acquire and apply the knowledge, skills, and attitudes, and skills to develop healthy identities, manage emotions and achieve personal and collective goals, feel and show empathy for others, establish and maintain supportive relationships, and make responsible and caring decisions. (CA-SEL, 2022, para. 1)
The organization’s SEL framework identifies five core competencies: self-awareness, self-management, social awareness, relationship skills, and responsive decision-making.
Strong Start (Carrizales-Engelmann et al., 2016; Merrell et al., 2008) is a collection of grade-banded SEL curricula based on the CA-SEL model with lessons designed to improve student behavioral outcomes by increasing their social and emotional competencies (Graves et al., 2017; Whitcomb & Parisi Damico, 2016). The curricula are not culturally responsive as written, and initial studies did not include cultural adaptations (Whitcomb & Parisi Damico, 2016). However, the developers provide recommendations for modifying lessons to reflect the interests, abilities, and culture of the participants while leaving the critical components in place (Carrizales-Engelmann et al., 2016). Suggestions include getting to know students by asking questions about their cultural activities and rituals, modifying the language to help students understand the key concepts better, and choosing literature with characters who reflect the cultural diversity of the students in the class.
A limited number of studies have assessed culturally adapted versions of the Strong Start program. Castro-Olivo (2014) evaluated the effectiveness of a culturally adapted version of the curriculum for middle school and high school Latinx English-language learners. Researchers adapted peripheral activities and literature and embedded cultural sensitivity into the teacher training. Findings showed students in the intervention group to report significantly higher levels of SEL knowledge and resilience. Social validity was high, with none of the items from the measure scoring below a 4.5 on a scale of 1 to 6. Graves et al. (2017) evaluated the effectiveness of a culturally adapted version of Strong Start on the social and emotional outcomes of 61 Black males determined to be at risk attending an urban elementary school. Adaptations aligned with Strong Start program recommendations and included modifications to the language (e.g., situations that matched the lives of the students, discussions on issues relevant to urban communities) and book selections (e.g., change in focus to literature with Black characters). Graves et al. (2017) found positive effects for self-regulation and self-competence, but no statistically significant changes in empathy, responsibility, or teacher-rated externalizing problem behavior via the Behavior Assessment System for Children, Second Edition (BASC-2; Reynolds & Kamphaus, 2004). Social validity scores indicated the need for more participatory, culture-specific research to determine the stakeholder view of critical components of SEL interventions and the need for more concentrated research with Black males.
Results from the few studies implementing culturally adapted versions of the Strong Start program have shown promise for increasing social and emotional competencies. However, there is little known about the impact of culturally adapted versions of Strong Start on externalizing problem behavior. Castro-Olivo (2014) cited the lack of an externalizing problem behavior measure as a limitation of their study. Graves et al. (2017) reported the Strong Start curriculum as having no significant effect on teacher-rated externalizing behaviors for students at risk, which is consistent with prior studies (Caldarella et al., 2009; Merrell et al., 2008; Neth et al., 2020) evaluating the impact of the Strong Start program as originally written (i.e., no cultural adaptations). At-risk students displaying challenging behavior may require additional intervention components, shown in the past to decrease externalizing behavior, alongside the Strong Start program to demonstrate effects across behavioral, social, and emotional domains.
Check-In/Check-Out
Check-in/check-out (CICO) is a Tier 2, evidenced-based intervention focused on improving positive behavior for at-risk students who consistently display externalizing behaviors that impede academic success (A. Campbell & Anderson, 2008; Maggin et al., 2015). The concept of the intervention is to prompt appropriate behavior before inappropriate actions and reactions occur (Crone et al., 2010). A. Campbell and Anderson (2011) noted several benefits of CICO: (a) instruction and frequent feedback on expected behavior, (b) more frequent contact with adults in the school, and (c) increased opportunities to reinforce expected behavior. The key components of CICO (A. Campbell & Anderson, 2008) are a morning check-in to review behavioral goals, an end-of-day check-out to review performance, preset times during the day for an adult to provide feedback on behavior, and acknowledgment of behavioral goals met.
Studies have shown that CICO has a positive effect on behavioral outcomes of Black male learners. Toms et al. (2018) evaluated the effectiveness of CICO used along with social skills training and academic planning for Black males identified with EBDs attending an urban high school. Researchers found the intervention combination to improve behavior and academic planning, as well as social validity to be positive for students and mentors. Miller et al. (2015) assessed the effectiveness of CICO for three Black male students using a fading approach that replaced teacher ratings and feedback on behavior with self-monitoring when participants displayed a stable or decreasing trend in problem behavior. With the introduction of CICO, all participants displayed an immediate decrease in problem behavior, and academic engagement improved. The problem behavior remained below baseline when researchers introduced the self-monitoring phase. Miller et al. (2015) asked students to use pencil and paper to record their behavior during the self-monitoring phase, but recently, web-based applications have supported self-monitoring using small devices such as tablets (Henry, 2017). I-Connect, a web-based behavioral self-monitoring application, allows students to discreetly monitor themselves. Pilot studies have shown that self-monitoring using I-Connect improves academic and behavioral outcomes for students with disabilities (e.g., Beckman et al., 2019; Clemons et al., 2016; Rosenbloom et al., 2016).
Purpose of the Study
The identification of Black males with EBDs is disproportionate, yet few studies have evaluated culturally responsive social, emotional, and behavioral curricula or interventions for this population of students. The limited number of studies with culturally adapted versions of Strong Start has shown promise for increasing SEL competencies. However, Graves et al. (2017) found their adapted version of the Strong Start curriculum to have no effects on teacher-rated (BASC-2) externalizing behavior scores with elementary Black male learners, which is consistent with previous studies (Caldarella et al., 2009; Merrell et al., 2008). The problem may require a more sensitive measure (i.e., direct observation); however, students with more intensive needs might need a more intensive intervention. In theory, an ideal intervention starts with explicitly teaching and practicing the replacement behavior(s), followed by individualized feedback and differential reinforcement. Possibly, the SEL curriculum failed to result in distal effects on externalizing behavior because while it explicitly taught and practiced skills, it lacked a formalized approach for individual feedback and differential reinforcement of behaviors aligned with the SEL competencies. CICO has shown positive effects in decreasing externalizing behavior through individualized feedback from school personnel and reinforcement of select skills. Furthermore, the addition of self-monitoring to CICO has shown promise for skill maintenance (Miller et al., 2015). It may be necessary, especially for students with and at risk for EBDs, to add interventions such as CICO and self-monitoring to SEL curricula to see improvements in proximal (i.e., SEL competency) and distal (i.e., externalizing behavior) outcomes.
The purpose of this study was to evaluate the effectiveness of a multicomponent intervention consisting of a culturally adapted SEL (CA-SEL) curriculum combined with CICO and self-monitoring on the social and emotional assets and externalizing behaviors of elementary Black male learners with or at risk for EBDs. During the school year before this study, a research team led by the first author implemented the Strong Start SEL curriculum by itself without cultural adaptations or modifications made based on student characteristics (A. R. Campbell, 2019). Results highlighted several nonresponders, and the social validity data strongly indicated a need to make cultural adaptations. The research team selected participants for this study from the list of nonresponders from the original related study. This study expands the currently limited research literature base related to CA-SEL curricula and interventions for Black learners. Furthermore, this study aimed to evaluate the hypothesis that at-risk students may need a more formalized feedback and reinforcement structure in their SEL curriculum by adding CICO and self-monitoring to the Strong Start program. The following questions were examined:
Method
The research team followed the procedures as written in the protocol approved by the institutional review board at the first author’s university and school leadership. Parents and guardians documented informed consent in writing, and students gave assent verbally.
Setting and Participants
This study took place at an urban public elementary school (Grades K–5) located in a large city in the south-central region of the United States. Records indicated that the Title I school located in a low-income community served approximately 735 students. The student body consisted of 99.9% minorities (68.5% Black, 30% Latinx, 0.1% Asian, 0.1% White, and 1.3% two or more races). All study procedures occurred in general education classroom settings during the 6-week 2019 summer session. The school district required students to attend summer school if they had not met educational benchmarks (i.e., reading, math, or attendance) from the school year or failed the statewide end-of-year assessment. Summer term structure mirrored the school year with all core and special subjects offered during a typical 7-hr day.
The research team identified participants for this study from a list of students who were not responsive in a separate study the previous year evaluating the effects of the Strong Start curriculum as originally written (i.e., no cultural adaptations or individualization based on student characteristics). A student qualified as a nonresponder if they had a lack of change in level or trend from baseline to intervention (e.g., a nonoverlap of less than 20%) and pre–posttest growth below the 20th percentile on the Social Emotional Assets and Resilience Scales Teacher (SEARS-T) form. Nonresponders from the study the previous year appeared on the roster in four summer school classes (Grades K, 1, 3, 4). The research team used Microsoft Excel to randomly select three nonresponder students from each grade level. In total, 12 Black male students participated in the study ranging from 6 to 11 years old. Grade level reported in this study matches their grade classification from the previous school year. All participants qualified for free and reduced-price lunch. Several participants failed to achieve math (n = 7, 58%), attendance (n = 4, 30%), and reading (n = 12, 100%) state benchmarks in the previous school year. The district had previously retained some participants as evidenced by age and grade-level comparisons (see demographic characteristics by student in Table S1 in the online supplemental material). Four teachers consented to participate (one from each grade level). Two teachers participated in the related SEL curriculum study the previous year. All teachers were Black females who had taught for at least 3 years. The teachers’ ages ranged from 29 to 52 years old; two had earned bachelor’s degrees, and two had earned master’s degrees.
Measures
Primary dependent variable
The primary dependent variable in this study was externalizing problem behavior. Using the research literature as a foundation (A. Campbell & Anderson, 2008), the research team defined externalizing behavior as including student behavior in the following four categories: (a) noncompliance: verbally or nonverbally refusing to follow directions given by an adult within 15 seconds of the request, (b) disruption: talking out (e.g., speaking without raising hand or speaking without adult prompt, speaking to peers) or other behavior that disrupts instruction (e.g., making noises or faces or banging objects), (c) out of seat: leaving the assigned work area without adult permission, and (d) negative verbal or physical interaction: any form of physical forceful touch or derogatory verbal statements toward adults or peers.
Data collection for externalizing behavior for each of the three students occurred daily for 20 min using partial interval recording (1-min intervals). The researcher sat in a position to see all three students. If a student engaged in externalizing behavior (i.e., out of seat, noncompliance, disruption, or negative verbal or physical interaction) at any point during the 1-min interval, the researcher marked the student as having externalizing behavior for that interval. Adding the number of intervals out of 20 in which the student displayed externalizing behavior determined the total score. This resulted in one data point per student for a total of three separate data points per class for each session. Partial interval recording systems may underestimate high-frequency behavior, especially as the intervals lengthen (Ledford & Gast, 2018; Maag, 2018). The team selected a partial interval recording approach with 1-min intervals for this study because of the duration of some behaviors in the identified categories and the demands of accurately collecting separate data for three students by one live observer.
The data collection procedures took place in the general education classroom during a 20-min segment of the reading block. The research team chose to concentrate data collection during reading for two reasons. First, all participants failed to meet state reading benchmarks the previous year. Second, administrators and teachers shared that discipline referrals were highest during the reading block. Study implementation during the reading block allowed evaluation of the impact of the intervention when student behavior showed a substantial need for improvement.
Interobserver agreement
The present study observed current guidelines (What Works Clearinghouse [WWC], 2020) recommending systematic measurement of the primary dependent variable by more than one independent observer in 20% of baseline and intervention phases for all participants. Participating teachers assisted researchers by recording (via camera) the 20-min data collection period during the reading block. The first author served as the primary in-person data collector, and two trained research assistants independently observed a percentage of the videos. Two observers (primary author and a research assistant) independently observed and collected data for 100% of baseline sessions for all classrooms with a mean of 96% agreement (range: 85%–100%) and 100% of intervention sessions for all classrooms with a mean of 97% agreement (range: 85%–100%). The team computed agreement by dividing the number of interval agreements (i.e., observers agreed on the presence or absence of externalizing behavior in a specific interval) by the total number of intervals and multiplying by 100 to get a percentage.
Social validity
A researcher-created survey with a Likert-type rating scale assessed teacher social validity. The teacher filled out one survey for each student participant in their class. Items addressed perceived student improvement and the efficacy and applicability of the CA-SEL/CICO program. The first author and a graduate student conducted semistructured social validity interviews with participating students following completion of the intervention. The social validity teacher survey and student interview protocol are available in the online supplemental material (Figures S1 and S2).
Social and emotional learning outcomes
This study used the SEARS-T (Merrell, 201l), a 41-item teacher-reported rating scale, to assess the distal impact of the multicomponent intervention (CA-SEL, CICO, self-monitoring) on overall SEL outcomes. This teacher report measures four constructs: (a) self-regulation, (b) social competence, (c) empathy, and (d) responsibility. Researchers added these raw score subscales for a total SEL score. Past samples during the development and validation process showed acceptable internal consistency estimates, with a Cronbach’s alpha for the total scale at 0.98 (Merrell et al., 2011). Teachers completed the form for all student participants preintervention and postintervention.
Experimental Design
A single-case, multiple-baseline-across-classes design (Ledford & Gast, 2018) examined the effects of a multicomponent intervention (CA-SEL, CICO, I-Connect) on externalizing problem behavior. Study protocol included concurrent baseline data collection across classes, and randomization before study onset determined intervention onset. Researchers chose randomization because prior data indicated high need and relatively stable data across all participants. Intervention onset for the following tiers occurred after a minimum of three sessions with a clear intercept gap or a decelerating trend. An embedded alternating-treatments design evaluated the relative benefit of the addition of I-Connect on externalizing problem behavior. Researchers randomly assigned treatment (Treatment 1: CA-SEL/CICO; Treatment 2: CA-SEL/CICO + I-Connect) order a priori, and the order remained consistent across classes.
Intervention Procedures
Teacher training and materials
The primary author trained the teachers 2 weeks before the start of summer school. Each teacher received a Strong Start K–2 manual or a Strong Kids Grades 3–5 manual, procedural fidelity checklists, audio recorder, CICO data collection materials, CICO daily report cards, incentives, and tablets. All teachers participated in the 3-hr Strong Start training, which involved an overview of the adapted curriculum and guidance on preparation. There was an additional hour of training for the CICO intervention. This training included video modeling the CICO steps, followed by role-play with performance feedback from the lead researcher. Teachers also reviewed the CICO fidelity checklist.
Baseline condition
Teachers and students engaged in school-wide behavioral initiatives, which did not include SEL lessons or curricula. The district employed a tiered behavioral support model (i.e., Positive Behavioral Interventions and Supports) and had a code of conduct outlining personnel duties and student responsibilities and consequences for misconduct. Student participants did not engage in a school-wide Tier 2 or Tier 3 intervention at any time during this study.
Intervention condition
The multicomponent intervention consisted of a CA-SEL curriculum with CICO and alternating-treatments adding I-Connect. Students received notification on which treatment (CA-SEL/CICO; CA-SEL/CICO + I-Connect) they would undergo at the beginning of each day.
CA-SEL curriculum
Prior to study onset, researchers adapted the K–5 Strong Start SEL curriculum (Whitcomb & Parisi Damico, 2016) for the Black-student participants to improve cultural and contextual fit. The curriculum for this study consisted of 10 highly structured 45-min lessons taught to the entire class. Teachers taught two to three lessons over the course of the week so that every day included a portion of an SEL lesson. The creators of Strong Start suggest that the scripts and individual lessons should mirror the academic level and culture of students involved to yield the highest level of participation and simplify implementation of the Strong Start program. The modifications to the Strong Start curriculum in this study align with creator suggestions and retain the core assumptions and skill domains of the program. Researchers also consulted suggestions by Brown et al. (2018) to (a) embrace learning about your students, (b) be mindful of communities where students reside, (c) use vocabulary that simply explains the key points, (d) provide a space that allows for differing viewpoints, and (e) seek responses on modifications made for cultural connections. In addition, researchers conducted an examination of cultural adaptations of Strong Start reported in previous studies (Castro-Olivo, 2014; Graves et al., 2017).
Feedback from the four teacher participants informed cultural adaptations to the Strong Start curriculum before the study. The teachers were all Black and had an appreciation for the histories of each student and the neighborhoods served by the school. Teachers gave feedback on script language and provided real-life situations that students may have experienced in their classroom, school, and neighborhood. For example, teachers focused on the nationwide tensions over the deaths of Black Americans in police custody, particularly at traffic stops. Specifically, the majority of students had heard of the unjust death of Sandra Bland or had heard of the hashtag “#Sayhername.” The research team used the teachers’ feedback to update the curriculum script to reflect the culture, academic level, and interests of the students (e.g., art, music, literature, dance, hairstyles, and food). Exercises focused on scenarios that participants had or may have experienced in and out of school. The team continued and expanded the adaptations by selecting read-aloud literature inclusive of Black main characters or race-neutral characters (e.g., animals, colors, and shapes) and supportive of the SEL competencies. Table S2 in the online supplemental material lists the alternative read-aloud texts.
Check-in/check-out
Participating teachers implemented CICO alongside CA-SEL. They applied the following steps to implement CICO in this study: (1) Morning check-in: Each morning the student met individually with their homeroom teacher. First, the teacher reviewed the prior day’s performance and then reminded the student of the expected behaviors. Following this, the teacher gave the student their CICO daily behavior form, which listed the expected behaviors that would be monitored that day. (2) Feedback throughout the day: At the beginning of each scheduled time on the CICO daily behavior form, which included five time slots, the student submitted their CICO daily behavior form to each classroom teacher. The teacher then completed the CICO daily behavior form when the period ended and provided feedback to the student, along with warranted praise. (3) Afternoon check-out: The student submitted their CICO daily behavior form to the homeroom teacher at the end of the school day. The teacher totaled and compared the earned points against the goal, provided praise and/or encouragement to the student, and gave the student a prize if they met their goal. (4) Home connection: Students took a copy of the CICO form home for parent review. There was no penalty for failing to return the signed copy to account for loss of the form during transportation and after-school care.
The primary adaptation made to CICO was to modify the goals or expectations to better align with the SEL competencies taught in the CA-SEL curriculum. The research team matched students to a CICO mentor of the same ethnicity (Black teacher/Black student). Based on the needs of the community and individual student needs, the team prompted teachers to ask students about their basic needs, food, shelter, and clean clothing, on a weekly basis. If the teacher identified needs, they provided items to support the student and family.
I-Connect
I-Connect is a self-management application that allows the teacher to program individualized self-monitoring goals, intervals, and prompts (e.g., screen flash, chime, and vibrate) for a student (University of Kansas, 2019). The students engaged with I-Connect on an iPad during every academic subject area and their lunch period on the identified treatment days. A unique feature of I-Connect is the ability to program up to five different prompts for each student. Researchers programed the prompts to align with the behaviors targeted in the CICO intervention (e.g., Are you keeping your hands to yourself? Is your voice louder than your friends right now? Are you on task?). At the end of each identified task or interval in this study, the device signaled self-monitoring questions that appeared on the screen to allow the student to assess the need for reinforcement. When prompts appeared, the student touched the device screen to select yes or no in response to each question. Once they selected the answer choice or if there was no response in 6 s, the next task/interval began.
Modified maintenance condition
Students entered the modified maintenance condition after a minimum of 8 or a maximum of 10 data points in intervention. During maintenance, students did not participate in CA-SEL or CICO. Students only used I-Connect to support skill maintenance. Researchers continued data collection on students’ externalizing behaviors daily during reading.
Implementation fidelity
The first author and a research assistant observed 100% of the CA-SEL lessons in person and jointly filled out an adapted version (i.e., changed to represent cultural adaptations) of the fidelity checklist included in the teacher’s Strong Start manual to evaluate adherence to the curriculum. The checklist consisted of four subcategories: pretest, activities, key terms, and closure. Only one of the CA-SEL lessons did not meet all the required criteria. This fourth-grade teacher did not finish the lesson and stopped before the activity (discussion) and closure.
Two members of the research team (first author and research assistant) jointly listened to 30% of the daily CICO sessions (check-ins and feedback throughout the day) for each participant to assess accurate implementation of CICO. They randomly selected audio recordings. See the CICO fidelity checklist in Figure S3 in the online supplemental material for the eight components. The research team calculated a daily fidelity score for each participant by dividing the number of components fully in place by the total number of components (8) and multiplying by 100 to get a percentage. Data demonstrate a mean of 99% fidelity across the selected sessions (range: 75%–100%).
Researchers assessed implementation fidelity for I-Connect on all the days assigned for this treatment condition during CA-SEL lessons and the data collection periods for externalizing behavior (i.e., reading). A brief checklist included ensuring (a) the student had turned on the iPad and opened to the application, (b) the iPad was in a standing position on the student’s desk so they could see the application flashing to notify them of a prompt, and (c) a place to note if the student did not engage in a prompt on the screen once it flashed. The I-Connect fidelity checklist data show that students responded to the iPad prompt 100% of the time during SEL lessons and reading, also indicating the iPad was always (a) open to the application and (b) standing in a visible position.
Visual Analysis
Researchers applied summative visual analysis to determine the presence or absence of a functional relation between CA-SEL and CICO with and without I-Connect (Q1). The research team visually inspected the data focusing on mean level change, trend, and variability within and between phases, as well as the immediacy of change at the intercept gap (Ledford & Gast, 2018). Baseline logic involves (a) prediction by determining whether the initial baseline is stable, indicating the likelihood the behavior would remain at the same level in the absence of an intervention, and (b) verification by examining subsequent tiers for similar stability. Finally, baseline logic includes (c) confirmation if and only if with intervention introduction, the data change in the predicted direction, and these data change replicates from baseline to intervention a minimum of three times (i.e., three demonstrations of effect). Researchers also used visual analysis to evaluate the differential effects between CA-SEL/CICO and CA-SEL/ CICO + I-Connect (Q2) by concentrating on stability, trend, spread, mean level differences, and any possible overlap. If there was a clear difference consistently across classes, the treatment with the lowest level indicated the most effective because the goal was to decrease behavior.
Results
Visual Analysis
Summative visual analysis, performed by the author team, determined whether there was a functional relation between the combination of two independent variables and a reduction in externalizing behaviors (Q1). Baseline levels were consistently high with little variability and trend across classes, with individual student scores ranging from 14 to 20 intervals with externalizing behavior (M = 18.5; M = 93%). The graphs shown in Figure 1 represent three demonstrations of a decrease in the number of intervals with externalizing behaviors upon introduction of CA-SEL/CICO and CA-SEL/CICO + I-Connect. All individual graphs show a consistent decelerating trend with minor variability during the intervention phase. No student showed overlap between their baseline and intervention phases. At least three demonstrations of effect indicate a functional relation (Maggin et al., 2013) between the combination of interventions and decreased externalizing behavior.
Student rate of externalizing problem behaviors.
The research team also used visual analysis to assess if there was a differential effect between the two independent variables on the reduction of problem behaviors (Q2). Data show a decelerating trend and externalizing behaviors occurring in 5 to 15 intervals for the CA-SEL/CICO intervention (M = 10.2, M = 51%). However, data from the CA-SEL/CICO + I-Connect intervention show a more rapid decelerating trend for each implementation, with the externalizing behaviors noticeably lower, ranging from 3 to 11 intervals (M = 6.3, M = 32%). A visual inspection clearly demonstrates a stronger effect for CA-SEL/CICO + I-Connect.
Fourth- and first-grade students entered a modified maintenance condition consisting of only I-Connect to support skill maintenance after the intervention condition. Summer school time restrictions (i.e., only lasting 6 weeks) and the multiple-baseline design structure precluded entering the other classes into maintenance. Both classes slightly decreased in level from the intervention phase and maintained low levels with little variability (M = 1.7; M = 9%).
Social Emotional Assets and Resilience Scales Teacher
Researchers examined SEARS-T scores to evaluate distal effects of the multicomponent intervention on the social and emotional competencies of participants (Q3). Measurement of SEL competencies occurred before baseline and after the intervention phase using the SEARS-T (Merrell, 2011). The results demonstrate growth for most students within each competency, with the highest growth scores in social competence, moderate growth scores in self-regulation and responsibility, and the lowest growth scores in empathy. Two students regressed with difference total scores of –17 (student 4) and –13 (student 12). See Table 1 for scores by participant.
Raw Score Differences Between SEARS-T Pretest and Posttest.
Note. Students are listed in the order the grade level entered the intervention phase. Difference total score represents the difference scores summed across the four subcategories. SEARS-T = Social Emotional Assets and Resilience Scales Teacher; K = kindergarten; Pre = pretest raw score; Post = posttest raw score; Diff = pretest raw score minus posttest raw score.
Social Validity
The research team solicited teacher and student responses on the social validity measures to gain knowledge on adapting SEL curriculums to grow the social and emotional competencies of Black male learners (Q4). All questions on the teacher social validity survey required teachers to respond on a 4-point Likert-type scale. Teacher participants agreed that externalizing behaviors improved because of the multicomponent intervention, with a mean score of 3.67 (range: 2–4). Similarly, teachers indicated growth in their students’ academic performance (M = 3.83; range: 3–4). Teacher ratings indicate that the CA-SEL/CICO + I-Connect intervention improved performance in the classroom (M = 3.92; range: 3–4). Ratings show that teachers would suggest CA-SEL/CICO + I-Connect intervention for students who display comparable social, emotional, and behavioral problems (M = 3.75; range: 3–4). Teacher participants were 100% in agreement that both interventions were practical and simple to utilize within a classroom setting (M = 4.0).
Student participants provided feedback via a semistructured interview about their intervention experience. All students noted that they felt at ease meeting with their CICO mentor (homeroom teacher). They noted feeling that someone cared about them, enjoying having their supplies for the day, and feeling comfortable expressing when something was wrong at home or asking for help with schoolwork. Furthermore, students reported that CICO helped them be excited about school and noted that their parents and/or grandparents were excited to hear about their day. Researchers surveyed students about whether they believe the CA-SEL/CICO + I-Connect intervention helped them to manage their behavior during class. They reported that the intervention improved their attitude and that they felt more focused and alert. Students mentioned looking forward to their CICO mentor telling them they were proud of them. They also reported that they were proud of themselves for keeping their word and enjoyed the prizes. Some additional comments included, “I like math now” and “I like having [a tablet] to help keep me on task.” All student participants indicated they would recommend (n = 1) or definitely recommend (n = 11) the CA-SEL/CICO + I-Connect intervention to other students.
Discussion
The purpose of this study was to evaluate the effects of using a culturally responsive multicomponent intervention (CA-SEL curriculum, CICO, self-monitoring) approach to reduce the externalizing problem behavior and increase the SEL competencies of Black male learners with or at risk for EBDs. Findings of this study show a reduction in externalizing behavior, an increase in social and emotional assets for the targeted learners, and strong social validity outcomes. Researchers culturally adapted the Strong Start SEL curriculum to reflect the cultural values, academic levels, and interests of the participating students, with adaptations based on the curriculum developers’ recommendations and a review of research literature. The team added CICO to the SEL curriculum to offer individualized adult feedback and differential reinforcement of prosocial behaviors aligned with SEL competencies, which they further supported by using a fading approach to self-monitoring (i.e., I-Connect) with technology. Results from this study are promising and suggest that leaders can adequately implement brief (<6 weeks), manualized, multicomponent treatments in underserved urban communities.
A reduction in externalizing behaviors is contrary to previous studies implementing the Strong Start curriculum alone with cultural adaptations (Graves et al., 2017) and without (Caldarella et al., 2009; Merrell et al., 2008; Neth et al., 2020). This signals promise for the addition of interventions such as CICO and self-monitoring to SEL curricula to improve behavioral outcomes with students with more intensive social, emotional, and behavioral needs. In a similar way, Toms et al. (2018) evaluated an intervention package consisting of social skills training and CICO with ninth-grade Black males identified with EBDs. Results demonstrated improved behavioral outcomes (i.e., daily report card scores) across all participants. The embedded alternating-treatments design in this study revealed lower levels of externalizing behavior and a more rapid decelerating trend for the treatment days including self-monitoring (i.e., I-Connect). A visual inspection of the data shows the greatest discrepancy between treatments with and without I-Connect at the fourth-grade level. The combination of teacher-led activities (e.g., monitoring, feedback, and reinforcement) and student monitoring increased the level of support, setting up a scaffolded approach where lower levels received higher levels of support in the maintenance condition. In a similar way, Miller et al. (2015) used a fading approach from teacher monitoring in CICO to self-monitoring in the maintenance condition and found that behavior maintained similar levels. Greater effects in the I-Connect condition may be the result of the increase in the number of prompts to engage in the prosocial behaviors from the iPad, as well as a heightened level of awareness of their behavior following regular prompts. It is also possible the novel technological aspect of using an iPad influenced student behavior. Future research should explore if adding a technological self-monitoring tool (i.e., I-Connect) without CICO to an SEL program results in similar social, emotional, and behavioral outcomes or if age or developmental level plays a role in the relative impact of the self-monitoring component.
Even though educational and behavioral literature often presents the adverse outcomes of Black males, hardly any preventive intervention research exists to specifically support this student population (Noonan et al., 2016) and offer resources to the teachers who serve them. Hence, there is a powerful justification for research on proactive, universal social, emotional, and behavioral supports (i.e., SEL curricula) and targeted interventions (i.e., CICO; self-monitoring) designed for and evaluated with culturally diverse learners. This is the second study to show positive results with a culturally adapted version of the Strong Start series in an urban area with a solely Black male sample (Graves et al., 2017), with stronger outcomes for externalizing behavior in this study potentially caused by the embedded interventions. Commercially available SEL curricula, such as the Strong Start series, often fail to explicitly address culture within the scripted lessons and instead offer somewhat vague guidelines for educators on how to make the cultural adaptations themselves. From a marketing and sales standpoint, a general approach ensures a wide target audience. However, the problem is that research shows educators often do not have the knowledge of culturally responsive pedagogy (Samuels, 2018) or the shared background experiences (i.e., mostly White, middle-class, female teachers) to make the appropriate adaptations to curricula or interventions. The teacher participants (Black females) provided guidance during the cultural adaptation process in this study. It is possible that this guidance and their understanding of their students’ backgrounds positively influenced the study in a way that White-teacher participants could not replicate, given the same student population (i.e., Black students). Additional research should explore the replicability of the current adaptation protocol across settings and implementors to guide the development of accessible curricula and interventions for culturally diverse learners.
Limitations
Audiences should view results with an understanding of the limitations of the study. First, as this study features a multicomponent intervention, the research team is unable to parse out the relative contribution of the CA-SEL curriculum or CICO in rates of externalizing behavior or in any of the three components in the SEARS-T scores. However, the embedded alternating-treatments design suggests that self-monitoring via technology (i.e., I-Connect) may provide an additive effect in decreasing externalizing behaviors and shows promise for skill maintenance. Second, this study only examined externalizing behaviors; however, research has shown that SEL interventions decrease internalizing behaviors (e.g., withdrawal, difficulty concentrating). Internalizing behaviors are not typically disruptive; nevertheless, they can negatively affect academic performance and students’ well-being. Researchers should explore these internalizing behaviors in the future. Third, formal data collection on fidelity for I-Connect only occurred during the reading and SEL lessons, even though the students used the application all day. It is possible that high fidelity did not continue across all subjects. However, the research team informally looked over the automated reports from the application to ensure regular engagement. Finally, it is important to note the limitation of conclusions solely based on the SEARS-T, given that the current single-case research design does not rule out other confounds in this pretest to posttest comparison.
Implications
Many studies have reported the poor outcomes associated with Black males in our public school systems; however, the number of interventions designed to address this issue has not followed suit. In addition, studies evaluating interventions for children with social, emotional, and behavioral problems often include large samples of Black children; however, the results do not usually disaggregate by race or ethnicity (Serpell et al., 2009). Therefore, persistent and focused intervention research with Black males with a culturally responsive focus should be a priority. Furthermore, adaptations to SEL curricula and the development of social, emotional, and behavioral interventions should have their basis in input from a variety of diverse stakeholders (i.e., students, teachers, parents, and community members) representative of the target population.
Despite the growth in research about evidence-based interventions in behavioral and medical fields, treatment and intervention translation to communities of color has been poor (Whaley & Davis, 2007). Because of the overrepresentation of Black students in the special education category of emotional disturbance, intervention studies that concentrate particularly on targeted groups who are at risk for poor outcomes (e.g., Black males) should be of concern for both state and local education agencies. In practice, this study demonstrates that relatively simple cultural adaptations to existing programs can result in positive outcomes and high levels of social validity for relevant stakeholders. Equity is imperative across teachers’ practices and procedures. School districts and state education leaders will need to find ways to provide their teachers with the resources and support to make sure content and materials are representative of their students.
Conclusion
While practitioners and researchers have espoused the need to implement culturally adapted, evidenced-based interventions to address the diverse needs of culturally and linguistically diverse learners, few studies have sought to examine the effectiveness of a culturally relevant intervention alone or as part of a culturally responsive program. It is important to conduct research on culturally relevant approaches to SEL and behavioral support, particularly for Black males. This population has a disproportionate identification of EBDs and exposure to risk factors that negatively affect their social and emotional well-being. An EBD affects a student’s academic performance and relationships with peers and adults. This study shows that a culturally relevant SEL curriculum as part of an intervention package with CICO and self-monitoring is effective in growing SEL competencies and decreasing externalizing behaviors. Teachers found the multicomponent intervention simple to implement and found self-monitoring to allow for much less redirection and cut down on loss of instructional time. Participating students reported that this intervention package was both helpful and fun.
Supplemental Material
sj-docx-1-rse-10.1177_07419325221143965 – Supplemental material for Culturally Adapted Social, Emotional, and Behavioral Support for Black Male Learners
Supplemental material, sj-docx-1-rse-10.1177_07419325221143965 for Culturally Adapted Social, Emotional, and Behavioral Support for Black Male Learners by Aaron Rachelle Campbell, Mary Rose Sallese, Julie L. Thompson, Constance J. Fournier and Meghan Allen in Remedial and Special Education
Supplemental Material
sj-docx-2-rse-10.1177_07419325221143965 – Supplemental material for Culturally Adapted Social, Emotional, and Behavioral Support for Black Male Learners
Supplemental material, sj-docx-2-rse-10.1177_07419325221143965 for Culturally Adapted Social, Emotional, and Behavioral Support for Black Male Learners by Aaron Rachelle Campbell, Mary Rose Sallese, Julie L. Thompson, Constance J. Fournier and Meghan Allen in Remedial and Special Education
Supplemental Material
sj-docx-3-rse-10.1177_07419325221143965 – Supplemental material for Culturally Adapted Social, Emotional, and Behavioral Support for Black Male Learners
Supplemental material, sj-docx-3-rse-10.1177_07419325221143965 for Culturally Adapted Social, Emotional, and Behavioral Support for Black Male Learners by Aaron Rachelle Campbell, Mary Rose Sallese, Julie L. Thompson, Constance J. Fournier and Meghan Allen in Remedial and Special Education
Supplemental Material
sj-docx-4-rse-10.1177_07419325221143965 – Supplemental material for Culturally Adapted Social, Emotional, and Behavioral Support for Black Male Learners
Supplemental material, sj-docx-4-rse-10.1177_07419325221143965 for Culturally Adapted Social, Emotional, and Behavioral Support for Black Male Learners by Aaron Rachelle Campbell, Mary Rose Sallese, Julie L. Thompson, Constance J. Fournier and Meghan Allen in Remedial and Special Education
Supplemental Material
sj-docx-5-rse-10.1177_07419325221143965 – Supplemental material for Culturally Adapted Social, Emotional, and Behavioral Support for Black Male Learners
Supplemental material, sj-docx-5-rse-10.1177_07419325221143965 for Culturally Adapted Social, Emotional, and Behavioral Support for Black Male Learners by Aaron Rachelle Campbell, Mary Rose Sallese, Julie L. Thompson, Constance J. Fournier and Meghan Allen in Remedial and Special Education
Footnotes
Authors’ Note
This manuscript is based on a portion of a dissertation completed by Aaron Rachelle Campbell. The dissertation document is presently under an embargo with Texas A&M University, and it will not be published in their online repository for 3 years.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
