Abstract
Positive behavior interventions and supports (PBIS) improve student behavior. Yet, teachers may not receive adequate training to implement PBIS at the classroom level. This study evaluated class-wide function-related intervention teams (CW-FIT) as a classroom-level behavior management system to determine whether the behavior of middle school students would improve with teacher implementation of CW-FIT. A multiple-baseline across conditions design was used to evaluate changes in on-task behavior of adolescent students in sixth and seventh grade from a socioeconomically and ethnically diverse middle school. In addition, the effects on teacher behavior-specific praise statements and teacher reprimands were assessed. Consistent with previous evaluations of CW-FIT, findings indicated a functional relation between the intervention and increases in on-task student behavior. In addition, the findings also showed improvements to teacher behavior with increases in behavior-specific praise statements; however, no effect was observed with teacher reprimands. Social validity measures indicated students and teachers found the intervention favorable. Implications, limitations, and areas for future inquiry are discussed.
Effective classroom management practices are critical to addressing challenging classroom behavior (Reinke et al., 2013). Yet, classroom teachers often demonstrate on overreliance on punitive classroom management strategies (Rusby et al., 2011). The use of punitive classroom management systems has a negative impact on student outcomes such as increased disruptive behaviors, decreased academic performance, and increased school dropout (Mallett, 2016; Perry & Morris, 2018; Reinke et al., 2013). Furthermore, punitive strategies are disproportionately applied to students of color and students with disabilities (Losen et al., 2015). One possible explanation for overreliance on punitive strategies is teachers receive little training in proactive classroom management practices (Ducharme & Shecter, 2011). To address the gap in teacher training, it is important to identify strategies easily implemented by classroom teachers that also result in improved student responses in classroom environments (Oliver & Reschly, 2007).
Classroom-Level Positive Behavior Strategies
One way to improve teacher response to challenging behavior is implementation of classroom-level positive behavior interventions and supports (PBIS; Reinke et al., 2013; Sugai & Horner, 2006). PBIS involves systematic implementation of proactive behavior strategies (Sugai & Horner, 2006). When teachers use proactive classroom management strategies, improvements in on-task behavior have been observed (Cook et al., 2017; Faul et al., 2012; Floress et al., 2018; Jenkins et al., 2015; Kelshaw-Levering et al., 2000). Increasing on-task behavior is essential as high levels of on-task behavior are associated with favorable outcomes for learners including improved academic performance (Wood et al., 1998) and reduced levels of aggression and peer rejection (Leflot et al., 2013). A large body of evidence exists on proactive strategies to improve on-task classroom behavior.
One such strategy leading to improvements in classroom behavior is maintaining a high behavior-specific praise-to-reprimand ratio. Recently, Cook and colleagues (2017) evaluated a 5:1 praise-to-reprimand ratio in six elementary and middle school classrooms. Their findings demonstrated increased praise led to improvements in student behavior (i.e., lower levels of disruptive behavior, increased academic engagement). Moreover, teachers expressed satisfaction with the strategy. Yet, Floress and colleagues (2018) found teachers naturally demonstrated low rates of behavior-specific praise and would benefit from training to increase praise.
Another strategy shown to demonstrate a positive effect on student behavior is precorrection (Faul et al., 2012). Precorrects are delivered prior to an expected behavior and serve as prompts for the target behavior. Precorrects can include verbal reminders, demonstration, or practice of the target behaviors (Faul et al., 2012). Faul and colleagues (2012) investigated the impact of precorrects on off-task behavior of two middle school students. With implementation of precorrection, the off-task behavior of the students decreased (Faul et al., 2012). However, as suggested by the authors, the students may have responded favorably to the increased teacher attention given their learning history.
A group contingency is another strategy to integrate PBIS into classroom management systems. With implementation of a group contingency, one or more members of a group gain access to a reward contingent on display of target behaviors (Kelshaw-Levering et al., 2000). Group contingencies have led to improvement in disruptive behavior and academic engagement for students, particularly those in general education settings (Maggin et al., 2017). There are three types of group contingency (i.e., independent, dependent, and interdependent; Litow & Pumroy, 1975). In an independent group contingency, individuals within a group gain access to the reward contingent on individual performance (Litow & Pumroy, 1975). With dependent group contingencies, access to the reward for a group is contingent on the performance of one or more selected individuals within the group (Litow & Pumroy, 1975). For interdependent group contingencies, groups gain access to the reward contingent on the group’s collective behavior. Of the three types of group contingency, interdependent group contingencies may promote teamwork, improve social inclusion, and are a more efficient means of reward delivery (Kelshaw-Levering et al., 2000). With interdependent group contingencies students work together toward a common group goal and earn rewards for goal completion. Despite improvement in student responses, as noted by Maggin and colleagues (2017), the effects of group contingencies have been primarily measured with elementary and early middle school students in general education settings. Thus, further investigation of interdependent group contingencies in other contexts is necessary.
Class-Wide Function-Related Intervention Teams (CW-FIT)
Researchers have assessed the impact of an interdependent group contingency strategy, CW-FIT, on student and teacher behaviors (Kamps et al., 2015; Wills et al., 2016; Wills et al., 2018b). CW-FIT is a multitiered intervention encompassing many evidence-based classroom management strategies (e.g., identification and teaching of expectations, precorrection, active supervision of behaviors, immediate and consistent feedback, 4:1 praise-to-reprimand ratio, minimizing reward of problem behaviors, and an interdependent group contingency; Caldarella et al., 2018). Implementation of CW-FIT Tier 1 begins with explicit instruction of target social behaviors (e.g., following directions the first time, gaining teacher attention). Each teacher using CW-FIT displays posters explicitly describing target behaviors and a chart with team numbers, point criteria, and the reward. To teach the target behaviors, teachers discuss and provide a rationale for the behavior (Wills et al., 2018b). Students practice examples and nonexamples of the behavior with teacher feedback. Following instruction of target behaviors, the learners are placed into teams to earn points as part of a game.
At the beginning of each CW-FIT session, the teacher reminds students to demonstrate target behaviors to earn the reward (Wills et al., 2019). The teacher then sets a point goal and identifies the reward. Immediately following the precorrect, the teacher sets a timer to sound every 3 to 5 min (Wills et al., 2019). When the timer sounds, the teacher quickly scans the classroom and awards points on the point chart to groups demonstrating the target behaviors. The timer serves as a prompt for teachers to award points and assists the teachers in maintaining a 4:1 behavior-specific praise-to-reprimand ratio. At the end of the game, the teacher provides all teams who met the goal access to the reward (e.g., free time, edible time, technology time) as part of an interdependent group contingency (Wills et al., 2014). Teachers and students determine rewards collectively prior to initiating game play. Tier 2 of CW-FIT includes a self-management chart and help card, and Tier 3 involves functional behavior assessment for students not responding to Tier 1 or Tier 2. Neither Tier 2 nor Tier 3 were evaluated in the present study.
CW-FIT has demonstrated a positive impact on elementary student and teacher behavior (Caldarella et al., 2018; Kamps et al., 2011; Wills et al., 2018a). Kamps and colleagues (2011) assessed the intervention with 107 students in culturally diverse elementary classrooms and found increases in on-task behavior, decreases in off-task behavior, and increased teacher praise. In an additional study, Kamps and colleagues (2015) evaluated CW-FIT in 17 elementary schools and found improvement in on-task behavior and teacher-praise statements. Importantly, classroom teachers rated the intervention favorably (Kamps et al., 2015). In a recent replication, Wills et al. (2018a) evaluated CW-FIT with a randomized-control trial in 21 elementary schools and found improvement in on-task student behavior in treatment classrooms. The intervention also led to increased teacher praise and decreased reprimands and teachers indicated satisfaction with CW-FIT (Wills et al., 2018a).
More recently, researchers have studied the impact of CW-FIT in middle schools (Conklin et al., 2017; Orr et al., 2020; Wills et al., 2019). Conklin et al. (2017) evaluated CW-FIT in a kindergarten, second grade, and seventh grade classroom. During the intervention, on-task behavior increased, off-task behavior decreased, and teacher-praise statements increased. Orr and colleagues (2020) found similar results when implementing CW-FIT with middle school students with disabilities noting increased student on-task behavior and teacher praise. In another middle school evaluation, Wills and colleagues (2019) implemented CW-FIT in three classrooms with seventh and eighth grade students. All classrooms demonstrated increases in on-task behavior, increases in teacher praise, and decreases in reprimands. In addition, teachers implemented the intervention with high fidelity, and students and teachers indicated satisfaction with the intervention (Wills et al., 2019).
The purpose of the current study was to continue evaluation of the generalizability of CW-FIT. The effects of the Tier 1 components of the intervention were explored in an ethnically and linguistically diverse middle school. The research questions guiding the inquiry were (a) What are the effects of CW-FIT on adolescent student on-task behavior? (b) What are the effects of CW-FIT on teacher behavior-specific praise and reprimand statements? and (c) Do teachers and students find the goals, procedures and outcomes satisfactory?
Method
Setting and Participants
A multiple baseline across conditions study was conducted in an ethnically and linguistically diverse public middle school in the Southern U.S. serving Grades 6 to 7 during the 2017–2018 school year. Students identified as Asian/Pacific Islander (18%), Hispanic (46%), Caucasian (30%), and African American (3%). A majority of students were English language learners (54%) and eligible for free/reduced lunch (84%). The school did not have a PBIS system in place, but instead implemented a school-wide system of punitive behavior management. When students demonstrated problem behaviors (e.g., disruption, not prepared, off-task), they received a check mark. The checks accumulated and students received consequences, such as, lunch detention and suspension.
Upon receipt of institutional review board and school district approvals, the primary researcher contacted the school administrator to initiate recruitment. Classrooms were identified according to the following criteria: (a) teachers had not previously implemented group contingencies to manage classroom behavior, (b) the classroom population included one or more students from a minority ethnic background or one or more students with a disability who demonstrated low rates of on-task behavior, (c) parents consented to their child’s participation in the study, and (d) teachers volunteered to participate in the study. Data were collected on group and individual student behavior to verify less than 80% on-task behavior.
Three classrooms participated in the study. As depicted in Table 1, all teachers had lesser than 5 years experience in the classroom. The teacher in Classroom 1 was completing her third year in the classroom and had a bachelor’s degree in education. The teacher in Classroom 2 was in her fourth year of teaching and had a master’s degree in secondary education. The teacher in Classroom 3 was completing her fourth year and held a master’s degree in education. The sample included only those students in each class (i.e., 10 students in Classroom 1, 14 students in Classroom 2, and 11 students in Classroom 3) with signed consent as per district regulations. All three classrooms included students from diverse ethnic backgrounds and English language learners (Table 1). Two of the three classrooms included students with disabilities (e.g., autism, attention deficit disorder, specific learning disability).
Teacher Characteristics, Classroom Context, and Student Demographics.
Note. Total sample represents students in the classroom with consent. Demographic data limited to students with reports including such information. IEP = individualized education program.
Ethnic demographic data reported for total sample in Classrooms 1 and 2 and nine students in Classroom 3. bLinguistic data reported for seven students in Classroom 1, 12 students in Classroom 2, and nine students in Classroom 3. cSocioeconomic and IEP demographic data reported for seven students in Classroom 1, 10 students in Classroom 2, and eight students in Classroom 3.
Classroom Context
Across experimental phases, the context of each classroom differed (see Table 1). Two classrooms were cotaught learning settings in which two certified teachers (one general and one special education) managed and instructed the students. Instructional periods in each of the participating classrooms ranged from 45 to 90 min. Data were collected across diverse learning activities as part of the typical classroom routines, including, whole group instruction (e.g., teacher reading, presenting material to entire class), small group instruction (e.g., teacher working with small group while rest of class completed assigned tasks), independent reading or writing (e.g., students working individually to complete assigned tasks), and small group work (e.g., students working in small groups of two to five students on given task). All students in the class were seated at desks or tables arranged to create small groups of two to five students.
Dependent Variables
On-task behavior
On-task data were recorded with a 30-s fixed interval momentary time sampling. On-task behavior was defined as: (a) attending to the teacher (e.g., looking at the teacher, taking notes, attending to materials, or awaiting instruction), (b) completing assigned tasks (e.g., eyes on paper, participating in group discussion while attending to materials, raising hand for assistance, complying with instruction), and (c) responding to teacher instruction (e.g., taking notes, gathering materials, waiting quietly). At the end of each 30-s interval, observers scanned each group and indicated whether the group was demonstrating on-task “+” or off-task “−” behaviors. The interval was scored as “+” when all students in the group showed on-task behavior. If one or more students in a group was not on-task, the interval was scored as “−.”
Teacher praise and reprimand statements
Behavior-specific praise and reprimands were recorded throughout each observational session as a frequency of occurrence measure. Examples of behavior-specific praise included, “Good job following directions.” “Nice work gathering materials.” Nonexamples included, “Thank you.” “Great job.” Upon occurrence of behavior-specific praise, observers tallied such occurrence to render a frequency measure. Similar procedures were used to record reprimands (e.g., “Shhhh,” “Sit down,” “Stop talking”).
Procedural Fidelity
Two fidelity measures were used to promote procedural fidelity across all phases of implementation. A start-up procedural fidelity measure, completed by the researcher at the conclusion of the first intervention session, included eight components to teach the identified target behaviors (i.e., skills are prominently displayed, discussion surrounding the importance of showing the target behaviors, and opportunities for students to respond in the discussion of the target skill). The ongoing procedural fidelity measure included nine components rated on a scale of 1 to 3 (a) classroom expectation posted in view of all students (e.g., be respectful, follow directions the first time), (b) team point chart clearly displayed to all students, (c) daily point goal posted on point chart, (d) teachers precorrected use of target skills (e.g., “Remember, to earn points you have to follow directions the first time. What does it mean to follow directions the first time?”), (e) timer set to 3- to 5-min intervals throughout game play, (f) points awarded to teams for displaying target skills, (g) teachers maintaining a 4:1 praise-to-reprimand ratio, (h) praise and reprimands were specific to the behavior demonstrated or the target skill, (i) points tallied at the end of the game and reward delivered to teams meeting the daily point goal. Teachers completed the measure daily to self-evaluate their fidelity of implementation. In addition, the primary researcher completed the procedural fidelity checklist during each training session and weekly thereafter during treatment phases. The ratings rendered a total score out of 27. Teachers received corrective feedback on their implementation when three or more components on the fidelity tool were rated below a three. Fidelity data were used in the following ways: (a) provide feedback to teachers on implementation of the procedures during follow-up meetings immediately after game play (e.g., “Don’t forget students should be given access to the reward immediately after the game ends”) and (b) precorrect teacher behavior before game play (e.g., “Use behavior-specific praise today when awarding points to teams”).
Interobserver Agreement (IOA)
The primary researcher acted as the primary data collector across all phases of the study. Prior to coding study data, three secondary observers (a faculty member and two graduate students) were trained by the primary researcher to collect dependent variable data. IOA training included memorization of the definitions of dependent variables and coding of dependent variable data in live classroom sessions in baseline-like conditions. During training, the primary researcher and secondary observer simultaneously collected data in 10-min durations following the aforementioned dependent variable coding procedures. The primary researcher then calculated percentage of agreement. The procedure continued until observers met agreement criteria of 90%. Recording of IOA occurred in live session observations. To record IOA data for on-task behavior, the primary researcher discreetly named each group, “Group 1, Group2, Group 3,” at which point observers scanned the group and recorded. Praise and reprimand data were collected as a frequency measure upon occurrence throughout each session.
For on-task behavior, IOA was calculated by dividing the number of intervals both observers showed agreement by the total number of intervals to render a percentage of agreement (Jolstead et al., 2017). High levels of agreement for on-task behavior were shown across all phases of the study. In Classroom 1, IOA was collected 40% of baseline sessions and averaged 94% (range = 89%–99%). In Classroom 2, IOA was collected 30% of baseline sessions and averaged 92% (range = 90%–95%). In Classroom 3, IOA was collected 41% of baseline sessions and averaged 85% (range = 75%–93%). During treatment conditions in Classroom 1, IOA was collected 42% of sessions and averaged 93% (range = 85%–98%). During treatment in Classroom 2, IOA was collected 33% of sessions and averaged 94% (range = 89%–97%). During treatment in Classroom 3, IOA was collected 50% of sessions and averaged 98% (range = 95%–100%).
IOA data for frequency of praise and reprimand statements were calculated with gross agreement procedures by dividing the smaller number of statements recorded by one observer by the larger number of statements recorded by the second observer (Jolstead et al., 2017). In Classroom 1, IOA averaged 100% for praise statements and 88% for reprimands during baseline (range = 75%–100%). During treatment in Classroom 1, IOA averaged 73% for praise statements (range = 62%–94%) and 52% for reprimands (range = 0%–83%). In Classroom 2, IOA averaged 100% for praise statements and 55% for reprimands during baseline (range = 50%–60%). During treatment in Classroom 2, IOA averaged 95% for praise statements (range = 88%–100%) and 81% for reprimands (range = 50%–100%). In Classroom 3, IOA averaged 88% for praise statements (range = 50%–100%) and 56% for reprimands during baseline (range = 0%–100%). During treatment in Classroom 3, IOA averaged 95% for praise statements (range = 93%–100%) and 89% for reprimands (range = 67%–100%). Lower rates of agreement with praise and reprimand statements can be attributed to low frequency of occurrence and the missed recording of one instance of behavior.
Consumer Satisfaction
To evaluate teacher and student satisfaction with the classroom management strategy the primary investigator collected two measures of social validity immediately after the final treatment phase. Teachers completed a modified version of the Intervention Rating Profile–15 (IRP-15) with 15 statements by selecting one of six options: 1 = strongly disagree, 2 = disagree, 3 = slightly disagree, 4 = slightly agree, 5 = agree, 6 = strongly agree (Martens & Witt, 1982). Students completed a modified version of the Children Intervention Rating Profile (C-IRP) rating seven statements as agree or disagree (Kratochwill, 1985).
Research Design and Procedures
Research design
A multiple baseline across conditions single-subject design was used to evaluate experimental effect. Data were collected concurrently on the dependent variables (a) whole group on task, (b) teacher-praise statements, and (c) teacher reprimands. Baseline data were collected in all classrooms. The decision to introduce the independent variable was determined by a visual analysis of whole-group on-task data. When baseline data demonstrated stability, the training phase was initiated. During the training phase, the primary researcher provided immediate feedback on the implementation of the intervention. Once the classroom teacher was able to implement the intervention with at least 90% fidelity, the classroom moved into the treatment phase. During the treatment phase, the primary researcher only provided feedback on implementation if the fidelity dropped below 90%. The intervention was implemented in subsequent conditions with stabilization of the data in the previous condition.
Baseline
Baseline data were collected in the previously described classroom contexts. Teachers were instructed to implement their typical classroom management strategies. Such strategies consisted of general praise statements, reprimands, and awarding of checks associated with the school-wide punitive system. Baseline data were collected with a 30-s fixed interval momentary time-sampling procedure. At the end of each 30-s interval, the observer(s) would scan the room and record “+” indicating all students in the group were on-task or “–” indicating one or more students in the group were off-task. Baseline sessions continued in each class until stabilization of the data or a decreasing trend was observed. Stability was achieved when at least three consecutive data points fell within a 25% range of the median of the data.
Teacher training
Teachers were trained by the primary investigator to implement the procedures of CW-FIT across two 45-min training sessions or one 90-min training session. The training took place in each teacher’s classroom and consisted of the following: (a) lessons and teaching scripts, (b) team seating arrangement, (c) identifying daily point goals and awarding points, (d) use of behavior-specific praise statements and reprimands, and (e) identification of rewards and allowing students to access rewards at the end of the game. During the training, teachers viewed a PowerPoint detailing the components of the classroom management system, observed two videos of teachers implementing CW-FIT in their classrooms, and studied the two procedural fidelity checklists associated with initial and ongoing implementation. In addition, during the training session, teachers identified target behaviors for students in their respective classrooms. The three teachers participating in training and intervention phases selected “Follow Directions the First Time” and “Be Respectful” as target behaviors. Teachers were given posters to prominently display the target behaviors in their classrooms.
During the training phase of the study, teachers were given immediate feedback to improve implementation of the intervention. Feedback statements focused on increasing behavior-specific praise, awarding bonus points to target students, and reducing reprimands. The training phase continued until teachers achieved 90% fidelity of implementation. The teacher in Classroom 1 required two training sessions. Classroom 2 teacher met fidelity after one training session, and Classroom 3 teacher met fidelity after three training sessions.
Student training
Teachers taught students the target behaviors and the procedures of the CW-FIT strategy across two training sessions. During the teaching of the target skill, the poster displaying the skill was in a prominent location, and teachers used the scripts provided during the training. Only one skill was taught each day. To teach the skill, teachers described the behavior, shared the rationale for demonstrating the behavior, and provided opportunities for students to practice examples and nonexamples of behaviors. During this time students also identified highly preferred items to include on the classroom reinforcement menu.
CW-FIT implementation
After teaching the second skill, teachers initiated the CW-FIT management system in their classrooms. Each teacher chose to implement the intervention Monday to Friday. Teachers divided the students into groups of two to five and labeled each set of desks indicating each group’s team number, which corresponded to the daily point chart. Posters depicting the target behaviors were prominently displayed next to the daily point chart.
To initiate the intervention, teachers described CW-FIT as a game in which students had the opportunity to earn points for a reward. Teachers indicated students would earn points as teams and the points would be recorded on the point chart. Teachers shared that for teams to earn points each team member had to demonstrate the behaviors the class had been discussing and practicing. Teachers told the students each day a specific number of points would be required to gain access to the reward. The reward would be selected each day from the reinforcement menu by majority vote at the beginning of the game.
At the onset of each subsequent session, teachers reviewed the target behaviors displayed on the posters, determined the point goal for the day, reminded students to demonstrate the target behaviors to earn points, and identified the reward. Teachers set the timer to beep every 3 to 5 min throughout the instructional session. At the end of each 3- to 5-min interval, teachers scanned the classroom and recorded points for teams demonstrating target behaviors. While scanning the classroom, teachers praised groups and individual students for demonstrating target behaviors. For example, “Great job, Team One, Three, and Six, staying on task and working together.” If teams were not demonstrating target behaviors, the teacher would use reminders such as “Team two, remember to follow directions.” Throughout game play, teachers awarded bonus points to teams or individual students demonstrating target behaviors (e.g., “Nice job (student name) following directions the first time and reading quietly. Your team gets a bonus point.”). At the end of game play, teachers tallied points, identified the teams who met criteria, and immediately provided access to the reward (e.g., free time, computer time, outside time, or snacks).
Data Analysis
Visual analysis
Dependent variable data were graphed, and a visual analysis was conducted to assess experimental effect. The visual analysis included an assessment of trend, variability, immediacy, level, magnitude and percent of nonoverlapping data (PND). The treatment demonstrated effectiveness if there was an immediate change in level from baseline to treatment, a high PND, an increasing trend in on-task behavior and behavior-specific praise statements, and a decreasing trend in teacher reprimands (Ledford & Gast, 2018, Chapter 8). To calculate PND, the researcher identified the data range in the baseline conditions, counted the number of data points in the associated treatment conditions, counted the total number of data points outside of the range previously identified in the baseline condition, divided the number of data points outside of the baseline data point range by the total number of data points in the treatment condition, then multiplied this number by 100 (Ledford & Gast, 2018, Chapter 8).
Results
Whole Class On-Task
Figure 1 shows the percentage of 30-s intervals students demonstrated on-task behavior in the participating classrooms. Breaks in the data are attributed to teacher absences or schedule changes. On-task levels for Classroom 1 averaged 54% (range = 49%–60%) and demonstrated moderate stability during baseline. The teacher in Classroom 1 began implementing the intervention during the sixth observational session. The data demonstrated an immediate abrupt change in level increasing 36% after the implementation of the intervention during the training condition. During the intervention phase, on-task behavior averaged 84% (range = 66%–98%). PND was 100% indicated a strong effect. At baseline, Classroom 2 on-task averaged 48% (range = 36%–61%). On-task data showed a decreasing trend in Classroom 2 in baseline. The data demonstrated an immediate change in level after the implementation of the intervention, increasing 45%. On-task behavior throughout treatment for Classroom 2 averaged 88% (range = 71%–94%). On-task behavior showed an increasing trend during treatment, and the data demonstrated moderate stability the last 12 sessions of treatment (range = 87%–95%). Between baseline and treatment, PND was 100% indicating a strong effect. On-task behavior in Classroom 3 was variable during baseline and averaged 40% (range = 6%–58%). An immediate change in level was observed during training increasing 58%. Procedural fidelity criteria of 90% was met after three training sessions. In the treatment condition, on-task data did not stabilize but continued to show variability, averaging 79% (range = 53%–91%). PND was 83% between baseline and treatment suggesting a moderate effect.
Percentage of intervals with participating students demonstrating on-task behavior.
Teacher Praise and Reprimand Statements
Changes in teacher behavior are displayed in Figure 2. All three teachers showed increased levels of praise. In Classroom 1, baseline levels of praise statements averaged 1.2 (range = 0–4). Praise statements averaged 19.4 (range = 5–37) after implementation. For Classroom 1, PND was 95% suggesting a strong effect. In Classroom 2, praise statements did not occur at any time during baseline. An abrupt change in praise levels was observed during treatment increasing from 0 to 28. Praise continued to occur at higher frequency during intervention phase averaging 19.9 statements per session (range = 7–30). In Classroom 2, PND for praise statements was 100% indicating a strong effect. Classroom 3 baseline levels of praise averaged one praise statement per session (range = 0–8). Upon implementation of the intervention, an accelerating trend was observed, but the last three sessions showed a decrease in the frequency of praise. Treatment levels of praise statements demonstrated stability and averaged 22.7 (range = 16–30). In Classroom 3, PND was 100% for praise statements indicating a strong effect.
Frequency of teacher praise and reprimand statements.
Conversely, little effect was shown on teacher reprimands. In Classroom 1, reprimand statements demonstrated high frequency and variability during baseline averaging 28 (range = 11–43). Treatment levels showed a decrease in reprimands averaging 5.1 (range = 0–17). Percentage of nonoverlapping data was 95% suggesting a strong effect. In Classroom 2, reprimand statements averaged 9.4 (range = 0–30). After implementation of the intervention, the frequency of reprimand statements averaged 1.7 (range = 0–4) decreasing from baseline. The data remained stable throughout treatment; however, PND was 0% indicating no effect. In Classroom 3 reprimands averaged 5.6 (range = 0–12) during baseline and were slightly higher during treatment averaging 6.8 (range = 0–15). PND was 0% for reprimands in Classroom 3 indicating no effect.
Teacher and Student Consumer Satisfaction
At the conclusion of the study, teachers completed a modified version of the IRP-15 (Martens & Witt, 1982) by rating 15 statements on a 6-point Likert-type scale ranging from strongly disagree (1) to strongly agree (6). Higher ratings indicate stronger agreement with the statement. The statement showing the strongest disagreement rating was “I like the procedures used in the intervention,” (M = 3). The teacher in Classroom 3 selected disagree (2) for the statements, “I would be willing to use this intervention in the classroom setting,” and “The intervention was a fair way to handle children’s problem behaviors.” The statement with the strongest agreement was “The children’s problem behaviors are severe enough to warrant the use of this intervention,” (M = 5.7) Other statements showing strong agreement (M = 4.7) were, “This would be an acceptable intervention for children’s problem behavior,” “This intervention would not result in any negative side effects for children,” “This intervention would be appropriate for a variety of children,” and “Overall, this intervention would be beneficial for children.”
Students completed a modified version of the C-IRP (Kratochwill, 1985). The seven statements were rated anonymously on a dichotomous scale with agree or disagree. For the statement, “My teacher was too harsh on me,” 94% of students indicated disagree. To the statement, “CW-FIT may cause problems with my friends,” 85% of students indicated disagree. About 32 children (94%) agreed with the statement, “CW-FIT would be a good game to use with other kids.” About 29 children (85%) also agreed with the statements, “I like CW-FIT,” and “I think CW-FIT would help me do better in school.”
Discussion
Results of the study indicate CW-FIT improved adolescent student on-task behavior. Baseline data showed low levels of on-task behavior. Upon implementation of the intervention, immediacy and strength of change in student on-task behavior was evident in all classrooms. Of the three classrooms, Classroom 2 showed the strongest improvement. The findings of the current study are consistent with previous evaluations of the impact of CW-FIT (Conklin et al., 2017; Kamps et al., 2015; Wills et al., 2014, 2016, 2018b, 2019). Yet, these results extend previous findings in middle school contexts and provide further evidence CW-FIT may improve on-task behavior of adolescent learners.
In addition, implementation of the intervention led to increases in teacher praise. During baseline, the frequency of praise was low in all classrooms. With CW-FIT implementation, strength of change and immediacy of effect were shown in Classrooms 1 and 2. In Classroom 3, increase in praise was not observed until Session 3 of the training. At the conclusion of training Session 2, the primary investigator requested the primary classroom teacher implement all components of the intervention for the remainder of the study including behavior-specific praise and awarding of points. During the subsequent session, the primary classroom teacher implemented the intervention and an increase in praise was observed. With implementation of CW-FIT, teacher reprimands decreased in two of the three classrooms.
Finally, teachers and students indicated high levels of satisfaction. The statements showing stronger agreement suggest teachers were satisfied with the goals and the outcomes of the intervention. In addition, results indicated teachers found the intervention to be acceptable, appropriate, and beneficial for children. However, all teachers indicated the procedures of the intervention were not favorable. During implementation of the CW-FIT intervention, teachers reported they did not like the timer going off during class. It is also worth noting the teacher in Classroom 3 rated the intervention least favorably of the participants. Yet, student responses to the survey showed high levels of satisfaction with the intervention. The students agreed CW-FIT was fair, would be good to use with other kids, and would help them do better in school. The students’ responses also indicated the students liked the intervention.
Implications
Although implementation of positive behavior interventions results in improved outcomes for learners, researchers have underscored the importance of integrating PBIS systems into classroom-level management practices to yield the most favorable outcomes (Reinke et al., 2013). The current study presents an extension to the literature demonstrating improvements in adolescent behavior with implementation of CW-FIT as a classroom-level management strategy. It is important to note student behavior improved after a relatively short training session of 90 min and limited coaching and feedback from the primary researcher to support teacher implementation. Given teacher overreliance on punitive strategies which may be a result of inadequate preparation, it is necessary to identify systems of classroom management that are easily implemented by classroom teachers and improve behavior (Oliver & Reschly, 2007). Furthermore, the findings present important implications for selection and implementation of evidence-based behavior management strategies. Although behavior-specific praise has been established as an effective behavior management tool (Reinke et al., 2013), student behavior improved across the majority of sessions despite variability in teacher praise. This suggests other components of the CW-FIT intervention facilitated improvement in student behavior. The intervention also improved peer interactions as students were observed prompting and redirecting their group members to demonstrate target behaviors.
Despite increases in on-task behavior and consumer satisfaction, the teachers chose not to implement the intervention in the new school year. This outcome is not surprising as middle schools and high schools demonstrate a greater likelihood of abandonment of positive behavior interventions (McIntosh et al., 2018). Given strong support networks at the state or district level have been linked to reduced abandonment (Nese et al., 2016), it is critical building and district administrators as well as policymakers work to ensure systems are in place to support implementation of evidence-based practices at the classroom level. State or district systems should include ongoing training, coaching, and data-driven evaluation to support teams in implementation of positive behavior strategies (Nese et al., 2016). By developing systems to assess implementation and provide technical support, intervention sustainability of classroom level management practices may improve in middle or high school settings.
Limitations and Future Research
There are several limitations of the current study. First, this study included only three classrooms, thus limiting the generalizability of the findings. In addition, school district regulations required consent from parents or guardians for collection and report of all student data. Despite concerted effort to gain consent from parents (e.g., consent forms in multiple languages, researcher presence at parent-teacher conferences, reminders from teachers), 50% or less of each class returned consent forms. As such, whole group on-task data represent only a percentage of the total classroom population. In addition, the instructional context was not consistent across sessions and settings as the students participated in various activities daily. Such diversity could have affected on-task data. Furthermore, this study did not report praise and reprimand statements for special education teachers in Classrooms 1 and 3. Thus, it is impossible to determine the cotaught teachers’ impact on student behavior. Future studies should evaluate the impact of CW-FIT on the behavior of both teachers in cotaught settings.
Additional limitations are presented with the IOA recording methods. First, the total count procedure used to report praise and reprimand IOA presents a methodological weakness. This procedure does not ensure data collectors observe the same behavior, but rather demonstrate summary agreement on the number of behaviors observed. Furthermore, secondary observers were trained to collect IOA in baseline conditions only. In addition, observers were not blind to experimental conditions, which could have led to bias in data recording. Given such methodological weaknesses, caution is warranted in interpretation of the results.
The final limitations are related to teacher training, satisfaction, and sustainability of the intervention. Because teachers volunteered to participate in the study, they may have been more receptive to training and feedback possibly explaining quick acquisition of the management strategy. In addition, gathering data related to satisfaction ratings could have informed modifications to improve satisfaction with the intervention procedures. Yet, such data were not collected with this study. Future investigations should seek to identify the procedures within the intervention viewed less favorably by teachers. Finally, follow-up data to assess sustainability were not collected due to the end of the school year.
Conclusion
The results of this study extend current literature informing practices to support students and teachers in adolescent classroom settings demonstrating CW-FIT may lead to increases in on-task behavior for adolescent students and improve middle school teacher implementation of classroom-level positive behavior strategies. This replication provides additional evidence for the generalization of CW-FIT effects in middle school settings.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.